For me it's not the outfit itself that doesn't work - it's the color combinations.
I think with a plain shirt and a vest that matched/blended with the saffron kilt oe brown shoes, it could work.

Looking at the pictures again (before I hit 'Submit') - the problem (at least to me) is that your upper and lower halves aren't 'tied' together. The shirt, vest and hat all work together; and the kilt, hose and garters all work together; but the two halves don't have any common colors to join them.