It's definitely a culture thing and doing what people expect. I suppose wearing anything other than a tartan kilt would be looked on as strange in Scotland too. People have very conservative ideas about how men should dress and look generally and anything out of the ordinary is criticised. Doesn't seem to happen for women no matter what weird and wonderful things they wear.