I usually reply "none" (nice 1 word answer) to which they usually look puzzled. The typical follow up conversation goes like this:
"Are you a bagpiper then"?
"No."
[insert 5 excrutiating seconds of silence and dumbfounded looks].
"Then are you Scottish/Irish?"
"No. I'm actually German"
"Do they wear QUILTS in Germany?"
[somewhere in the conversation they'll call it a quilt]
"some guys might, but not on the whole"
"Well then why are you wearing one?"
"I wanted to".
"ummmm.... ok".

At this point, I've usually confused a complete stranger and they are looking a bit 'perturbed' since I didn't fit into 1 of 2 mental categories they have for guys wearing kilts... bagpiper or formal occasion for a Scotsman. At this point, Kelly usually steps in to ruin my fun...

"We own a Kilt company and he wears one every day" - Kelly
"OHHHHH... NOW it makes sense" - confused stranger
"Wearing a kilt didn't make sense before you knew I owned a kilt company?" - Rocky