It seems to me that if someone tells you he has a kilt (though he isn't wearing it at the moment), he's trying to show his support for you in your kilt. Maybe it never occurred to him, before encountering you, that someone could wear a kilt for other than formal occasions -- you may have planted a little seed that will sprout a new kilt-wearer! So please don't mow it down right away!