I think the Curmudgeon (aka Steve from Freedom Kilts) has it down pretty well. It's the overall style that either does or does not elicit a comment.

That said, I've never gotten the "Bagpipes" question when wearing my old solid black Stillwater kilt, or my camouflage X-Kilt!