I just say, "Boys wear pants. Men wear kilts." On occasion I've even said to women who have said it's a skirt, "Yes...and it's prettier than yours."

That tends to get a laugh....usually if they keep insisting it's a skirt it's because they are 1) drunk or 2) think they'll just get under your skin by calling it a skirt all the time. Once you show that you don't care, they tend to stop.