How far should you take this? To the committee. Since there's already a process in motion here, if you back out now you will be admitting that wearing the kilt was wrong to begin with.

But you must not go further. If the committee gives thumbs-down, abide by that decision for your professional health. If you continue wearing the kilt after that, it WILL be defiance. If you enlist the ACLU or some legal body in this, you may win the right to wear the kilt to work, but you will alienate the entire management structure forever.

I think you've got a good case to take to committee. If the woman supervisor/detractor happens to wear pants to the meeting, that's a bonus: you can cite what Mary Tyler Moore did for women's choices in workplace attire. Stress CHOICE, not fashion. And mention, casually, that of COURSE you wear underwear with the kilt; that's likely a big issue, and the question is so uncomfortable that they may not even ask it.