It has happened to me. I usually just give them one of those eyebrows raised "What are you looking at?" looks and walk on out of the restroom. If you're feeling really mean, nervous glances at them while acting like you're holding your kilt down tends to put them in their place. (Talk about sudden looks of embarrassment and shame!) Either way tends to work.