Andrew,

It seems to me as one moves up the fashion "ladder" (work clothes, casual, business casual, dressy, semi-formal, and formal) that the rules and restrictions for men become more and more rigid. Formal wear for men abides by certain rules. Do you need to follow them? No. But be aware you are breaking them. In one sense as a kilt wearer you are in luck. Brown accessories with a tuxedo would be rseen as completely out of place by the majority. Few people are in the know for kilted fashion and will not recognize your fashion faux pas. There are the rules. If you want to be daring, feel free to break them. But you are heading into territory without precedent here.

Cheers