I have always taken the view that the kilt is a very versatile garment, and that is what makes it so useful. You can wear it fairly casually during the day and then dress the same one up with a formal jacket for the evening. You couldn't do that with a dinner suit/tuxedo could you? If you like the tartan go ahead and wear it as you like.