I'll go with the majority and say leave it out. Just looks better to my eyes.

What really looks tacky IMHO, is when someone wears one with low slung trousers, and there is a band of shirt between the bottom of the sweater and the belt of the trousers! Of course a kilt is worn higher up (in most cases). So that would not be a problem.