What a great thread. JerseyLawyer and An t-Ileach have it right. It may be a slight excess of zeal that keeps people from wearing a tartan they cannot prove was worn by their family. The Clan tartan system is a recent fabrication, and specific tartans were not historically assigned to specific clans. The way clansmen identified each other in battle, if not by face recognition, was by plant badge or other insignia worn in the cap or headgear, not the particular tartan they wore.

Historically tartans were not assigned to families until the 19th Century, as part of Victorian fashion. Matt Newsome discusses this on his website, as he and others have pointed out, prior to the 19th Century it is impossible to establish that particular clan chiefs for whom a portrait records exists wore a specific tartan--where a portrait record exists, they show up wearing different tartans over the years. J. Charles Thompson has a concise discussion in his book So You're Going to Wear the Kilt, p. 18-26. Barb Tewksbury has a nice summary at pgs. 11-15 of her The Art of Kiltmaking.

As Tewksbury relates, many tartans were assigned to clans by the commercial weaver that produced them, giving tartans the name of the last person who ordered it. My own family tartan, MacPherson, was assigned in that fashion, when in the 19th Century the Clan Chief requested the clan tartan from the weaving house, they sent him a tartan previously named Kidd, because the last person who ordered it was a West Indies plantation owner named MacPherson. The Art of Kiltmaking, p. 14.

Perhaps one source of the feeling that one should not wear a tartan not "belonging" to ones family is that after the Act of Proscription following the '45 Jacobite uprising, wearing the kilt was outlawed except for those serving in the Highland Regiments, which sometimes differentiated themselves by adopting distinct tartans, frequently variations of the Government sett, or Black Watch tartan. I suppose one might not want to wear the tartan of a unit in which they did not serve [except Black Watch tartan, a universal] at the risk of being accused of Walting about. [However, I think that any regimental tartan should be fair game, and should be considered a Government sett, belonging to all the people].

The Lord Lyon is the governmental authority for registering the heraldry of the Scots nobility, the Lord Lyon registered the arms and insignia of the Scottish aristocracy, which heraldry did not at first include assignment of specific tartans to specific clans. Eventually, a Scottish chieftan who wished to bear arms would apply to the Lord Lyon and seek to meet the requirements thereof. The Scottish Tartans Authority is likewise a recent invention.

Family association is a good reason for wearing a particular tartan, but it is not historically accurate to take the whole clan tartan system too seriously, or to let it's existence keep you from wearing a tartan you admire.

Cheers!