I have a friend who is recently retired from the RAF. He went to a Highland Games with me last summer, and was absolutely stoked to find a Black Watch kilt there in his size (he's stout!) He seemed to think that wearing the tartan was a good way to pay respect to his service, so it seems that even native Scots have no problem with it. So I would say wear it!