I usually pin it to my jacket, but I have a special stitched part on my jacket especially for putting any sort of pin through.
Perhaps you could have something like that sewn to your jacket?

I know quite a few people who don't pin it to their jacket at all and it doesn't give them any sort of trouble at all.