Personally, I do wear my Sgian Dubh a lot because I use it a lot. The only times I feel any great discomfort are if it's laced in too-tight.
Try loosening your garters just a little more and see how it goes.