I did something very similar with a black 3-button jacket, it worked very well for me, so go for it!