I would wear a pullover sweater or a pullover sweater vest with a jacket, but not just the sweater vest. Just not my style. I would wear a button-up vest without a jacket, though.