I think it would look peculiar, if you didn't wear neckwear.

If not a jabot, how about a stock? Can be got from most horse riding suppliers. Bow tie, stock or jabot should be worn with a winged collar shirt.

Regards

Chas