To dress down an argyle, I prefer a colored dress shirt, open collar with a black T-shirt underneath:

Such as here.

I've also got a midnight blue that I like to wear with the jacket when a tie doesn't seem appropriate.