You didn't say what color hair you have on your head, if any. For some people a black hat looks like odd hair, while most of us have the sense to recognize green as something you put on and take off.

I expect the man who wanted to sell you a green hat went through this process:
1) I sell hats.
2)Here is a man who wears a hat.
3) I have green and black ones to sell.
4) He already has a black one.
5) He may be more inclined to buy a green one than a second black one.

I believe you have already heard the best advice:
ASK YOUR WIFE.