The Beret in the American military was fought for EXTREMELY hard, and was granted to the US Army Special Forces by JFK. They would, as a group, probably rather part with a testicle than their berets.

That said, I understand their defensiveness towards anything similar being worn by a civilian.