on first glance I thought you were going for a PC Coat look.

After pondering it a bit more...

You might consider putting some kind of collar at the top, perhaps a standing color like on a nehru jacket...taing that a step farther, maybe make the front to look like a crail jacket or one of the waist coat styles (in fact something like this http://kiltstore.net/sr_swhdr_lowlanderwaistcoat.html )

My initial thoughts are, it's an interesting idea. Could I see wearing it around town? I don't know. It could be a handy hiking item though in certain weather when you don't really need to shed a lot of water but just need to trap a bit more heat than a t-shirt.

Would I use it as a scarf? Probably not. I'd just pull it over the head. I would keep the buttons though because of the look it offers unless you replaced it with some kind of design.

I'd also square of the back horizontally, I think it would look more masculine.