My personal thoughts on this are that metal cantles are fine for any time of day - it's when you mix metal cantles with hair or fur that you start pushing your way to formal evening wear (though this shouldn't stop you from wearing a fur sporran during the day if you so choose).

That said, the brass cantle certainly brings it down a bit in formality. I'd say a nice fur sporran with a brass cantle would be the happy medium between a hunting sporran with silver cantle and a full dress sporran; so it would be perfectly acceptable for semi-formal evening wear and good for any time of day.

Though I'll add my usual disclaimer that I'm certainly no expert in these matters!