Router or modem?

First, what sort of broadband do you have? Cable or DSL are the most likely. Also, are you on a wireless connection or wired?

If you have the broadband modem, and then go to a router and you are not running additional machines off it, you can take the router out of the picture.
You have to have a modem but not a router. Routers do make great hardware firewalls though, and I STRONGLY recommend them for anyone with an 'always on' connection. Otherwise, your computer will very likely become someone's spam spewing Zombie and you wont even know it.
If it's a wireless router you will have to go with a wired connection to the modem instead.

Better yet would be to solve the problem. First, try unplugging the router and leaving it off for 30 seconds. Plug it back in and let it reset. That may fix your problem. If it doesnt, you may need a firmware update. Go to the website for the manufacturer of your router and look up your model number. Download any updates they offer.

Also, if you are on a wireless connection make sure that you are not accidentally switching signals while surfing. Windows will automatically switch over when it finds a stronger signal unless you tell it not to. Sometimes it does it when the signal is actually poor though, so you lose connectivity until reconnects or you reset the connection yourself.

Let us know if you cannot resolve the problem.