This may be a keep alive problem. A router shouldn't drop the connection to your provider unless your provider requests it. You're not with Tiscali per chance?
Anyway, you could try a keep alive program. This is a bit of software which randomly pings a website here and there to make your provider think you're still browsing.
I did have a quick search, but didn't have time to yield results. But to be honest, if you have broadband - i'd contact the ISP and tell them to sort it.