Issue with wireless connectivity through my Linksys

Okay, hi all. Hopefully this will be an easy fix. The issue i am having is with my linksys BEFW11S4. I just recently switched over to cable internet, and this problem has always been here but i am just now realizing it thanks to having enough bandwith to run more than one pc at a time in my house. It seems that when i use my homebuilt pc with windows 7 (which is wired into the linksys) that everything else trying to connect to the wireless is able to... however when i turn off my windows 7 desktop and try to connect to the linksys's wireless without the wired pc on, i can only get local access and the internet won't work. How can i make it so that the wireless internet will always be active no matter what i do with my wired pc? Thanks in advance. if you need any sort of info, let me know and i'll supply it asap.

P.S. i upgraded the router's firmware yesterday, and cloned my mac address to the router's page in order to connect to the internet from my desktop machine, so that shouldn't be the issue.

Your Internet provider doesn't use PPTP, does it? If so, do you have it set up in the router, or your PC?