How to configure Firewall for VPN ?

I am using OPEN VPN to connect to a VPN service. I am running Windows 7 and it's built in firewall.

My issue: I am unable to get the firewall to block all traffic when VPN is disconnected.

I created the following rules :

1. block all connections - outbound/inbound - domain level

2. allow tcp connection on port 443 - outbound

3. allow tcp connection on port 443 - inbound



With the above setup, when the VPN is disconnected:

I cannot browse HTTP websites, however, I am able to browse HTTPS websites.
I cannot allow this.


In the simplest way, I want my computer to ONLY connect to the internet using VPN...if VPN is not connected, my computer should be completely cut off from the internet on all ports.


What rules am I missing to make this work ?

Should I use a 3rd party firewall program to make this easier ?

Thanks