You could also use an imaging / back-up program like Acronis True Image.
If your system becomes compromised by malware you can reimage to a clean state. Acronis also has a try and decide feature which allows you to mess with the operating system and if you aren't happy with the result decide not to keep the changes and everything is reverted to how it was pre-try and decide, but with the older versions you couldn't reboot the system without deciding if you want to keep the changes (not sure about new Acronis products).
You can also look into sandboxie for isolating programs from the OS:http://www.sandboxie.com/
The best option though is using a virtual machine as manojgj has suggested and a decent RAM upgrade is better all round too.