Check that its set to auto speed rather than manual.
If it's already on Auto, then its likely your CPU is getting too hot and the system needs to run the fans on high in order to keep it from overheating.
Downloading something like HWMonitor is good for keeping and eye on temperatures and fan RPM too.