Sign in with
Sign up | Sign in

WBGP10 Test Suite And Methodology

Web Browser Grand Prix: Chrome 18, Firefox 11, Windows XP
By

We restart the computer and allow it to idle for a few minutes before benchmarking each browser. Other than the conformance benchmarks, all of our final scores are an average of several iterations. More iterations are run on tests that have short durations, lower scales, and/or higher variance. 

All tests are placed into one of four groups: core, observation, dated, and quarantine. Core tests are considered current. These tests are usually trusted industry standards or our own creations, and they make up the foundation of the WBGP suite. Tests that are either generally unknown, mostly untested, or just too bleeding-edge are placed under observation. Tests classified as dated are either outdated, losing relevance, or otherwise need replacing. We are actively seeking community feedback and contributions regarding alternatives to these benchmarks. The final group is for quarantined benchmarks. Benchmarks find their way into quarantine by delivering dubious results or by being gamed. Whenever benchmarks that test the same thing yield conflicting results, more weight is given to tests with a better rating when creating the analysis tables.

The table below lists all 51 of the tests currently in our suite (along with a version number and link, where applicable), number of iterations performed, and current rating:

Tom's Hardware Web Browser Grand Prix Test Suite v9.0
Test Name
Iterations
Rating
Performance Tests (44)
Cold Startup Time: Single Tab
3
Core
Cold Startup Time: Eight Tabs3
Core
Hot Startup Time: Single Tab3
Core
Hot Startup Time: Eight Tabs
3
Core
Uncached Page Load Times (9 Test Pages)
5
Core
Cached Page Load Times (9 Test Pages)
5
Core
Kraken v1.1
2
Core
Google SunSpider v0.91 Mod2
Core
FutureMark Peacekeeper 2.02
Core
Dromaeo DOM2
Core
Maze Solver5
Core
GUIMark2 Flash Vector Charting3
Core
GUIMark2 Flash Bitmap Gaming3
Core
GUIMark2 Flash Text Columns3
Core
Flash Benchmark 2008 v1.09.1
2
Core
GUIMark Java3
Dated
Encog Silverlight3
Dated
Facebook JSGameBench v4.12
Core
GUIMark 2 HTML5 Vector Charting (1 pixel variant)
3
Core
GUIMark 2 HTML5 Bitmap Gaming
3
Core
GUIMark 2 HTML5 Text Columns
3
Core
Asteroids HTML5 Canvas 2D And JavaScript2
Observation
Psychedelic Browsing
2
Core
Hardware Acceleration Stress Test
2
Dated
Mozilla WebGL FishIE
5
Core
WebGL Solar System5
Observation
Efficiency Benchmarks (5)
Memory Usage: Single Tab
3
Core
Memory Usage: 40 Tabs
3
Core
Memory Management: -39 Tabs
3
Core
Memory Management: -39 Tabs (extra 2 minutes)
3
Core
Reliability Benchmarks (1)
Proper Page Loads
3
Core
Conformance Benchmarks (3)
Ecma test2621
Core
Peacekeeper 2.0 HTML5 Capabilities1
Core
HTML5Test.com1
Core


Detailed individual methodologies are described on the pages corresponding to each benchmark.

React To This Article