Sign in with
Sign up | Sign in

Help Us With the Web Browser Grand Prix Scoring System

By - Source: Tom's Hardware US | B 3 comments
Tags :

Requests for a points-based scoring system have appeared in the comments section of the Web Browser Grand Prix articles for some time now. Today we're ready to implement one, but we need help. And who better to ask than you, the Tom's Hardware readers.

Over the past couple of years we've implemented several reader suggestions into the Web Browser Grand Prix, such as adding the analysis tables alongside raw placing, later dropping the placing tables entirely, and de-emphasizing the winner over other strong finishers. However, one of the most frequent requests has been to incorporate some kind of points-based scoring system. One which gives added weight to the more important categories of testing, and less weight to areas that have little or no bearing on everyday real-world Web browsing.

We've received numerous emails suggesting such a system, but so far they've all been too simplistic or far too complicated (think Dungeons & Dragons rule-set). With the tenth installment of the Web Browser Grand Prix just around the corner, we think it's about time to grant this request. So, we're seeking your help.

First, let's look at the the current analysis table from which the champion is largely determined. Today the Web Browser Grand Prix has 48 individual tests which fall into the following 14 categories:


Winner
Strong
Average
Weak
Page Load Time




JavaScript




DOM




CSS




Page Load Reliability




Standards Conformance




Flash




HTML5




Startup Time




Memory Efficiency




Java




Silverlight




HTML5 Hardware Acceleration




WebGL





From here we need to rank these categories into brackets which reflect their importance to the average Web browsing experience. We've come up with the following four brackets:

Essential
Page Load Time, JavaScript, DOM, CSS, Page Load Reliability, Standards Conformance
Important
Flash, HTML5
Nonessential
Startup Time, Memory Efficiency, Java, Silverlight
Unimportant
HTML5 Hardware Acceleration, WebGL


The Essential bracket holds everything that makes up the core of what it is to browse the Web. The Important bracket includes the ubiquitous Flash plug-in and the rapidly-evolving HTML5 spec. The Nonessential bracket is for tests that could apply to any application (not just browsers) as well as the common, but lesser-used plug-ins. The Unimportant bracket is for upcoming technologies that simply aren't found in the wild, outside of testing and demo pages. While these brackets aren't set in stone and we're still open to feedback, the next step is where we really need your help.

This is where the points come in. We need to assign point values to the bracketed analysis table. There are a variety of ways to go about this. We could have a simple system where each type of finish (winner, strong, average, and weak) has a set score and a different modifier is applied to each bracket. Alternatively, we could have different point values assigned to each finish in each bracket.

Either way, there are more questions to be answered. Does an acceptable finish rate any points at all? Should weak be given negative points? Or should every type of finish in every bracket merit some points? How much of a bonus does the winner deserve over the strong finishers? Et cetera, et cetera.

Testing for the tenth installment of the Web Browser Grand Prix is complete - this one has a twist, and it's not what you'd think. Give us your feedback on the scoring system in the comments below so we can declare a champion. The outcome of Web Browser Grand Prix 10 is up to you!

Display 3 Comments.
This thread is closed for comments
  • 0 Hide
    Anonymous , 13 April 2012 02:00
    I'd say Startup Time and Memory efficiency are both important; if you had a browser that took 10 minutes to load it would be unusable and, similarly, if you had a browser that used memory as fast as the cookie monster eats cookies, it hardly would be the "best" browser. These two impact the user of a browser much more than the dying flash techonology and the widely unused html5 does currently.

    As for the scoring system, a fair way of doing it would be 100 for the winner, and the rest being percentages of that score. Eg, for load up time:

    Browser | Loadup time | Inverse | Percentage/Score
    1 | 10 | 0.1 | 50
    2 | 20 | 0.05 | 25
    3 | 5 | 0.2 | 100

    This way, if two browsers are really close, the score will be a better indicator of the overall better browser.
  • 1 Hide
    mi1ez , 13 April 2012 10:06
    They won't look at the UK comments, so meh.
  • 1 Hide
    AndrewJacksonZA , 13 April 2012 19:50
    Thanks for taking the time to gather our input, we appreciate it.

    annymmoThe most important things of course if it does what it should do in the first place:p age reliability and correctness.Speed always comes after that.
    +1 It's the browser's MOST IMPORTANT job to render the pages correctly.

    I have noticed a disturbing trend with browsers once memory usage goes above +-800MB/1024MB: they seem to die. To try and reproduce what I'm talking about, load up 20-40 tabs that have big pages in them (e.g. lots of images, adverts and interactive content like js/flash) and then browse around- how well does the browser perform? Is it still responsive? Does it crash? Do graphical artefacts appear?
    I disagree on the topics assigned to the brackets. My breakdown:

    Essential (in descending order)
    Page Load Reliability
    CSS
    Standards Conformance
    HTML5
    DOM
    JavaScript
    Ability for the browser to still work once memory usage goes above +-800MB/1024MB - how well does the browser perform? Is it still responsive? Does it crash? Do graphical artefacts appear?

    Important
    Page Load Time
    Flash
    Memory Efficiency for one tab that is a blank page and then 10, 20 & 40 tabs that contain pages.

    Nonessential
    Startup Time - How many times do you start up a browser a day? 10 times? 10 x 20sec = 2min 20sec.
    Java
    Silverlight

    Unimportant
    Hardware Acceleration - What, besides games, use this?
    WebGL

    Thanks