Sign in with
Sign up | Sign in

The Myths Of Graphics Card Performance: Debunked, Part 1

The Myths Of Graphics Card Performance: Debunked, Part 1
By

Did you know that Windows 8 can gobble as much as 25% of your graphics memory? That your graphics card slows down as it gets warmer? That you react quicker to PC sounds than images? That overclocking your card may not really work? Prepare to be surprised!

If you're an auto enthusiast, you've no doubt debated the performance of two sports cars with a friend at some point. One might have made more horsepower. Maybe it had a higher top speed, superior handling, or lighter weight. Typically, those conversations come down to comparing lap times on the Nürburgring and end when someone spoils the fun by reminding us that we can't afford any of the contenders anyway. 

In many ways, high-end graphics cards can be quite similar. You have average frame rate, frame time variance, noise from the cooling solution, and a range of price points, which can incidentally double the cost of a current-gen gaming console. And if you needed any further convincing, some of the latest video cards have aluminum and magnesium alloy frames, just like race cars. Alas, some differences remain. Despite my best attempts at impressing my wife with the latest graphics processor, she remains impervious.

So, what is the lap time equivalent for a video card? What is the one measure that distinguishes winners from losers, cost being equal? It's clearly not just average frames per second, as demonstrated by all of the coverage we've given to frame time variance, tearing, stuttering, and fans that sound like jet engines. Then you get into the more technical specifications: texture fill rate, compute performance, memory bandwidth. What significance do all of those numbers hold? And, like a Formula 1 pit crew member, does your new card require headphones just to be tolerated? How do you account for the overclocking headroom of each card in an evaluation?

Before we dig into the myths that envelop modern graphics cards, let's start by defining what performance is and what it is not. 

Performance Is An Envelope, Not One Number

Discussions of GPU performance are often distilled down to generalizations based on FPS, or average frames per second. In reality, a graphics card's performance includes far more than the rate at which it renders frames. It's better to think in terms of an envelope, rather than one data point, though. This envelope has four major dimensions: speed (frame rate, frame latency, and input lag), quality (resolution and image quality), quietness (acoustic performance, driven by power consumption and cooler design), and of course affordability. 

Other factors play into a card's value, such as game bundles and vendor-specific technologies. I'll cover them briefly, but won't try to weigh them quantitatively. Truly, the importance of CUDA, Mantle, and ShadowPlay support is very user-dependent.

The above graph illustrates the GeForce GTX 690's position in this variable envelope I'm describing. Stock, it achieves 71.5 FPS using a test system I'll detail on the following page in Unigine Valley 1.0 at the ExtremeHD preset. It generates an audible, but not bothersome 42.5 dB(A). If you're willing to live with a borderline-noisy 45.5 dB(A), you can easily overclock the card and get a stable 81.5 FPS using the same preset. Lower the resolution or anti-aliasing level (affecting quality), and you get a big bump up in frame rate, all else being equal. Of course, the (un)affordable £800 price point doesn't change.

For the sake of running tests in a more controlled manner than you're used to seeing, let's define a reference for video card performance.

For the purposes of today's story, I'll specify performance as the frames per second a graphics board can output at a given resolution, within a specific application along the described envelope (and under the following conditions):

  1. Quality settings in a given application set to their highest value (typically the Ultra or Extreme preset)
  2. Resolution set to a constant level (typically 1920x1080, 2560x1440, 3840x2160, or 5760x1080 in a three-monitor array)
  3. Driver settings at each manufacturer's defaults (whether global or application-specific)
  4. Operating in a closed enclosure at a set 40 dB(A) noise level measured three feet away from the enclosure (ideally, tested on a reference platform that gets updated annually)
  5. Operating with an ambient temperature of 20 °C/68 °F and one atmosphere air pressure (this is important; it directly affects thermal throttling)
  6. Core and memory operating at temperature equilibrium as far as thermal throttling is concerned (so that core/memory clock speeds under load remain fixed or vary within a tight range, given a constant 40 dB(A) noise level (and corresponding fan speed) target
  7. Maintaining a 95th percentile frame time variance below 8 ms, which is half a frame at a typical display refresh rate of 60 Hz
  8. Operating at or near 100% of GPU utilization (this is important to demonstrate a lack of platform bottlenecks; if there are bottlenecks, GPU utilization will be below 100% and the test results will not be very meaningful)
  9. Averaged FPS and frame time variance data from no fewer than three runs per data point, each run no less than one minute long, with individual samples exhibiting no more than 5% deviation from the mean(ideally we want to sample different cards of the same time, particularly when there is reason to believe a vendor's products exhibit significant variance)
  10. Measured with either Fraps for a single card or any built-in frame counter; FCAT is required for multiple cards in SLI/CrossFire

As you can imagine, the reference performance level is both application- and resolution-dependent. But it's defined in a way that allows for independent repetition and verification of tests. In this sense, it's a truly scientific approach. As a matter of fact, we encourage the industry and enthusiasts alike to repeat the tests we perform and bring any discrepancies to our attention. Only in this way will the integrity of our work be assured.

This definition of reference performance does not account for overclocking, or the range of behaviors a given GPU might exhibit from one card to another. Fortunately, we'll see that's only an issue in a few cases. Modern thermal throttling mechanisms are designed to eke out maximum frame rates in as many situations as possible, so cards are operating closer than ever to their limits. Ceilings are often hit before overclocking adds any real-world benefit.

Unigine Valley 1.0 is a benchmark we use extensively in this article. It features a number of DirectX 11-based features and produces highly repeatable tests. It also doesn't rely on physics (and thus CPU) as much as 3DMark (at least in its overall and combined tests).

What Are We Setting Out To Do Here?

In the course of this two-part story, I plan to look at each of the dimensions that compose a video card's performance envelope, and then try to answer common questions about them. We'll extend the conversation to input lag, display ghosting, and tearing, all of which relate to your gaming experience, but not specifically to frame rates. I'd also like to compare cards using this criteria. As you can imagine, testing this way is extremely time consuming. However, I think the additional insight is worth the effort. That doesn't mean our graphics card reviews are going to change; we're experimenting, and taking you with us.

With the definition of graphics card performance already covered, the rest of today's piece involves methodology, V-sync, and a look at the amount of video memory you really need. Part two will look at anti-aliasing technologies, noise and the noise level-adjusted performance of graphics cards, the impact of display choice, various PCI Express link configurations, and the idea of value for your money.

Time to move on to the test system setup. More so here than in other reviews, you will want to read that page carefully, since it contains important information about the tests themselves.

Ask a Category Expert

Create a new thread in the UK Article comments forum about this subject

Example: Notebook, Android, SSD hard drive

Display all 11 comments.
This thread is closed for comments
  • 0 Hide
    Jak_Sparra , 10 February 2014 13:38
    Can't wait for part 2. I have a Sapphire R9 290 Tri-X and am enjoying the smoothest gameplay I've ever experienced at 1920x1200 with ultra settings in games like BF4. BUT, I'm interested in getting a 2560x180p monitor for gaming. The only thing holding me back is that I'm worried how much of a drop in FPS I will see. Hope the next article covers stuff like that. Also, as more and more gamers get more system RAM, I'd love to see an article that covers what happens when you use RAM as a RAMdisc and stick the pagefile on it. Would it act nearly as fast as VRAM?
  • 0 Hide
    rolli59 , 10 February 2014 13:51
    Great article, waiting on part 2.
  • 0 Hide
    Jonathan Cave , 10 February 2014 15:11
    Great Article.
  • 0 Hide
    Jak_Sparra , 10 February 2014 15:43
    Can't wait for part 2. I have a Sapphire R9 290 Tri-X and am enjoying the smoothest gameplay I've ever experienced at 1920x1200 with ultra settings in games like BF4. BUT, I'm interested in getting a 2560x180p monitor for gaming. The only thing holding me back is that I'm worried how much of a drop in FPS I will see. Hope the next article covers stuff like that. Also, as more and more gamers get more system RAM, I'd love to see an article that covers what happens when you use RAM as a RAMdisc and stick the pagefile on it. Would it act nearly as fast as VRAM?
  • 0 Hide
    Jonathan Cave , 10 February 2014 15:47
    Great Article.
  • 0 Hide
    kyzarvs , 10 February 2014 15:54
    @Jak - this is covered on page 6?"What happens when graphics memory is completely consumed? The short answer is that graphics data starts getting swapped to system memory over the PCI Express bus. Practically, this means performance slows dramatically, particularly when textures are being loaded. You don't want this to happen. It'll make any game unplayable due to massive stuttering."
  • 0 Hide
    Sunius , 10 February 2014 16:40
    Hey, about memory usage and Windows AERO: did you try benchmarking peak memory usage between various windows versions when in fullscreen mode? Going to fullscreen mode should effectively make Windows use 0 graphics memory as far as it's concerned (hence why it takes a while to switch to and out of fullscreen - it is moving data out and into video memory).
  • 0 Hide
    Wossnames , 11 February 2014 14:05
    10 dB isn't "twice as loud". It is ten times the sound pressure. 3 dB is about the double sound pressure.However, as we humans do not perceive sound linearly, around 6 dB (actually about four times the sound pressure) is generally perceived as "twice as loud".
  • 0 Hide
    HEXiT , 12 February 2014 22:19
    nice... pretty much confirms what i was thinking about overclocking, the results little in the way of real performance gains.for your next foray into overclocking could you do real world cpu performance. as gamers may well be in for a shock... with very limited returns for a massive overclock, power draw and reduced cpu life.productivity on the other hand can bring real returns... so it would be nice to have this confirmed.
  • 0 Hide
    cdrkf , 13 February 2014 11:16
    One myth I think you missed and really should highlight with respect to graphics memory is that as you say the amount doesn't effect performance, but the TYPE of memory really does.There are a lot of lower end cards popping up equipped with large amounts of DDR3 memory (e.g. an R9 250 with 2gb DDR3), and these are categorically a worse buy than a similarly priced card equipped with less but faster GDDR 5 memory...
  • 0 Hide
    jabel_sk , 23 February 2014 00:42
    Skyrim is a not a good game to benchmark VRAM because it's a horrible port. However, the modding community has in what some would argue, redesigned the engine's memory allocation system altogether.