Valve Fixes The Steam Hardware Survey

Valve identified a problem with its Steam Hardware Survey and has deployed a fix. The survey tracks a portion of Steam's 125 million active users, so we in the media often uses it to track the hardware and software that gamers use most frequently. Unfortunately, the widely cited survey has suffered from extreme changes in key tracking areas over the last seven months due to an error in the reporting system. That means the survey has largely been useless for its intended purpose of reflecting broad trends in CPU and GPU usage, among other areas.

Steam's survey asks users to agree to participate, and if a user opts in, the survey queries the system for basic information about the hardware and software installed on the system. Valve adds the information to its database and then shares some of the most important data with the public.

Steam designed the survey to query its users’ systems once per year, but the company discovered that Asia-based cyber cafes manage their systems in such a way that the survey could be completed multiple times per year on a single PC. This duplicated the entries multiple times, which skewed the results of the survey. Typically, we would expect an automated system to weed out multiple entries through identifying information from the host computer, but Steam either protects its users by not collecting such data, or the duplicate entries simply went undetected.

The duplicate entries led to inflated statistics for Windows 7 usage, CPU and GPU market share, and an erroneous report of a rise in the number of quad-core systems. Steam has deployed a fix to correct the issue, and the company contends that all its data from April 2018 onward will be correct.

However, it appears that the company isn't correcting the erratic data it gathered in the previous months. That means the April 2018 Steam Hardware and Software Survey is the only accurate measure of hardware usage by Steam members in the last seven months. It also isn't clear if the company made other changes to its survey methodology since it last posted accurate data in July 2017. 

AMD's return to prominence in the desktop processor market finds it trading blows with Intel as they release new waves of competing chips, and while it's difficult to find accurate market share information, the company has undoubtedly made up some ground. The latest survey doesn’t reflect that change. Assuming Steam's data in July 2017 was accurate, this chart outlines the changes over the previous seven months:

Steam User Market ShareAMD CPUIntel CPUAMD GPUNvidia GPUIntel GPU
Steam July 2017 Survey19.42%80.49%20.23%53.9%15.52%
Steam April 2018 Survey15.96%84.04%14.89%75.26%9.69%
Increase/Decrease-3.46%+3.55%-5.34%+11.36%-5.83%

Given the success of Ryzen, the 3.46% reduction in systems with AMD processors seems questionable. Steam doesn't share much public information about its survey methodologies, such as how many systems it queries per year or the percentage of laptops and desktop PCs, so we can’t use these numbers as a direct comparison of AMD's penetration into gaming rigs.

The GPU market is also tricky to accurately measure, especially given the enormous number of GPUs that have found a home in mining rigs. In either case, the survey results claim that AMD has suffered a 5.34% decline in Steam users' gaming machines, while Nvidia has made significant headway against AMD and Intel (iGPU).

Steam's database also tracks other key metrics, such as how many CPU cores the average system has, Windows distributions, gaming resolutions, and memory capacity, among others. While these measurements are also subject to many of the same caveats as the rest of the Steam survey, they're the best available measure of trends in the gaming market.

Unfortunately, for now, we're limited to the April 2018 results alone. Because the statistics for preceding months are skewed wildly, we won't know the true monthly changes until next month when Valve releases more statistics with the fixed reporting system. That means you should ignore many of the recent articles that analyze the month-over-month changes.

Create a new thread in the UK News comments forum about this subject
This thread is closed for comments
1 comment
Comment from the forums
    Your comment
  • fogoldfarmer
    "Given the success of Ryzen, the 3.46% reduction in systems with AMD processors seems questionable." I suspect there is significant lag in this info.