Sign in with
Sign up | Sign in

Researchers develop program to help developers catch bugs faster

By - Source: Tom's Hardware UK | B 1 comment

Arlington (TX) - A group of NIST researchers, in combination with University of Texas researchers, are developing an open-source tool which they believe will enable software developers to catch software errors more quickly. It’s called combinatorial testing, and promises to squash the bugs faster, and with fewer errors.

The researchers examined a wide array of software bugs and resulting crashes. Their research gave heed to the age-old adage which states, most software bugs are the result of simple errors, not complex ones. Taking this uncovered reality into consideration, the team began analyzing the number of dependent variables which could result in an error. This began with something called "pairwise testing", which looks at two variables in a program. The team then progressed until they were studying six. According to the release, "Their technique resembles combinatorial chemistry in which scientists screen multiple chemical compounds simultaneously rather than one at a time."

Sign up for our newsletter and be in with a chance to win one of two high performance graphics cards.

The example that’s given relates to word processing. Suppose there are 10 different settings, superscript, bold, italics, etc. If a program has a bug when three specific ones are set, then doing a hard-code test of all possible three-combinations would require 1,024 tests. However, by looking at the fact that you’re testing groups of three in the set of 10, only 13 tests are required.

This new software program examines the software variables and, based on the number of variables, generates sets of tests which can be run along those same lines using up to six variables. It generates fewer tests because it’s testing the combination of things which have been statistically shown to be the most common cause of software errors. Rather than testing every single possible condition exhaustively, the team believes that this interaction between a few common variables can result in a better capture rate of errors before the software is released, specifically in web-commerce software.

The team is going to open-source the project and release it next year. They are seeking beta-testers right now, though no links were given.

Discuss
Display 1 comment.
This thread is closed for comments
  • 0 Hide
    NickTu , 17 December 2007 21:52
    Nonsense.
    1. There are any number of free tools available to do this already. It's the sort of tool you can knock up yourself in a quiet evening - you don't need 2 teams of researchers working for months, backed up by beta testers.
    2. Developers will *never* check their work themselves anyway.
    3. This technique *may* produce better results than random test cases in very specific circumstances. There is no proof that it works as a general rule (which is presumably why they say they "believe" it will produce a better capture rate).