Bad testing, not design defects cause majority of software bugs 2 June, 2010 By Mark Cox

PromoPipeline Exclusive Channel Promotions Find Out How You Can Make Money Today! ENROLL FREE! >> Factory Direct Should Not be Cheaper William Vanderbilt - Innovative Learning Channels Cloud Ecosystem II: A Candid Conversation with Oracle Beth Vanni - Amazon Consulting Cloud Ecosystem: A Candid Conversation with Rackspace Hosting Beth Vanni - Amazon Consulting Channel Manager Compensation William Vanderbilt - Innovative Learning Channels Financial Expertise William Vanderbilt - Innovative Learning Channels

Fifty-eight percent of respondents pointed to problems in the testing process or infrastructure as the cause of their last major bug found in delivered or deployed software, not design defects. Specifically, the survey found:

-- Completely automated software testing environments are still rare, with just 12 percent of software development organizations using fully automated test systems. Almost 10 percent reported that all testing was done manually.

-- Forty-six percent of software developers said they do not have time to test as much as they should.

-- More than a third of developers, 36 percent, said they do not believe their companies perform enough pre-release testing. -- Fifty-three percent said their testing is limited by compute resources.

On occasions when a bug was found in released software, the reported impact to the organization in time and money lost was significant.

-- Fifty-six percent of respondents reported that bugs discovered late in development almost always affected release dates.

-- Fifty-six percent of respondents estimated that their last significant software bug resulted in an average of $250,000 in lost revenue and 20 developer-hours to correct.

-- Developers who felt their companies allotted sufficient time to pre-release testing were less impacted by bugs, spending less than half as much time resolving those bugs compared to other developers--a median of 12 developer-hours compared to 25 developer-hours.

"These survey results illustrate the challenges faced by companies in their software testing process," said Electric Cloud CEO Mike Maciag. "The continued reliance on slow, resource-intensive manual processes prevents organizations from being as thorough as necessary in their testing. Fully automated test systems save time and effectively utilize physical, virtual or cloud resources, while greatly reducing the risk of human error. Automated test systems help make developers and testers more efficient and effective in finding bugs before they reach the end user."

"As the software we rely on each day continues to grow in complexity, it becomes more and more essential that bugs are caught and repaired quickly," said Michael Osterman, CEO of Osterman Research. "With 88 percent of the companies represented in this survey still using manual testing to some extent and respondents rating dealing with their test systems as more painful than dealing with a fender bender, it's clear that software developers still have a long way to go toward full automation and effective test systems."

A total of 144 software development professionals including software developers, testers, managers, and executives were surveyed from organizations with at least 1,000 employees and 50 developers across a variety of industries. Respondents were selected using a combination of the Osterman Research survey panel and an external provider's survey panel. The median number of employees at the organizations surveyed was 5,000 and the median number of developers was 300. Most organizations surveyed were located in North America.