Software inspections and debugging a must for effective SQA

Regular readers may have gathered that I'm old, even though Network World is still using a picture of me from 10 years ago when I started writing this column in 2000. Certainly the (mostly) 18 to 22 year-old students in my freshman through senior classes at Norwich University perceive me as absolutely ancient. Readers older than they may be shocked to realize that these young adults were not born when the Morris Worm hit the Internet on Nov. 2, 1988 – and that when I tell them that I began programming in assembler in 1965 it's a bit like someone telling me (I was born in 1950) that they started writing on stone tablets with chisels in 1928.

Nonetheless, mostly by making fun of myself and of what to them is my advanced age, I still seem to reach the students with experiences from my years working in industry. Today I'll tell you more about fundamentals of software testing, in which I've been professionally involved since (gasp) 1979 when I first began programming for pay.

In general, software developers and systems engineering experts have found that the cost of correcting mistakes rises by about 10 times with every additional stage of the software development life cycle (SDLC) and related software development processes (you can download a PowerPoint lecture reviewing these topics from my software engineering course notes by distinguished author and teacher Ian Somerville, author of the well-known text Software Engineering, 8th Edition). The 309-page study, "The Economic Impacts of Inadequate Infrastructure for Software Testing" prepared in May 2002 for the National Institute of Standards and Technology (NIST) summarizes the situation as follows (pp 5.3 & 5.4):

The relative cost … of repairing defects found at different stages of software development increases the longer it takes to find a bug. . . . For example, errors introduced during this stage and found in the same stage cost 1X to fix. But if the same error is not found until the integration and component/RAISE [Reliability, Availability, Install Serviceability, and Ease of Use] system test stage, it costs 10 times more to fix. This is due to the reengineering process that needs to happen because the software developed to date has to be unraveled and rewritten to fix the error that was introduced earlier in the production process. However, bugs are also introduced in the coding and integration stages of software design.

Human testing can be highly effective, especially if it is applied after analysis and design but before coding. Inspection using a team approach gets the programmer who wrote the code to explain every line to other programmers; tests repeatedly show that the teams find 30% to 70% more errors than the original programmer. Particularly effective are walkthroughs, (described in a lecture my 1990s SQA course at John Abbott College) which are inspections where programmers pretend to be the computer and discuss how every line of code is going to be executed).

Just a week before writing this article, I was explaining to a second-year computer science student that practically the only way to debug assembler code is to establish a matrix of registers across one axis vs. steps in the code across the orthogonal axis and to keep a record (for example in a spreadsheet) of exactly what the contents of every register is at every step of execution. We also discussed debug tools. The basic approach in debugging is to run a compiled program in a shell that allows setting breakpoints where the programmer can inspect and, if necessary, change the state of stored information (control flags, array indexes, loop counters, and so on) in an attempt to understand and correct for programming errors.

In the next article in this series, I'll introduce the case study I promised some weeks ago that demonstrates the economic value of automated testing.

* * *

You can find my notes on software development and quality assurance in a supplementary PowerPoint lecture  (also available as a PDF file) in the IS342 Management of Information Assurance course I teach at Norwich.

Learn more about this topic

Pushing for software quality assurance

Software development and quality assurance

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.

Copyright © 2010 IDG Communications, Inc.

IT Salary Survey 2021: The results are in