Editorial: Is software testing essential or accidental?

This issue contains three papers. Automatic instantiation of abstract tests on specific configurations for large critical control systems, by Flammini, Mazzocca, and Orazzo, proposes the interesting idea of creating abstract tests from system requirements and automatically instantiating them into concrete tests. Transition covering tests for systems with queues, by Huo and Petrenko, proposes another technique to automatically generate tests, this time for concurrent systems.Generating input data structures for automated program testing, by Chung and Bieman, studies another aspect of automatic test data generation, describing how statements are connected in terms of constraints that can be solved to yield test inputs. The common theme in these three papers is that they are trying to find automated ways to solve the hardest essential problem in testing software—generating test input values. When I was a college senior, a manager from IBM visited one of my classes. I will never forget his message. He said that computing was in its infancy and we should expect great changes during our careers. He also said that programmers spent 90% of their time on activities that were not directly related to the problem the software was trying to solve. The most time-consuming activity was debugging, followed by things such as keeping decks of punched cards in order, struggling with odd syntax and even odder semantics in poorly designed languages, wrestling with computers that had too little memory, convincing poorly designed operating systems to run our programs the way we wanted, and understanding what customers wanted. In graduate school, I read Brooks’ famous paper ‘No Silver Bullet: Essence and Accidents of Software Engineering’ [1]. His paper echoed the talk from the IBM manager and connected those issues with philosophy courses I took in college. If you haven’t read it, Brooks philosophized that software developers solve accidental problems, which are ‘difficulties that today attend its production but are not inherent,’ and essential problems, which are ‘difficulties inherent in the nature of software.’ The IBM talk and Brooks’ paper sparked my interest in software engineering, which I still view in large part as reducing the time software developers spend on accidental problems. I tell my students that we have made great progress; we are probably near the 50/50% level, but still far from a mature level, which I estimate to be about 10/90%. As my interest in software testing grew, I sometimes wondered whether testing is solving an accidental or essential problem. Testing is often treated as accidental in industry, where it is sometimes barely done at all, often done poorly, and seldom done well. Testing is often cut when schedules and budgets overrun, which sounds like testing is inessential. On the other hand, processes such