Risk and Requirements-Based Testing

I proposed that requirements development is not something that happens all at once at the start of a project. In real life, requirements are negotiated in the course of two simultaneous dialogues throughout the project life cycle. Those dialogues entail asking, " What do we want? " and " What can we build? " The quality of these dialogues goes a long way to determining the ultimate quality of the product. After reading that column, Jerry Weinberg e-mailed me the gentle criticism that the example I used (a file conversion tool) was too simplis-tic to illustrate the requirements testing problem effectively. What about safety-critical or otherwise complex products? Since Jerry co-authored, with Don Gause, my favorite book about requirements , Exploring Requirements: Quality Before Design (Dorset House, 1989), I feel compelled to address that problem and expand upon the role of testing in requirements development. I define requirements as the set of ideas that collectively define quality for a particular product. I define testing as a process of developing an assessment of product quality. First, I'll reframe the requirements testing problem in terms of risk, and then I'll show what happens in complex or high-risk situations. There are at least four alleged truisms about testing a product against requirements. Most of the testing textbooks on my bookshelf promote these principles, and each principle reflects some truth about the dynamics of testing. 1. Without stated requirements, no testing is possible. 2. A software product must satisfy its stated requirements. 3. All test cases should be traceable to one or more stated requirements, and vice versa. 4. Requirements must be stated in testable terms. When we think in terms of risk, however, I believe a richer set of ideas emerges. If it is very important to satisfy a requirement, and it is the job of the tester to evaluate the product against that requirement , then clearly the tester must be informed of that requirement. So there are situations where this statement is basically true. The deeper truth is that stated requirements are not the only requirements. Because of incompleteness and ambiguity , testing should not be considered merely as an evaluative process. It is also a process of exploring the meaning and implications of requirements. Thus, testing is not only possible without stated requirements, it's especially useful when they're not stated. I think we need to break out of the mythology …