Understanding the Value of Systems Engineering

The practices of systems engineering are believed to have high value in the development of complex systems. Heuristic wisdom is that an increase in the quantity and quality of systems engineering (SE) can reduce project schedule while increasing product quality. This paper explores recent theoretical and statistical information concerning this heuristic value of SE. It explores the underlying theoretical relationships among project cost and schedule, technical value, technical size, technical complexity, and technical quality, summarizing prior work by the author. It then identifies and summarizes six prior statistical studies with conclusions that relate to the value of SE. Finally, the paper provides final results of a statistical study by the INCOSE Systems Engineering Center of Excellence (SECOE) that presents evident correlations supporting the heuristics. The results indicate that optimal SE effort is approximately 15-20% of the total project effort. Background The discipline of systems engineering (SE) has been recognized for 50 years as essential to the development of complex systems. Since its recognition in the 1950s [Goode 1957], SE has been applied to products as varied as ships, computers and software, aircraft, environmental control, urban infrastructure and automobiles [SE Applications TC 2000]. Systems engineers have been the recognized technical leaders [Hall 1993, Frank 2000] of complex project after complex project. In many ways, however, we understand less about SE than nearly any other engineering discipline. Systems engineering can rely on systems science and on many domain physics relationships to analyze product system performance. But systems engineers still struggle with the basic mathematical relationships that control the development of systems. SE today guides each system development by the use of heuristics learned by each practitioner during the personal experimentation of a career. The heuristics known by each differ; one need only view the fractured development of SE “standards” and SE certification to see how much they differ. As a result of this heuristic understanding of the discipline, it has been nearly impossible to quantify the value of SE to projects [Sheard 2000]. Yet both practitioners and managers intuitively understand that value. They typically incorporate some SE practices in every complex project. The differences in understanding, however, just as typically result in disagreement over the level and formality of the practices to include. Presciptivists create extensive standards, handbooks, and maturity models that prescribe the practices that “should” be included. Descriptivists document the practices that were “successfully” followed on given projects. In neither case, however, are the practices based on a quantified measurement of the actual value to the project. The intuitive understanding of the value of SE is shown in Figure 1. In traditional design, without consideration of SE concepts, the creation of a system product is focused on production, integration, and test. In a “system thinking” design, greater emphasis on the system design creates easier, more rapid integration and test. The overall result is a savings in both time and cost, with a higher quality system product. The primary impact of the systems engineering concepts is to reduce risk early, as shown in Figure 2. By reducing risk early, the problems of integration and test are prevented from occurring, thereby reducing cost and shortening schedule. The challenge in understanding the value of SE is to quantify these intuitive understandings. Field of Study Because there is wide variance in the perceptions of SE, any theoretical effort must start with a definition of terms that bounds the field of study. For this paper, “systems engineering” is taken in a broad sense that includes all efforts that apply science and technology (“engineering”) to the development of interacting combinations of elements (“systems”). Such efforts are frequently characterized as having both technical and management portions because of the interdisciplinary nature of system development teams. The breadth of skills necessary for good SE was studied well by [Frank 2000]. We take “SE management” to be the efforts that guide and control the systems engineering. There are obvious overlaps with (a) “development engineering,” the use of specific engineering disciplines to create the elements of a system, (b) “test engineering,” the application of engineering to verify and/or validate the system, and (c) “program management,” the overall control of a project. SE distinguishes itself from development engineering by the use of interdiscipline coordination and inter-element technical analyses. Test engineering when applied at the system level is taken to be a subset of SE, while test engineering applied to system elements is not. SE management is distinguished from program management in the use of technical analysis and control and in the emphasis on technical quality as opposed to financial and schedule concerns. Underlying Mathematical Theory Understanding the value of SE requires quantifying that value. Quantification requires understanding the numerical parameters that matter to SE. A useable mathematical theory that SYSTEM DESIGN DETAIL DESIGN PRODUCTION INTEGRATION TEST Traditional Design Saved Time/ Cost “System Thinking” Design Figure 1. Intuitive Value of SE. Time Risk Time Risk Traditional Design “System Thinking” Design Figure 2. Risk Reduction by SE. underlies SE can contribute to an understanding of the important parameters, particularly those important to SE management. There are frequently stated arguments against the feasibility of such a mathematical theory [Sheard 2000]. First, each system development is one of a kind. In presenting new challenges, each such development might invalidate the scientific basis of any prior theory. Second, projects vary widely in important parametric measures – size, schedule, and acceptable risk. This variance leads, in the few data collected previously [Mar 2002], to wide variance in the data points. Third, such a theory of necessity includes a statistical representation of the human nature of the developers, a representation that is frequently viewed with skepticism. Fourth, SE applies itself to systems that contain components developed by virtually any other engineering discipline [Honour 1999]. This highly varied application might defeat codification. Yet none of these arguments prove the impossibility of such a mathematical theory. This section summarizes the author’s work [Honour 2002] toward a mathematical theory of SE management and applies that theory to quantifying the value of SE. The intent of this summary is to lead up to a working hypothesis for the value of SE.