Test and Evaluation of Highly Complex Systems

Abstract : I have been working in Army test and evaluation (T&E) since 1974 and have seen enormous increases in the complexity of systems since I first started in the business. Systems in 1974 were largely stand-alone, analog, and mechanically controlled. Now to conduct and win the next major conflict with a conventional enemy or against violent extremist movements, previously unimagined systems are being developed and fielded to our warfighters. The complexity of these new systems is a result of addressing today's and tomorrow's threats with more accurate, lethal, reliable, survivable, interoperable, and maintainable systems. Most of today's systems are very software intensive and network enabled, and have on-board, complex subsystems. The complexities that often arise are a result of the interactions among the systems and subsystems, and as a result, they cannot be tested and evaluated in isolation. These systems are vital enablers that assist the warfighters in accomplishing their missions. These new systems are often a system of systems (SoS) on a single platform such as the mine-resistant ambush-protected (MRAP) vehicles or a family of systems such as the Future Combat Systems (FCS) and the Stryker family of vehicles. The Army Test and Evaluation Command (ATEC) uses a number of processes and best practices to T&E SoS and FoS per the policy and guidelines directed in the June 2007 Office of the Secretary of Defense Section 231 Report. Some of the challenges for the command in implementing the guidance found in the Section 231 Report are to maximize the use of mission-based test and evaluation, Modeling and Simulation (M&S), joint and distributed testing, reliability growth testing, and determining system interoperability (subsystem, system, and with multiple systems). This article discusses a few of these challenges.