Non-Monotonicity, Chaos and Combat Models

Simulation is a widely accepted means of analyzing systems that are too complex to model analytically Most communications systems fall into this category. But when a continuing program of verification and validation is not maintained, the credibility of the simulation suffers and the value of analyses that the simulation supports is diminished. The primary goal of any verification and validation process is to enhance both the correctness of a simulation and the confidence placed in its results. A persistent challenge facing the modeler is to develop a process that is both feasible and compatible with an organization's needs, and that is widely applicable. Multivariate statistical procedures can be used to assess the agreement between simulated predictions and empirical observations. This paper describes such a test that is useful for the validation of simulations of battlefield communications networks. The procedure is applied to a simulation that was developed to duplicate an experimental configuration in which messages were passed over a communications network using the combination of the Tactical Fire (TACFIRE) Direction System protocol and Single Channel Ground and Airborne Radio System (SINCGARS) Combat Net Radios (CNR). LIMITED BANDWIDTH TACTICAL NETWORKS The purpose of a communications network is to serve as a carrier of information from one location to another. The effective distribution of information can improve the decision process on the battlefield, whereas the consequences of making decisions based on obsolete information can be catastrophic. The maximum available bandwidth of the VHF-FM radios still utilized by lower echelon units is only 1,200 bits per second, a very limited data exchange rate. On a limited bandwidth tactical network, the number of nodes and the amount of information to pass can be large, especially during peak battle periods. To measure a network's effectiveness, a determination must be made of whether the messages arrive at their destination intact and in time to be useful. The amount of correctly passed information is referred to as "network throughput," and the amount of time required to pass that information as "network delay." There are a number of conditions that can impact throughput and delay, including the number of messages to be transmitted, the size of the messages, the number of nodes on the network, the communications protocol, and the communications hardware. If the effects of these factors on network performance are better understood, attempts to optimize the network's effectiveness are more likely to succeed. One way to examine the interactions of network parameters is through simulation. Simulation is a widely accepted means of analyzing real-world systems that are too complex to model analytically. Most communications networks fall into this category. The simulations commonly require as input the probability that two or more messages will collide, the expected delay in message transmission, or the arrival rate of messages at a given node, and then extrapolate those estimates to a complex network of multiple nodes. This approach is usually taken to simplify the simulation but requires stringent assumptions that may result in an unrealistic representation of the protocol. A computer simulation is only a surrogate for actual experimentation with an existing or conceptual system. Simulation credibility suffers and the value of analyses the simulation supports is reduced when a program of continuing verification and validation is not undertaken. A fundamental goal of validation is to ensure that a simulation is developed that can be used by a decision-maker to arrive at the same decision that would have been made if it were possible to experiment with the actual system. Validation should serve to increase both the logical correctness of a simulation and the confidence placed in its results. The challenge confronting modelers is to develop a validation process that is both feasible and effective and sufficiently general to allow its application to a broad class of simulations. It is not uncommon to find several groups in a military organization each developing a network simulation that performs essentially the same tasks; the differences usually lie in the assumptions and/or definitions of simulation responses. Ideally, a validation procedure should be able to accommodate the simultaneous comparison of several candidate simulations. Statistical Validation of a Communications Network Simulation Ann E. M. Brodeen and Malcolm S. Taylor U.S. Army Research Laboratory Application Areas: C3 OR Methodologies: Nonparametric Statistics Military Operations Research, V2 N2 1996 Page 21 STATISTICAL VALIDATION OF A COMMUNICATIONS NETWORK SIMULATION This initiative is part of a broader-based Army research program whose goal is to improve the ability of communications networks to deliver critical information on the battlefield when and where it is needed despite a rapidly changing and often hostile environment. It will also support the ongoing effort to formalize the validation process for communications network simulations that, in turn, provide the groundwork for testing hypotheses throughout the research program. This formalization needs to be readily transmitted to other organizations that rely on communications network simulations for