An Agent-Based Model of Miscommunication in Complex System Engineering Organizations

Communication in complex system design organizations affects the performance of the systems they design. Miscommunication occurs when communication results in a “deficiency” or “problem” that hinders parties from fulfilling their individual or collective values. A recent study demonstrated widespread miscommunication in a Fortune 500 engineering firm about the definition of “an estimate” in a complex system design context. Building on that work, this study used a Monte Carlo simulation (8800 runs) of an agent-based model to demonstrate how systemic design process miscommunication may affect complex system performance. Each run of the simulation created a unique 1,000-artifact system using a network generation algorithm and converged its design through optimization. Systems where estimates communicated “current” designs outperformed systems where estimates communicated “future” projections of their designs instead. Varying the fraction of the population which uses each definition of an estimate varied system performance and uncertainty. Whether related to estimate definitions or more generally, miscommunication may affect system performance.

[1]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[2]  Forrest Stonedahl,et al.  The Complexities of Agent-Based Modeling Output Analysis , 2015, J. Artif. Soc. Soc. Simul..

[3]  Christopher L. Magee,et al.  Engineering Systems: Meeting Human Needs in a Complex Technological World , 2011 .

[4]  Jesse Austin-Breneman,et al.  Estimate Uncertainty: Miscommunication About Definitions of Engineering Terminology , 2020, Journal of Mechanical Design.

[5]  Derek de Solla Price,et al.  A general theory of bibliometric and other cumulative advantage processes , 1976, J. Am. Soc. Inf. Sci..

[6]  Anna-Maria Rivas McGowan,et al.  The Design of Large-Scale Complex Engineered Systems: Present Challenges and Future Promise , 2012 .

[7]  Murat Yuksel,et al.  Scale-Free Overlay Topologies with Hard Cutoffs for Unstructured Peer-to-Peer Networks , 2006, 27th International Conference on Distributed Computing Systems (ICDCS '07).

[8]  Yaneer Bar-Yam,et al.  The Statistical Mechanics of Complex Product Development: Empirical and Analytical Results , 2007, Manag. Sci..

[9]  Paul D. Collopy.  Systems engineering theory: What needs to be done , 2015, 2015 Annual IEEE Systems Conference (SysCon) Proceedings.

[10]  Wen-Hsing Liu,et al.  A comprehensive model of project team technical performance , 2016 .

[11]  Christopher McComb,et al.  Lifting the Veil: Drawing insights about design teams from a cognitively-inspired computational model , 2015 .

[12]  Mariano Ruiz Espejo,et al.  Design of Experiments for Engineers and Scientists , 2006, Technometrics.

[13]  Jitesh H. Panchal,et al.  Agent-Based Modeling of Mass-Collaborative Product Development Processes , 2009, J. Comput. Inf. Sci. Eng..

[14]  Wander Jager,et al.  From the Special Issue Editors: Agent-Based Modeling of Innovation Diffusion , 2011 .

[15]  J. Gareth Polhill,et al.  The ODD protocol: A review and first update , 2010, Ecological Modelling.

[16]  Capers Jones,et al.  Why software fails , 1996 .

[17]  K. W. Hess,et al.  Columbia Accident Investigation Board Report. Volume Five , 2003 .

[18]  Udo Lindemann,et al.  Reflecting communication: a key factor for successful collaboration between embodiment design and simulation , 2009 .

[19]  Gildas Morvan,et al.  Multilevel Agent-Based Modeling of System of Systems , 2017, IEEE Systems Journal.

[20]  J. Pennebaker,et al.  Lying Words: Predicting Deception from Linguistic Styles , 2003, Personality & social psychology bulletin.

[21]  Krishna R. Pattipati,et al.  Markov Modeling and Analysis of Team Communication , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[22]  Thomas W. Lucas,et al.  Exploring the world of agent-based simulations: simple models, complex analyses , 2002, Proceedings of the Winter Simulation Conference.

[23]  Bernard P. Zeigler,et al.  A layered modeling and simulation architecture for agent-based system development , 2001, Proc. IEEE.

[24]  Eric Bonabeau,et al.  Agent-based modeling: Methods and techniques for simulating human systems , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[25]  Hamid R. Darabi,et al.  Governing Competition and Collaboration in Network Industries Using Agent-Based Modeling: A Case-Study of US Air Transportation Network , 2017, IEEE Systems Journal.

[26]  Dan Braha,et al.  The Topology of Large-Scale Engineering Problem-Solving Networks , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[27]  Charles Johnson.  Forensic software engineering: are software failures symptomatic of systemic problems? , 2002 .

[28]  Claudia Baier,et al.  Principles Of Optimal Design Modeling And Computation , 2016 .

[29]  Christopher F. Earl,et al.  Formality in design communication , 2013, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[30]  Christina Bloebaum,et al.  Value Impact of an Organization Structure in the Context of Value-Driven Design , 2015 .

[31]  Dorothy McKinney.  A Complexity Primer for Systems Engineers , 2013 .

[32]  R. Schroeder,et al.  A THEORY OF QUALITY MANAGEMENT UNDERLYING THE DEMING MANAGEMENT METHOD , 1994 .

[33]  James V. Beck,et al.  Parameter Estimation in Engineering and Science , 1977 .

[34]  Steven D. Eppinger,et al.  Information Flow Through Stages of Complex Engineering Design Projects: A Dynamic Network Analysis Approach , 2015, IEEE Transactions on Engineering Management.

[35]  William Rand,et al.  An Introduction to Agent-Based Modeling: Modeling Natural, Social, and Engineered Complex Systems with NetLogo , 2015 .

[36]  RACHAEL LUCK,et al.  Articulating (mis)understanding across design discipline interfaces at a design team meeting , 2013, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[37]  Michael J. North,et al.  Tutorial on agent-based modelling and simulation , 2005, Proceedings of the Winter Simulation Conference, 2005..

[38]  Charles M. Macal,et al.  Everything you need to know about agent-based modelling and simulation , 2016, J. Simulation.

[39]  Birgit Müller,et al.  A standard protocol for describing individual-based and agent-based models , 2006 .

[40]  Beom Jun Kim,et al.  Growing scale-free networks with tunable clustering. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[41]  Ray Reagans,et al.  Commitment, Learning, and Alliance Performance: A Formal Analysis Using an Agent-Based Network Formation Model , 2013 .

[42]  Murat Yuksel,et al.  Ad-hoc limited scale-free models for unstructured peer-to-peer networks , 2008, 2008 Eighth International Conference on Peer-to-Peer Computing.

[43]  Tyson R. Browning,et al.  Degree Distribution and Quality in Complex Engineered Systems , 2011 .

[44]  Alexander Artikis,et al.  Interleaving multi-agent systems and social networks for organized adaptation , 2011, Comput. Math. Organ. Theory.

[45]  Anna-Maria Rivas McGowan,et al.  A Socio-Technical Perspective on Interdisciplinary Interactions During the Development of Complex Engineered Systems , 2013, CSER.

[46]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[47]  Jesse Austin-Breneman,et al.  Gaming the System: An Agent-Based Model of Estimation Strategies and Their Effects on System Performance , 2017, DAC 2017.

[48]  Sara A. McComb,et al.  An investigation of project complexity's influence on team communication using Monte Carlo simulation , 2011 .

[49]  P. John Clarkson,et al.  Assessing Organizational Capabilities: Reviewing and Guiding the Development of Maturity Grids , 2012, IEEE Transactions on Engineering Management.

[50]  Ruth A. Guthrie,et al.  The Columbia Disaster: Culture, Communication & Change , 2005, J. Cases Inf. Technol..

[51]  J. Doye,et al.  Global Optimization by Basin-Hopping and the Lowest Energy Structures of Lennard-Jones Clusters Containing up to 110 Atoms , 1997, cond-mat/9803344.

[52]  Jason Swarts,et al.  Communication design , 2012, CDQR.

[53]  Matthias Meyer,et al.  Opening the ‘black box’ of simulations: increased transparency and effective communication through the systematic design of experiments , 2011, Computational and Mathematical Organization Theory.

[54]  Carliss Y. Baldwin,et al.  The Mirroring Hypothesis: Theory, Evidence and Exceptions , 2016 .

[55]  Anna-Maria Rivas McGowan,et al.  Interdisciplinary Interactions During R&D and Early Design of Large Engineered Systems , 2014 .

[56]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[57]  Andrew M. Colman,et al.  The complexity of cooperation: Agent-based models of competition and collaboration , 1998, Complex..

[58]  Richard M. Crowder,et al.  The Development of an Agent-Based Modeling Framework for Simulating Engineering Team Work , 2012, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[59]  J.Steven Newman.  Failure-Space: A systems engineering look at 50 space system failures , 2001 .

[60]  Christopher McComb,et al.  Drawing Inspiration From Human Design Teams for Better Search and Optimization: The Heterogeneous Simulated Annealing Teams Algorithm , 2016 .

[61]  W. P. Rogers,et al.  Report of the Presidential Commission on the Space Shuttle Challenger Accident, Volume 1 , 1986 .

[62]  Steven D. Eppinger,et al.  The Misalignment of Product Architecture and Organizational Structure in Complex Product Development , 2004, Manag. Sci..

[63]  Manuel E. Sosa,et al.  Can Informal Communication Networks Disrupt Coordination in New Product Development Projects? , 2015, Organ. Sci..

[64]  Christopher McComb,et al.  Optimizing Design Teams Based on Problem Properties: Computational Team Simulations and an Applied Empirical Test , 2017 .

[65]  Angelika Mueller.  Communication And Gender , 2016 .

[66]  Mark E. J. Newman,et al.  Power-Law Distributions in Empirical Data , 2007, SIAM Rev..

[67]  Joaquim R. R. A. Martins,et al.  Multidisciplinary design optimization: A survey of architectures , 2013 .