Project dynamics and emergent complexity

This paper presents a theoretical analysis of project dynamics and emergent complexity in new product development (NPD) projects subjected to the management concept of concurrent engineering. To provide a comprehensive study, the complexity frameworks, theories and measures that have been developed in organizational theory, systematic engineering design and basic scientific research are reviewed. For the evaluation of emergent complexity in NPD projects, an information-theory quantity—termed “effective measure complexity” (EMC)—is selected from a variety of measures, because it can be derived from first principles and therefore has high construct validity. Furthermore, it can be calculated efficiently from dynamic generative models or purely from historical data, without intervening models. The EMC measures the mutual information between the infinite past and future histories of a stochastic process. According to this principle, it is particularly interesting to evaluate the time-dependent complexity in NPD and to uncover the relevant interactions. To obtain analytical results, a model-driven approach is taken and a vector autoregression (VAR) model of cooperative work is formulated. The formulated VAR model provided the foundation for the calculation of a closed-form solution of the EMC in the original state space. This solution can be used to analyze and optimize complexity based on the model’s independent parameters. Moreover, a transformation into the spectral basis is carried out to obtain more expressive solutions in matrix form. The matrix form allows identification of the surprisingly few essential parameters and calculation of two lower complexity bounds. The essential parameters include the eigenvalues of the work transformation matrix of the VAR model and the correlations between components of performance fluctuations.

[1]  Kai Yang,et al.  The components of complexity in engineering design , 1999 .

[2]  Thomas Gärtner,et al.  Optimization of Concurrent Engineering Projects using an Information-theoretic Complexity Metric , 2009 .

[3]  Yaneer Bar-Yam,et al.  The Statistical Mechanics of Complex Product Development: Empirical and Analytical Results , 2007, Manag. Sci..

[4]  Cosma Rohilla Shalizi,et al.  Blind Construction of Optimal Nonlinear Recursive Predictors for Discrete Sequences , 2004, UAI.

[5]  Kaushik Sinha,et al.  Spectral and Topological Features of ‘Real-World’ Product Structures , 2009 .

[6]  James L. Rogers,et al.  Development of a Genetic Algorithm to Automate Clustering of a Dependency Structure Matrix , 2013 .

[7]  D. Chalmers Strong and Weak Emergence , 2006 .

[8]  Pierre Duchesne,et al.  On modelling and diagnostic checking of vector periodic autoregressive time series models , 2009 .

[9]  Kim B. Clark,et al.  Design Rules: The Power of Modularity Volume 1 , 1999 .

[10]  Alexander Verbraeck,et al.  Grasping project complexity in large engineering projects: The TOE (Technical, Organizational and Environmental) framework , 2011 .

[11]  Wallace J. Hopp,et al.  The Impact of Misalignment of Organizational Structure and Product Architecture on Quality in Complex Product Development , 2010, Manag. Sci..

[12]  Kevin Murphy,et al.  Bayes net toolbox for Matlab , 1999 .

[13]  James P. Crutchfield,et al.  Computational Mechanics: Pattern and Prediction, Structure and Simplicity , 1999, ArXiv.

[14]  Helmut Ltkepohl,et al.  New Introduction to Multiple Time Series Analysis , 2007 .

[15]  Tyson R. Browning,et al.  Design Structure Matrix Methods and Applications , 2012 .

[16]  Young,et al.  Inferring statistical complexity. , 1989, Physical review letters.

[17]  Ming Li,et al.  Minimum description length induction, Bayesianism, and Kolmogorov complexity , 1999, IEEE Trans. Inf. Theory.

[18]  J. Norris Appendix: probability and measure , 1997 .

[19]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[20]  A. Griffin The Effect of Project and Process Characteristics on Product Development Cycle Time , 1997 .

[21]  Avraham Shtub,et al.  Project Management: Processes, Methodologies, and Economics , 1994 .

[22]  T. Mori,et al.  On the discrete Lyapunov matrix equation , 1982 .

[23]  Thomas Gärtner,et al.  A Complexity Measure for Concurrent Engineering Projects based on the DSM , 2008 .

[24]  James P. Crutchfield,et al.  Prediction, Retrodiction, and the Amount of Information Stored in the Present , 2009, ArXiv.

[25]  H. Akaike A new look at the statistical model identification , 1974 .

[26]  Jorma Rissanen,et al.  Fisher information and stochastic complexity , 1996, IEEE Trans. Inf. Theory.

[27]  Christopher M. Schlick,et al.  Organizational simulation of complex process engineering projects in the chemical industry , 2008, 2008 12th International Conference on Computer Supported Cooperative Work in Design.

[28]  D. V. Steward,et al.  The design structure system: A method for managing the design of complex systems , 1981, IEEE Transactions on Engineering Management.

[29]  A. U.S.,et al.  Predictability , Complexity , and Learning , 2002 .

[30]  Matthias Kreimeyer,et al.  Structural Metrics to Assess Processes , 2008 .

[31]  S. Duckwitz,et al.  A complexity measure for new product development projects , 2007, 2007 IEEE International Engineering Management Conference.

[32]  Harvey Maylor,et al.  Managerial Complexity in Project-Based Operations: A Grounded Model and Its Implications for Practice , 2008 .

[33]  M SchlickChristopher,et al.  Project dynamics and emergent complexity , 2013 .

[34]  Steven D. Eppinger,et al.  The Misalignment of Product Architecture and Organizational Structure in Complex Product Development , 2004, Manag. Sci..

[35]  James D. Herbsleb,et al.  Socio-technical congruence: a framework for assessing the impact of technical and work dependencies on software development productivity , 2008, ESEM '08.

[36]  Jorma Rissanen,et al.  Information and Complexity in Statistical Modeling , 2006, ITW.

[37]  N. N. Puri Fundamentals of Linear Systems for Physical Scientists and Engineers , 2009 .

[38]  Gerik Scheuermann,et al.  Knowledge Assisted Visualization: Steady visualization of the dynamics in fluids using ε-machines , 2009 .

[39]  JoAnne Yates,et al.  Life in the Trading Zone: Structuring Coordination Across Boundaries in Postbureaucratic Organizations , 2006, Organ. Sci..

[40]  Grégoire Nicolis,et al.  Foundations Of Complex Systems: Nonlinear Dynamics, Statistical Physics, Information And Prediction , 2007 .

[41]  Mohan V. Tatikonda,et al.  Technology novelty, project complexity, and product development project execution success: a deeper look at task uncertainty in product innovation , 2000, IEEE Trans. Engineering Management.

[42]  David Wilemon,et al.  Sources and assessment of complexity in NPD projects , 2003 .

[43]  Michael K. Tippett,et al.  Predictability: Recent insights from information theory , 2007 .

[44]  Philip Hans Franses,et al.  Periodic Time Series Models , 1996 .

[45]  Robert P. Smith,et al.  Identifying Controlling Features of Engineering Design Iteration , 2015 .

[46]  Douglas Lind,et al.  An Introduction to Symbolic Dynamics and Coding , 1995 .

[47]  P. Grünwald The Minimum Description Length Principle (Adaptive Computation and Machine Learning) , 2007 .

[48]  Elaine J. Weyuker,et al.  Evaluating Software Complexity Measures , 2010, IEEE Trans. Software Eng..

[49]  Wolfgang Koch,et al.  On Bayesian tracking and data Fusion: A tutorial introduction with examples , 2010, IEEE Aerospace and Electronic Systems Magazine.

[50]  Christopher M. Schlick,et al.  Self-generated complexity and human-machine interaction , 2006, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[51]  Udo Lindemann,et al.  Structural Complexity Management: An Approach for the Field of Product Design , 2008 .

[52]  Marcus Hutter,et al.  Algorithmic Information Theory , 1977, IBM J. Res. Dev..

[53]  Maik Maurer,et al.  Structural Awareness in Complex Product Design , 2007 .

[54]  Christoph H. Loch,et al.  Spiraling out of Control: Problem-Solving Dynamics in Complex Distributed Engineering Projects , 2006 .

[55]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[56]  Christoph H. Loch,et al.  Exchanging Preliminary Information in Concurrent Engineering: Alternative Coordination Strategies , 2002, Organ. Sci..

[57]  R. I. Winner,et al.  The Role of Concurrent Engineering in Weapons System Acquisition , 1988 .

[58]  F. Mandl,et al.  Statistical Physics, 2nd Edition , 1988 .

[59]  Sebastian Schneider,et al.  Modelling of cooperative work in concurrent engineering projects based on extended work transformation matrices with hidden state variables , 2012 .

[60]  C. Krattenthaler Advanced Determinant Calculus: A Complement , 2005, math/0503507.

[61]  and Charles K. Taft Reswick,et al.  Introduction to Dynamic Systems , 1967 .

[62]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[63]  Tyson R. Browning,et al.  Managing complex product development projects with design structure matrices and domain mapping matrices , 2007 .

[64]  Robert Shaw,et al.  The Dripping Faucet As A Model Chaotic System , 1984 .

[65]  Mikhail Prokopenko,et al.  An information-theoretic primer on complexity, self-organization, and emergence , 2009, Complex..

[66]  Suh Nam-pyo,et al.  Complexity: Theory and Applications , 2005 .

[67]  H. Hotelling The most predictable criterion. , 1935 .

[68]  Bernardo A. Huberman,et al.  Performance Variability and Project Dynamics , 2005, Comput. Math. Organ. Theory.

[69]  J. Alberto Espinosa,et al.  Crossing Spatial and Temporal Boundaries in Globally Distributed Projects: A Relational Model of Coordination Delay , 2009, Inf. Syst. Res..

[70]  Jorma Rissanen,et al.  Stochastic Complexity in Statistical Inquiry , 1989, World Scientific Series in Computer Science.

[71]  Wentian Li,et al.  On the Relationship between Complexity and Entropy for Markov Chains and Regular Languages , 1991, Complex Syst..

[72]  Cosma Rohilla Shalizi Optimal Nonlinear Prediction of Random Fields on Networks , 2003, DMCS.

[73]  Arnold Neumaier,et al.  Estimation of parameters and eigenmodes of multivariate autoregressive models , 2001, TOMS.

[74]  Mark Mortensen,et al.  Go (Con)Figure: Subgroups, Imbalance, and Isolates in Geographically Dispersed Teams , 2009, Organ. Sci..

[75]  James P. Crutchfield,et al.  Synchronization and Control in Intrinsic and Designed Computation: An Information-Theoretic Analysis of Competing Models of Stochastic Computation , 2010, Chaos.

[76]  Christoph H. Loch,et al.  Hierarchical Structure and Search in Complex Organizations , 2010, Manag. Sci..

[77]  Christopher M. Schlick,et al.  Activity- and actor-oriented simulation approach for the management of development projects , 2010, Int. J. Comput. Aided Eng. Technol..

[78]  Manuel E. Sosa A structured approach to predicting and managing technical interactions in software development , 2008 .

[79]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[80]  Marina Diakonova,et al.  Quantifying Emergence in Terms of Persistent Mutual Information , 2010, Adv. Complex Syst..

[81]  Katja Hölttä-Otto,et al.  Estimating factors affecting project task size in product development-an empirical study , 2006, IEEE Transactions on Engineering Management.

[82]  Jami J. Shah,et al.  Mechanical Engineering Design Complexity Metrics: Size, Coupling, and Solvability , 2010 .

[83]  Christoph H. Loch,et al.  Problem - Solving Oscillations in Complex Engineering Projects , 2003, Manag. Sci..

[84]  Jan W. Rivkin,et al.  Patterned Interactions in Complex Systems: Implications for Exploration , 2007, Manag. Sci..

[85]  Richard A. Davis,et al.  Time Series: Theory and Methods , 2013 .

[86]  Gregory. J. Chaitin,et al.  Algorithmic information theory , 1987, Cambridge tracts in theoretical computer science.

[87]  Cosma Rohilla Shalizi,et al.  Methods and Techniques of Complex Systems Science: An Overview , 2003, nlin/0307015.

[88]  C. Jordan Advanced Determinant Calculus: a Complement , 2008 .

[89]  J. Crutchfield,et al.  Regularities unseen, randomness observed: levels of entropy convergence. , 2001, Chaos.

[90]  Some background on information theory , 2008 .

[91]  P. Murmann Expected development time reductions in the German mechanical engineering industry , 1994 .

[92]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[93]  Oded Maimon,et al.  The measurement of a design structural and functional complexity , 1998, IEEE Trans. Syst. Man Cybern. Part A.

[94]  Kaushik Sinha,et al.  Technology Insertion in Turbofan Engine and assessment of Architectural Complexity , 2011 .

[95]  H. Lütkepohl COMPARISON OF CRITERIA FOR ESTIMATING THE ORDER OF A VECTOR AUTOREGRESSIVE PROCESS , 1985 .

[96]  F. R. Gantmakher The Theory of Matrices , 1984 .

[97]  Jeffrey K. Liker,et al.  The Toyota way : 14 management principles from the world's greatest manufacturer , 2004 .

[98]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[99]  U. Lindemann,et al.  Structural Complexity Management , 2009 .

[100]  Steven D. Eppinger,et al.  Methods for Analyzing Design Procedures , 2011 .

[101]  Zoubin Ghahramani,et al.  An Introduction to Hidden Markov Models and Bayesian Networks , 2001, Int. J. Pattern Recognit. Artif. Intell..

[102]  James P. Crutchfield,et al.  Computation at the Onset of Chaos , 1991 .

[103]  Paul R. Carlile,et al.  A Pragmatic View of Knowledge and Boundaries: Boundary Objects in New Product Development , 2002, Organ. Sci..

[104]  Luis A. Nunes Amaral,et al.  Complex Systems - A New Paradigm for the Integrative Study of Management, Physical, and Technological Systems , 2007, Manag. Sci..

[105]  Geoffrey E. Hinton,et al.  Parameter estimation for linear dynamical systems , 1996 .

[106]  Dirk V. Arnold,et al.  Information-theoretic Analysis of Phase Transitions , 1996, Complex Syst..

[107]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[108]  Ralf Der,et al.  Information-driven self-organization: the dynamical system approach to autonomous robot behavior , 2011, Theory in Biosciences.

[109]  H. Akaike Autoregressive model fitting for control , 1971 .

[110]  Kim B. Clark,et al.  Design Rules: The Power of Modularity , 2000 .

[111]  K. D. Cock Principal Angles in System Theory, Information Theory and Signal Processing , 2002 .

[112]  Chrystopher L. Nehaniv,et al.  Relevant information in optimized persistence vs. progeny strategies , 2006 .

[113]  Jan W. Rivkin,et al.  Balancing Search and Stability: Interdependencies Among Elements of Organizational Design , 2003, Manag. Sci..

[114]  Bin Yu,et al.  Model Selection and the Principle of Minimum Description Length , 2001 .

[115]  Sebastian Schneider,et al.  Modeling of Periodically Correlated Work Processes in Large-Scale Concurrent Engineering Projects Based on the DSM , 2011 .

[116]  Christopher M. Schlick,et al.  Complexity assessment of human–computer interaction , 2010 .

[117]  Peter Lancaster,et al.  The theory of matrices , 1969 .

[118]  Jami J. Shah,et al.  Developing Measures of Complexity for Engineering Design , 2003 .

[119]  L. Arnold,et al.  Stationary solutions of linear systems with additive and multiplicative noise , 1982 .

[120]  David Wilemon,et al.  An empirical investigation of complexity and its management in new product development , 2009, Technol. Anal. Strateg. Manag..

[121]  Zoubin Ghahramani,et al.  Learning Nonlinear Dynamical Systems Using an EM Algorithm , 1998, NIPS.

[122]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[123]  Jorma Rissanen,et al.  Minimum Description Length Principle , 2010, Encyclopedia of Machine Learning.

[124]  S. Griffies,et al.  A Conceptual Framework for Predictability Studies , 1999 .

[125]  Arnold Neumaier,et al.  Algorithm 808: ARfit—a matlab package for the estimation of parameters and eigenmodes of multivariate autoregressive models , 2001, TOMS.

[126]  James D. Herbsleb,et al.  Identification of coordination requirements: implications for the Design of collaboration and awareness tools , 2006, CSCW '06.

[127]  Tyson R. Browning,et al.  Applying the design structure matrix to system decomposition and integration problems: a review and new directions , 2001, IEEE Trans. Engineering Management.

[128]  P. Grassberger Toward a quantitative theory of self-generated complexity , 1986 .

[129]  Mohamed Lebcir,et al.  Impact of project complexity factors on new product development cycle time , 2011 .