How stock-flow failure and general cognitive ability impact performance in operational dynamic control tasks

Abstract Previous research has highlighted the importance of individual factors predicting human beings’ relatively poor performance in controlling dynamic operational systems. Taking an industrial and organizational (IO) psychology perspective within a behavioral operations research context, this study focuses on erroneous stock-flow thinking and general cognitive ability. It uses data from laboratory experiments to test a mediating structural equation model firmly grounded in IO psychology theory, proposing both a direct concave impact of general cognitive ability on control task performance and an indirect concave effect mediated by stock-flow thinking. With our results we are the first to demonstrate that stock-flow thinking serves as a mediator of general cognitive ability on performance, while at the same time, general cognitive ability non-linearly impacts performance. These findings contribute to theory building in behavioral operational research and demonstrate how adoption of perspectives other than the traditional paradigms used can help. It also has practical implications as it improves the relative assessment of different levers to increase operational control performance. We find that the total impact of general cognitive ability on control performance is higher than the direct effect of stock-flow thinking. Therefore, companies should add stock-flow thinking tests to their personnel selection toolbox, but put more weight on selecting operational managers with high cognitive abilities and somewhat less weight on training efforts that aim to reduce stock-flow failures.

[1]  R. Cattell.  Intelligence : its structure, growth and action , 1987 .

[2]  Detmar W. Straub,et al.  Common Beliefs and Reality About PLS , 2014 .

[3]  Arunachalam Narayanan,et al.  Decision Making and Cognition in Multi‐Echelon Supply Chains: An Experimental Study , 2015 .

[4]  J. Sterman,et al.  Order Stability in Supply Chains: Coordination Risk and the Role of Coordination Stock , 2004 .

[5]  B. Tabachnick,et al.  Using Multivariate Statistics , 1983 .

[6]  P. Costa,et al.  Revised NEO Personality Inventory (NEO-PI-R) and NEO-Five-Factor Inventory (NEO-FFI) , 1992 .

[7]  Robert Hogan,et al.  In Defense of Personality Measurement: New Wine for Old Whiners , 2005 .

[8]  Mark Paich,et al.  Boom, bust, and failures to learn in experimental markets , 1993 .

[9]  Cleotilde González,et al.  Why don ’ t well-educated adults understand accumulation ? A challenge to researchers , educators , and citizens , 2008 .

[10]  E. Olsen,et al.  Uranus: Variability of the Microwave Spectrum , 1982, Science.

[11]  Y. Trope,et al.  The effect of temporal distance on level of mental construal. , 2002 .

[12]  Matthias Holweg,et al.  Supply chain simulation - a tool for education, enhancement and endeavour , 2002 .

[13]  Rachel Croson,et al.  IMPACT OF POS DATA SHARING ON SUPPLY CHAIN MANAGEMENT: AN EXPERIMENTAL STUDY , 2003 .

[14]  J. Strohhecker.  A Pilot Study for Testing the Effect of Stock and Flow Thinking on Stock and Flow Management Performance , 2009 .

[15]  J. Sterman.  Misperceptions of feedback in dynamic decision making , 1989 .

[16]  H. Nyborg,et al.  The stability of general intelligence from early adulthood to middle-age , 2008 .

[17]  Karen Donohue,et al.  Behavioral Causes of the Bullwhip Effect and the Observed Value of Inventory Information , 2006, Manag. Sci..

[18]  C. Spearman.  General intelligence Objectively Determined and Measured , 1904 .

[19]  M. K. Thomas,et al.  Monetary incentives versus class credit: Evidence from a large classroom trust experiment , 2014 .

[20]  John D. Sterman,et al.  System Dynamics: Systems Thinking and Modeling for a Complex World , 2002 .

[21]  R. Cattell.  Theory of fluid and crystallized intelligence: A critical experiment. , 1963 .

[22]  J. Sterman,et al.  Understanding public complacency about climate change: adults’ mental models of climate change violate conservation of matter , 2007 .

[23]  Robert M. O'Keefe,et al.  Experimental behavioural research in operational research: What we know and what we might come to know , 2016, Eur. J. Oper. Res..

[24]  Bo Hu,et al.  Strategy map concepts in a balanced scorecard cockpit improve performance , 2017, Eur. J. Oper. Res..

[25]  Scott B. MacKenzie,et al.  Common method biases in behavioral research: a critical review of the literature and recommended remedies. , 2003, The Journal of applied psychology.

[26]  J. Forrester.  Industrial Dynamics: A Major Breakthrough for Decision Makers , 2012 .

[27]  D. Sterman,et al.  Misperceptions of Feedback in a Dynamic Decision Making Experiment , 1989 .

[28]  Geoffrey S. Hubona,et al.  Using PLS path modeling in new technology research: updated guidelines , 2016, Ind. Manag. Data Syst..

[29]  Jürgen Strohhecker,et al.  Factors influencing strategy implementation decisions: an evaluation of a balanced scorecard cockpit, intelligence, and knowledge , 2015, Journal of Management Control.

[30]  P. Mauger,et al.  Long-term predictive validity of the Scholastic Aptitude Test. , 1975 .

[31]  S. Frederick.  Journal of Economic Perspectives—Volume 19, Number 4—Fall 2005—Pages 25–42 Cognitive Reflection and Decision Making , 2022 .

[32]  Kishore Sengupta,et al.  ANATOMY OF A DECISION TRAP IN COMPLEX NEW PRODUCT DEVELOPMENT PROJECTS , 2013 .

[33]  D. Detterman,et al.  Scholastic Assessment or g? , 2004, Psychological science.

[34]  J. L. Holland,et al.  Making vocational choices : a theory of vocational personalities and work environments , 1984 .

[35]  P. Ackerman.  A theory of adult intellectual development: Process, personality, interests, and knowledge , 1996 .

[36]  John D. Sterman,et al.  Does formal system dynamics training improve people's understanding of accumulation? , 2010 .

[37]  Andreas Größler,et al.  System Dynamics as a Structural Theory in Operations Management , 2008 .

[38]  John Brocklesby,et al.  The what, the why and the how of behavioural operational research - An invitation to potential sceptics , 2016, Eur. J. Oper. Res..

[39]  Colin Camerer,et al.  The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework , 1999 .

[40]  John D. Sterman,et al.  All models are wrong: reflections on becoming a systems scientist† , 2002 .

[41]  Daniel Kainz,et al.  Can Students Learn Stock-Flow-Thinking? An Empirical Investigation , 2002 .

[42]  Cleotilde Gonzalez,et al.  How analytic reasoning style and global thinking relate to understanding stocks and flows , 2015 .

[43]  Jamie B. Kruse,et al.  A comparison of salient rewards in experiments: money and class points , 2001 .

[44]  C. Spearman.  The Abilities of Man their Nature and Measurement , 2020, Nature.

[45]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[46]  Rudolf R. Sinkovics,et al.  The Use of Partial Least Squares Path Modeling in International Marketing , 2009 .

[47]  J. Sterman,et al.  Cloudy Skies: Assessing Public Understanding of Global Warming , 2002 .

[48]  Saul Geiser,et al.  UC and the SAT: Predictive Validity and Differential Impact of the SAT I and SAT II at the University of California , 2001 .

[49]  K. Donohue,et al.  Individual differences in the newsvendor problem: Behavior and cognitive reflection , 2013 .

[50]  Erik Demeulemeester,et al.  Personnel scheduling: A literature review , 2013, Eur. J. Oper. Res..

[51]  William M. K. Trochim,et al.  Research methods knowledge base , 2001 .

[52]  Jürgen Strohhecker,et al.  Implementing Sustainable Business Strategies , 2012 .

[53]  J. Sterman,et al.  The impact of endogenous demand on push–pull production systems , 2005 .

[54]  J. Carroll.  Human Cognitive Abilities-a sur-vey of factor-analytic studies , 1993 .

[55]  Gary E. Bolton,et al.  Learning-by-Doing in the Newsvendor Problem: A Laboratory Investigation of the Role of Experience and Feedback , 2008, Manuf. Serv. Oper. Manag..

[56]  Raimo P. Hämäläinen,et al.  On the importance of behavioral operational research: The case of understanding and communicating about dynamic systems , 2013, Eur. J. Oper. Res..

[57]  D. Morrice,et al.  The “physics” of capacity and backlog management in service and custom manufacturing supply chains , 2005 .

[58]  R. G. Coyle.  The technical elements of the system dynamics approach , 1983 .

[59]  Heinz Holling,et al.  Spearman's law of diminishing returns. A meta-analysis , 2017 .

[60]  A. Furnham.  Personality and Intelligence at Work: Exploring and Explaining Individual Differences at Work , 2008 .

[61]  J. Holland.  A personality inventory employing occupational titles. , 1958 .

[62]  Y. Ganzach,et al.  General mental ability and pay: Nonlinear effects. , 2013 .

[63]  A. J. Vinchur,et al.  A History of Personnel Selection and Assessment , 2012 .

[64]  Jonas W. B. Lang,et al.  Specific Abilities in the Workplace: More Important Than g? , 2017, Journal of Intelligence.

[65]  J. Strohhecker,et al.  Do personal traits influence inventory management performance?—The case of intelligence, personality, interest and knowledge , 2013 .

[66]  A. Tversky,et al.  The framing of decisions and the psychology of choice. , 1981, Science.

[67]  Werner W. Wittmann,et al.  The relationship between performance in dynamic systems and intelligence , 2004 .

[68]  Andreas Größler,et al.  Don't let history repeat itself—methodological issues concerning the use of simulators in teaching and experimentation , 2004 .

[69]  Marko Sarstedt,et al.  PLS-SEM: Indeed a Silver Bullet , 2011 .

[70]  B. Brehmer.  Dynamic decision making: human control of complex systems. , 1992, Acta psychologica.

[71]  Kenneth P. Yusko,et al.  Intelligence 2.0: Reestablishing a Research Program on g in I–O Psychology , 2012, Industrial and Organizational Psychology.

[72]  In-Sue Oh.  A Call for More Research on the Relationship Between Intelligence and Job Performance: Non-Task Performance, Non-Euro-American Contexts, and the Science-Practice Gap , 2014 .

[73]  Susanne M. Jaeggi,et al.  Challenges for Research on Intelligence , 2013 .

[74]  D. Saccuzzo,et al.  Psychological Testing: Principles, Applications, and Issues , 1982 .

[75]  T. Coyle.  Relations among general intelligence (g), aptitude tests, and GPA: Linear effects dominate , 2015 .

[76]  T. Judge,et al.  IN SUPPORT OF PERSONALITY ASSESSMENT IN ORGANIZATIONAL SETTINGS , 2007 .

[77]  John D. Sterman,et al.  Learning in and about complex systems , 1994 .

[78]  J. Sterman.  Risk Communication on Climate: Mental Models and Mass Balance , 2008, Science.

[79]  Malcolm James Ree,et al.  Predicting job performance: Not much more than g.. , 1994 .

[80]  D. Wechsler,et al.  Wechsler Adult Intelligence Scale—Fourth Edition (WAIS-IV) , 2010 .

[81]  Benedikt Hell,et al.  Eine Metaanalyse der Validität von fachspezifischen Studierfähigkeitstests im deutschsprachigen Raum , 2007 .

[82]  Rachel T. A. Croson,et al.  Upstream versus downstream information and its impact on the bullwhip effect , 2005 .

[83]  J. Miner.  Organizational Behavior 4: From Theory to Practice , 2005 .

[84]  Corwin Boake,et al.  Genes Determine Stability and the Environment Determines Change in Cognitive Ability During 35 Years of Adulthood , 2009, Psychological science.

[85]  Enno Siemsen,et al.  Judgmental Forecasting: Cognitive Reflection and Decision Speed , 2014 .

[86]  John Maule,et al.  What lies beneath: Reframing framing effects , 2007, Thinking & Reasoning.

[87]  F. Schmidt,et al.  The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. , 1998 .

[88]  John D. Sterman,et al.  Bathtub dynamics: initial results of a systems thinking inventory , 2000 .

[89]  Thomas S. Redick,et al.  Working Memory Training May Increase Working Memory Capacity but Not Fluid Intelligence , 2013, Psychological science.

[90]  J. Sterman,et al.  Effects of feedback complexity on dynamic decision making , 1995 .

[91]  Neil Anderson,et al.  A meta-analytic study of general mental ability validity for different occupations in the European community. , 2003, The Journal of applied psychology.

[92]  Jürgen Strohhecker,et al.  Performance in Tangible and in Cognitive Stock-Flow Tasks , 2015 .

[93]  Lisa M. Penney,et al.  A review of personality and performance: Identifying boundaries, contingencies, and future research directions , 2011 .

[94]  Kai Helge Becker,et al.  An outlook on behavioural OR - Three tasks, three pitfalls, one definition , 2016, Eur. J. Oper. Res..

[95]  K. McGrew.  CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research , 2009 .

[96]  D. Frisch.  Reasons for framing effects. , 1993 .