Learning from mixed OR method practice : the NINES case study

Despite continued interest in the use of mixed OR/MS methods, limited attention has been paid in the literature to generic lessons that could be gained from mixing methods in practice. Many organisational problems demand the use of a mixed method approach and thus recognising and sharing lessons could prove beneficial to both practitioners and researchers. This paper reports on an in-depth evaluation of a case study involving risk identification and quantification of the Northern Isles New Energy Solutions (NINES) project which sought to trial and plan a new energy system. The intervention involved a mixed method approach and client feedback on the efficacy of the approach was sought. The evaluation reported in this paper is carried out using a set of themes taken from the literature and seeks to highlight transferable lessons. The set of lessons that emerge are presented along with their implications for both general OR modelling practice and the specific situation of mixing OR/MS methods. The paper concludes by discussing the implications of the work and directions for future work which will be of interest to both practitioners and researchers interested in mixed method OR/MS work.

[1]  Jonathan Rosenhead,et al.  Rational Analysis for a Problematic World Revisited , 2001 .

[2]  Fran Ackermann Teaching Soft OR/MS Methodologies: What, Why, and How , 2015 .

[3]  Martin Skitmore,et al.  Project management turnover: causes and effects on project performance , 2005 .

[4]  John Quigley,et al.  Systemic risk elicitation: Using causal maps to engage stakeholders and build a comprehensive view of risks , 2014, Eur. J. Oper. Res..

[5]  Fran Ackermann,et al.  Mixing OR methods in practice: Past, present and future directions , 2011, Eur. J. Oper. Res..

[6]  W. Orlikowski Using Technology and Constituting Structures: A Practice Lens for Studying Technology in Organizations , 2000 .

[7]  J Pollack,et al.  Multimethodology in series and parallel: strategic planning using hard and soft OR , 2009, J. Oper. Res. Soc..

[8]  B MilesMatthew,et al.  Qualitative Data Analysis , 2009, Approaches and Processes of Social Science Research.

[9]  Richard J. Ormerod,et al.  The mangle of OR practice: towards more informative case studies of ‘technical’ projects , 2014, J. Oper. Res. Soc..

[10]  C. Eden,et al.  The analysis of cause maps , 1992 .

[11]  G. Susman,et al.  An Assessment of the Scientific Merits of Action Research. , 1978 .

[12]  L. Phillips,et al.  Faciliated Work Groups: Theory and Practice , 1993 .

[13]  L. Franco,et al.  Understanding multi-methodology: Evaluating the perceived impact of mixing methods for group budgetary decisions , 2011 .

[14]  Wanda J. Orlikowski,et al.  Using Technology and Constituting Structures: A Practice Lens for Studying Technology in Organizations , 2000, Theory in CSCW.

[15]  Valerie Belton,et al.  A toolkit of designs for mixing Discrete Event Simulation and System Dynamics , 2017, Eur. J. Oper. Res..

[16]  Colin Eden,et al.  Use of 'soft-OR' models by clients - what do they want from them? , 2004 .

[17]  Michael Pidd,et al.  Systems Modelling: Theory and Practice , 2004 .

[18]  John Mingers,et al.  Combining PSMs with hard OR methods: the philosophical and practical challenges , 2006, J. Oper. Res. Soc..

[19]  Richard J. Ormerod,et al.  Information Systems Strategy Development at Sainsbury's Supermarkets Using Soft OR , 1996 .

[20]  John Mingers,et al.  A classification of the philosophical assumptions of management science methods , 2003, J. Oper. Res. Soc..

[21]  Peter Bennett,et al.  On Linking Approaches to Decision-Aiding: Issues and Prospects , 1985 .

[22]  John Mingers,et al.  Multimethodology: Towards a framework for mixing methodologies , 1997 .

[23]  Jesper Simonsen,et al.  Soft Systems Methodology , 2020, Systems Approaches to Making Change: A Practical Guide.

[24]  John Mingers,et al.  Multi-Paradigm Multimethodology , 1997 .

[25]  M. Hatch,et al.  Living With Multiple Paradigms the Case of Paradigm Interplay in Organizational Culture Studies , 1996 .

[26]  Michael Jackson Towards coherent pluralism in management science , 1999, J. Oper. Res. Soc..

[27]  Ann Taket,et al.  Creative Problem Solving: Total Systems Intervention. , 1991 .

[28]  T. Rickards Sense making in organizations , 1997 .

[29]  George P. Richardson,et al.  ScriptsMap: A tool for designing multi-method policy-making workshops , 2011 .

[30]  Mark Lauby,et al.  Reliability considerations from the integration of smart grid , 2012, 2012 IEEE PES Innovative Smart Grid Technologies (ISGT).

[31]  Richard J. Ormerod,et al.  Critical rationalism in practice: Strategies to manage subjectivity in OR investigations , 2014, Eur. J. Oper. Res..

[32]  Thomas B. Lawrence,et al.  Researching organizations using action research , 2006 .

[33]  C. Huxham,et al.  From many to one--and back. An exploration of some components of facilitation , 1994 .

[34]  Zhichang Zhu,et al.  After paradim: why mixing-methodology theorising fails and how to make it work again , 2011, J. Oper. Res. Soc..

[35]  George P. Richardson,et al.  Integrating modes of policy analysis and strategic management practice: requisite elements and dilemmas , 2009, J. Oper. Res. Soc..

[36]  Colin Eden,et al.  Tackling Strategic Problems: The Role of Group Decision Support , 1991 .

[37]  Susan Howick,et al.  Supporting strategic conversations: the significance of a quantitative model building process , 2011, J. Oper. Res. Soc..

[38]  John Mingers,et al.  Multimethodology : the theory and practice of combining management science methodologies , 1997 .

[39]  Robert Y. Cavana,et al.  Towards a new framework for evaluating systemic problem structuring methods , 2013, Eur. J. Oper. Res..

[40]  Andrew Pickering,et al.  The mangle of practice : time, agency, and science , 1997 .

[41]  T. Bedford,et al.  Expert elicitation for reliable system design , 2006, 0708.0279.

[42]  Colin Eden,et al.  On evaluating the performance of 'wide-band' GDSS's , 1995 .

[43]  Tom Houghton,et al.  Modelling systemic risks to inform a repowering decision , 2013 .

[44]  Fran Ackermann,et al.  Getting "Messy" with Problems: The Challenges of Teaching "Soft" OR , 2011, INFORMS Trans. Educ..

[45]  L. Alberto Franco,et al.  Unpacking multimethodology: Impacts of a community development intervention , 2016, Eur. J. Oper. Res..

[46]  George P. Richardson,et al.  Scripts for group model building , 1997 .

[47]  David W. Wainwright,et al.  SSM and technology management: Developing multimethodology through practice , 2014, Eur. J. Oper. Res..

[48]  John Mingers,et al.  Variety is the spice of life: combining soft and hard OR/MS methods , 2000 .

[49]  Fran Ackermann,et al.  Problem structuring methods 'in the Dock': Arguing the case for Soft OR , 2012, Eur. J. Oper. Res..

[50]  K. Seers Qualitative data analysis , 2011, Evidence Based Nursing.

[51]  Fran Ackermann,et al.  Building confidence in models for multiple audiences: The modelling cascade , 2008, Eur. J. Oper. Res..

[52]  John Mingers,et al.  Multimethodology: Mixing and Matching Methods , 2001 .

[53]  Colin Eden,et al.  Strategic Options Development and Analysis , 2010, Systems Approaches to Making Change: A Practical Guide.

[54]  R. Bostrom,et al.  Group Facilitation and Group Support Systems , 1993 .

[55]  K. Weick FROM SENSEMAKING IN ORGANIZATIONS , 2021, The New Economic Sociology.

[56]  Katharina Burger,et al.  Understanding PSM Interventions Through Sense-Making and the Mangle of Practice Lens , 2015, GDN.