Recent evidence on the effectiveness of group model building

Group model building (GMB) is a participatory approach to using system dynamics in group decision-making and problem structuring. This paper considers the published quantitative evidence base for GMB since the earlier literature review by Rouwette et al. (2002), to consider the level of understanding on three basic questions: what does it achieve, when should it be applied, and how should it be applied or improved? There have now been at least 45 such studies since 1987, utilising controlled experiments, field experiments, pretest/posttest, and observational research designs. There is evidence of GMB achieving a range of outcomes, particularly with regard to the behaviour of participants and their learning through the process. There is some evidence that GMB is more effective at supporting communication and consensus than traditional facilitation, however GMB has not been compared to other problem structuring methods. GMB has been successfully applied in a range of contexts, but there is little evidence on which to select between different GMB tools, or to understand when certain tools may be more appropriate. There is improving evidence on how GMB works, but this has not yet been translated into changing practice. Overall the evidence base for GMB has continued to improve, supporting its use for improving communication and agreement between participants in group decision processes. This paper argues that future research in group model building would benefit from three main shifts: from single cases to multiple cases; from controlled settings to applied settings; and by augmenting survey results with more objective measures.

[1]  João Ferreira Dias,et al.  A system dynamics‐based simulation experiment for testing mental model and performance effects of using the balanced scorecard , 2009 .

[2]  K. Stave,et al.  Group Model Building Wins: The results of a comparative analysis , 2008 .

[3]  Paul A. Langley,et al.  Performance and learning in a simulation of oil industry dynamics , 2004, Eur. J. Oper. Res..

[4]  E.A.J.A. Rouwette,et al.  Quantification and Evaluation Issues in Group Model Building: An Application to Human Resource Management Transition , 2001 .

[5]  Duncan Shaw,et al.  Evaluating electronic workshops through analysing the ‘brainstormed’ ideas , 2003, J. Oper. Res. Soc..

[6]  Kristan Cockerill,et al.  Cooperative modeling : Building bridges between science and the public , 2006 .

[7]  A. Diduck,et al.  An Assessment of Stakeholder Advisory Committees in Forest Management: Case Studies from Manitoba, Canada , 2006 .

[8]  D. Andersen,et al.  Using Visual Representations as Boundary Objects to Resolve Conflict in Collaborative Model-Building Approaches , 2012 .

[9]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[10]  George P. Richardson,et al.  Group model building: problem structuring, policy simulation and decision support , 2007, J. Oper. Res. Soc..

[11]  Etiënne A. J. A. Rouwette,et al.  Group Model Building , 2009, Encyclopedia of Complexity and Systems Science.

[12]  J. Levine,et al.  Progress in Small Group Research , 1990 .

[13]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[14]  Jac A. M. Vennix,et al.  On Evaluating the Performance of Problem Structuring Methods: An Attempt at Formulating a Conceptual Model , 2009 .

[15]  J.A.M. Vennix,et al.  Group model building: what does the client think of it now? , 2000 .

[16]  George P. Richardson,et al.  Using a Group Decision Support System to add Value to Group Model Building , 2010 .

[17]  Rogelio Oliva,et al.  The greater whole: Towards a synthesis of system dynamics and soft systems methodology , 1998, Eur. J. Oper. Res..

[18]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[19]  Kathleen E. Halvorsen,et al.  Assessing Public Participation Techniques for Comfort, Convenience, Satisfaction, and Deliberation , 2001, Environmental management.

[20]  Michael Cole,et al.  Evaluating the impact of community appraisals: some lessons from south-west England , 2006 .

[21]  Inge Bleijenbergh,et al.  Group model building evaluation in single cases: a method to assess changes in mental models , 2009 .

[22]  Andreas Größler,et al.  System Dynamics Projects That Failed to Make an Impact , 2007 .

[23]  Donald Cameron,et al.  Interpersonal success factors for strategy implementation: a case study using group model building , 2015, J. Oper. Res. Soc..

[24]  B. Kamler Rethinking doctoral publication practices: writing from and beyond the thesis , 2008 .

[25]  Jac A. M. Vennix,et al.  Group model building effectiveness: a review of assessment studies † , 2002 .

[26]  John Mingers,et al.  The use of multimethodology in practice—results of a survey of practitioners , 2002, J. Oper. Res. Soc..

[27]  Birgit Kopainsky,et al.  Automated assessment of learners' understanding in complex dynamic systems , 2012 .

[28]  K. Maani,et al.  Links between systems thinking and complex decision making , 2004 .

[29]  T. Jick Mixing Qualitative and Quantitative Methods: Triangulation in Action. , 1979 .

[30]  Jackie Phahlamohlaka,et al.  Community planning for rural education in South Africa , 2004, Eur. J. Oper. Res..

[31]  Etiënne A. J. A. Rouwette,et al.  A Comparison of Group Model Building and Strategic Options Development and Analysis , 2011 .

[32]  James K. Doyle The cognitive psychology of systems thinking , 1997 .

[33]  Aimé Heene,et al.  Learning from playing with microworlds in policy making: An experimental evaluation in fisheries management , 2012, Comput. Hum. Behav..

[34]  René Victor Valqui Vidal,et al.  Using soft OR in a small company--The case of Kirby , 2004, Eur. J. Oper. Res..

[35]  Michael Jackson,et al.  Towards a System of Systems Methodologies , 1984 .

[36]  Anthony M. Cresswell,et al.  Anatomy of a group model‐building intervention: building dynamic theory from case study research , 2006 .

[37]  Barbara Wilmot Creative Problem Solving and Red Yarn. , 1985 .

[38]  Robert Y. Cavana,et al.  Applied Business research: Qualitative and Quantitative Methods , 2001 .

[39]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[40]  Steven A. Cavaleri,et al.  Evaluating the Performance Efficacy of Systems Thinking Tools , 2002 .

[41]  Miroljub Kljajić,et al.  Group exploration of system dynamics models—is there a place for a feedback loop in the decision process? , 2003 .

[42]  Nuno Videira,et al.  Mapping Maritime Sustainability Issues with Stakeholder Groups , 2012 .

[43]  Cisca Joldersma,et al.  The impact of soft OR-methods on problem structuring , 2004, Eur. J. Oper. Res..

[44]  Jac A. M. Vennix,et al.  Effectiveness of group model building in discovering hidden profiles in strategic decision-making , 2008 .

[45]  H. Weisberg,et al.  Social Experimentation: A Method for Planning and Evaluating Social Intervention. , 1975 .

[46]  Etiënne A. J. A. Rouwette,et al.  Simulating urban transformation in Haaglanden, the Netherlands , 2009 .

[47]  Geoff Coyle,et al.  Qualitative and quantitative modelling in system dynamics: some research questions , 2000 .

[48]  George P. Richardson,et al.  Best practices in system dynamics modeling , 2013 .

[49]  Miroljub Kljajić,et al.  The Role of Information Feedback in the Management Group Decision-Making Process Applying System Dynamics Models , 2007 .

[50]  Jac A. M. Vennix,et al.  Group model-building: tackling messy problems , 1999 .

[51]  George P. Richardson,et al.  Group Model-Building 'Scripts' as a Collaborative Planning Tool , 2012 .

[52]  George P. Richardson,et al.  Model-building for group decision support: Issues and alternatives in knowledge elicitation , 1992 .

[53]  G. Rowe,et al.  Evaluating Public-Participation Exercises: A Research Agenda , 2004 .

[54]  Simon Peck,et al.  Group Model Building: Facilitating Team Learning Using System Dynamics , 1996, J. Oper. Res. Soc..

[55]  M. Greenberger,et al.  Models in the Policy Process: Public Decision Making in the Computer Era , 1976 .

[56]  Edward Lumsdaine,et al.  Creative problem solving , 1995 .

[57]  Laura J. Black,et al.  When visuals are boundary objects in system dynamics work , 2013 .

[58]  N. Blaikie Approaches to Social Enquiry , 1993 .

[59]  Steffen Bayer,et al.  Business dynamics: Systems thinking and modeling for a complex world , 2004 .

[60]  Daniel Kahneman,et al.  Availability: A heuristic for judging frequency and probability , 1973 .

[61]  L. Alberto Franco,et al.  Assessing the impact of problem structuring methods in multi-organizational settings: an empirical investigation , 2007, J. Oper. Res. Soc..

[62]  George P. Richardson,et al.  Scripts for group model building , 1997 .

[63]  Miroljub Kljajić,et al.  The relevance of facilitation in group decision making supported by a simulation model , 2011 .

[64]  Charles C. Ragin,et al.  Fuzzy-Set Social Science , 2001 .

[65]  L. Duram,et al.  Insights and Applications Assessing Public Participation in U.S. Watershed Planning Initiatives , 1999 .

[66]  Raimo P. Hämäläinen,et al.  On the importance of behavioral operational research: The case of understanding and communicating about dynamic systems , 2013, Eur. J. Oper. Res..

[67]  Martijn Eskinasi,et al.  Lessons learned from unsuccessful modelling interventions , 2006 .

[68]  K. J. Ives,et al.  Experimental Methods (2) , 1978 .

[69]  R. Wood,et al.  Mental Models, Decision Rules, and Performance Heterogeneity , 2011 .

[70]  M. V. D. Belt,et al.  Mediated Modeling: A System Dynamics Approach To Environmental Consensus Building , 2004 .

[71]  Hakan Yasarcan,et al.  Improving understanding, learning, and performances of novices in dynamic managerial simulation games : A Gradual-Increase-In-Complexity Approach , 2010, Complex..

[72]  James W. Bryant,et al.  Exploring inter-organisational relationships in the health service: An immersive drama approach , 2004, Eur. J. Oper. Res..

[73]  Hakan Yasarcan,et al.  Improving understanding, learning, and performances of novices in dynamic managerial simulation games: A Gradual-Increase-In-Complexity Approach , 2010 .

[74]  L. Alberto Franco,et al.  Decision development in facilitated modelling workshops , 2011, Eur. J. Oper. Res..

[75]  Fran Ackermann,et al.  Where next for problem structuring methods , 2006, J. Oper. Res. Soc..

[76]  Jac A. M. Vennix,et al.  Power-leveling as an effect of group model building , 2012 .

[77]  Ann Taket,et al.  Evaluating user involvement in primary healthcare , 2003 .

[78]  S. Charnley,et al.  Evaluating public participation in environmental decision-making: EPA's superfund community involvement program. , 2005, Journal of environmental management.

[79]  B. Kopainsky,et al.  ASSESSING UNDERSTANDING AND LEARNING ABOUT DYNAMIC SYSTEMS , 2012 .

[80]  Henk Akkermans,et al.  Clients' opinions on group model-building : an exploratory study , 1997 .

[81]  Nicholas Frank Pidgeon,et al.  Difficulties in evaluating public engagement initiatives: reflections on an evaluation of the UK GM Nation? public debate about transgenic crops , 2005 .

[82]  Richard Plate Assessing individuals' understanding of nonlinear causal structures in complex systems , 2010 .

[83]  Jonathan Rosenhead,et al.  Problem structuring methods in action , 2004, Eur. J. Oper. Res..

[84]  Erling Moxnes,et al.  Misperceptions of basic dynamics: the case of renewable resource management , 2004 .

[85]  Jac A. M. Vennix,et al.  Is Group Model Building worthwhile? Considering the effectiveness of GMB , 2009 .

[86]  Jac A. M. Vennix,et al.  Modeling as persuasion: The impact of group model building on attitudes and behavior , 2010 .

[87]  T. Jong,et al.  Comparing two types of model progression in an inquiry learning environment with modelling facilities , 2011 .

[88]  Jac A. M. Vennix,et al.  Group Model Validation , 2012 .

[89]  Qiping Shen,et al.  Comparative Study of Idea Generation between Traditional Value Management Workshops and GDSS-Supported Workshops , 2007 .

[90]  John Mingers,et al.  The Use of Soft Systems Methodology in Practice , 1992 .

[91]  Peter Checkland,et al.  Systems Thinking, Systems Practice , 1981 .

[92]  E. Rouwette,et al.  Group Model-Building to Support Public Policy: Addressing a Conflicted Situation in a Problem Neighbourhood , 2016 .

[93]  J. Forrester Industrial Dynamics , 1997 .

[94]  Jac A. M. Vennix,et al.  Group Model Building: A Decision Room Approach , 2000 .

[95]  M. Orne On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. , 1962 .

[96]  Birgit Kopainsky,et al.  A Blend of Planning and Learning: Simplifying a Simulation Model of National Development , 2010 .

[97]  Andrew Ford,et al.  Reports from the Field: Assessing the Art and Science of Participatory Environmental Modeling , 2010, Int. J. Inf. Syst. Soc. Chang..

[98]  Jerry Fjermestad,et al.  An analysis of communication mode in group support systems research , 2004, Decis. Support Syst..

[99]  Paul N. Finlay On evaluating the performance of GSS: Furthering the debate , 1998, Eur. J. Oper. Res..

[100]  Aldo A. Zagonel,et al.  DEVELOPING AN INTERPRETIVE DIALOGUE FOR GROUP MODEL BUILDING , 2004 .

[101]  Michael Shayne,et al.  Testing the Effects of a System Dynamics Decision Aid on Mental Model Accuracy and Performance on Dynamic Decision Making Tasks , 2007 .

[102]  Michael Pidd,et al.  Contemporary OR/MS in strategy development and policy-making: some reflections , 2004, J. Oper. Res. Soc..

[103]  George P. Richardson,et al.  Foundations of Mental Model Research , 1994 .

[104]  Dean Tjosvold,et al.  Book review : Group model building : facilitating team learning using system dynamics , 1998 .

[105]  Robert Y. Cavana,et al.  Client Perceptions of Reported Outcomes of Group Model Building in the New Zealand Public Sector , 2016 .

[106]  George P. Richardson,et al.  Teamwork in group model building , 1995 .

[107]  Eva Jensen,et al.  Learning and transfer from a simple dynamic system. , 2005, Scandinavian journal of psychology.

[108]  michelle. shields The Role of Group Dynamics in Mental Model Development : An Experimental Comparison of the Effect of Case Study and Management Flight Simulator under Two Levels of Facilitation , .

[109]  Robert Y. Cavana,et al.  Mechanisms for Understanding Mental Model Change in Group Model Building , 2013 .

[110]  George P. Richardson,et al.  Concept models in group model building , 2013 .

[111]  Birgit Kopainsky,et al.  Exploratory strategies for simulation-based learning about national development , 2009 .

[112]  S. Fiske,et al.  The Handbook of Social Psychology , 1935 .

[113]  Krystyna Anne Stave,et al.  Assessing the Effectiveness of Systems Thinking Interventions in the Classroom , 2008 .

[114]  Jac A. M. Vennix,et al.  System dynamics and organizational interventions , 2006 .

[115]  Donald Cameron,et al.  Evaluating immediate and long-term impacts of qualitative group model building workshops on participants' mental models , 2013 .

[116]  Jac A. M. Vennix,et al.  Group model‐building to facilitate organizational change: an exploratory study , 1996 .

[117]  Etiënne A. J. A. Rouwette,et al.  Facilitated modelling in strategy development: measuring the impact on communication, consensus and commitment , 2011, J. Oper. Res. Soc..

[118]  L. Alberto Franco,et al.  Rethinking Soft OR interventions: Models as boundary objects , 2013, Eur. J. Oper. Res..

[119]  Roy Marsh,et al.  Evaluation of a Deliberative Conference , 2004 .

[120]  John Mingers,et al.  Multimethodology: Towards a framework for mixing methodologies , 1997 .

[121]  George P. Richardson,et al.  Group model building: adding more science to the craft , 1997 .

[122]  Agata Sawicka,et al.  Simulator-supported descriptions of complex dynamic problems: experimental results on task performance and system understanding , 2011 .

[123]  Aldo A. Zagonel,et al.  MODEL CONCEPTUALIZATION IN GROUP MODEL BUILDING : A REVIEW OF THE LITERATURE EXPLORING THE TENSION BETWEEN REPRESENTING REALITY AND NEGOTIATING A SOCIAL ORDER , 2002 .