Spatial decision support systems: Exploring differences in pilot-testing with students vs. professionals

Abstract This study explores the implications of engaging students, versus professionals/experts, in pilot-testing of SDSS, and discusses likely differences in terms of expected outcomes for the given pilot-test. To this end we use data collected during two pilot tests of a novel SDSS that was developed by members of our project team. The pilot-tests were done with two different groups; one made of 13 doctoral students, while the other of 12 professionals/stakeholders. The pilot-test served to gather feedback on SDSS usability and other aspects of interest to the development team. On the basis of the outcomes obtained we develop an analytical framework meant to summarise what we come to notice as key aspects distinguishing how different types of testers will engage in an SDSS pilot-test, and the type of feedback these will consequently provide. These key aspects include expertise, stage of life, and institutional context (ESI). This framework could offer some help to other teams in planning, organizing, and delivering pilot-test, and processing the assessments received.

[1]  Jay J Van Bavel,et al.  Contextual sensitivity in scientific reproducibility , 2016, Proceedings of the National Academy of Sciences.

[2]  James Boxall,et al.  Lessons for PPGIS from the application of a decision-support tool in the Nova Forest Alliance of Nova Scotia, Canada. , 2009, Journal of environmental management.

[3]  K. A. Ericsson,et al.  The making of an expert. , 2007, Harvard business review.

[4]  Carlo Giupponi,et al.  Decision Support Systems for implementing the European Water Framework Directive: The MULINO approach , 2007, Environ. Model. Softw..

[5]  Maria Francesca Costabile,et al.  Towards satisfying practitioners in using Planning Support Systems , 2018, Comput. Environ. Urban Syst..

[6]  Marta Pérez-Soba,et al.  Scenarios: tools for coping with complexity and future uncertainty? , 2015 .

[7]  Ron Janssen,et al.  Why are spatial decision support systems not used? Some experiences from the Netherlands , 2003, Comput. Environ. Urban Syst..

[8]  Gregory E. Kersten,et al.  An experimental study of software agent negotiations with humans , 2014, Decis. Support Syst..

[9]  Carolien Kroeze,et al.  Can computer models be used for social learning? A serious game in water management , 2016, Environ. Model. Softw..

[10]  Robert J. Crutcher,et al.  The role of deliberate practice in the acquisition of expert performance. , 1993 .

[11]  Miquel Sànchez-Marrè,et al.  Designing and building real environmental decision support systems , 2004, Environ. Model. Softw..

[12]  John Bohannon,et al.  REPRODUCIBILITY. Many psychology papers fail replication test. , 2015, Science.

[13]  R. Peterson,et al.  Convenience samples of college students and research reproducibility , 2014 .

[14]  Sander Janssen,et al.  QUICKScan as a quick and participatory methodology for problem identification and scoping in policy processes , 2016 .

[15]  Xinyue Ye,et al.  A group-based spatial decision support system for wind farm site selection in Northwest Ohio , 2013 .

[16]  D. Stolle,et al.  Why Can’t a Student Be More Like an Average Person?: Sampling and Attrition Effects in Social Science Field and Laboratory Experiments , 2010 .

[17]  R. Peterson.  On the Use of College Students in Social Science Research: Insights from a Second‐Order Meta‐analysis , 2001 .

[18]  Brian S. McIntosh,et al.  Tools to think with? Towards understanding the use of computer-based support tools in policy relevant research , 2007, Environ. Model. Softw..

[19]  Tal Ben-Zvi,et al.  Measuring the Perceived Effectiveness of Decision Support Systems and Their Impact on Performance , 2012, Decis. Support Syst..

[20]  Wynne W. Chin,et al.  The Effects of Group Attitudes Toward Alternative GDSS Designs on the Decision‐making Performance of Computer‐Supported Groups* , 1994 .

[21]  Anthony J. Jakeman,et al.  Environmental decision support systems (EDSS) development - Challenges and best practices , 2011, Environ. Model. Softw..

[22]  Marisa A. Zapata.  Deliberating across differences: Planning futures in cross-cultural spaces , 2009 .

[23]  A. K. Bregt,et al.  A Special Section entitled “Challenges to and opportunities for biodiversity science-policy interfaces” , 2015 .

[24]  Ron Janssen,et al.  Effectiveness of collaborative map-based decision support tools: Results of an experiment , 2013, Environ. Model. Softw..

[25]  Arnold K. Bregt,et al.  The social side of spatial decision support systems: Investigating knowledge integration and learning , 2017 .

[26]  Graham Pervan,et al.  Eight key issues for the decision support systems discipline , 2008, Decis. Support Syst..

[27]  Matthew S. Merrifield,et al.  MarineMap: A web-based platform for collaborative marine protected area planning , 2013 .