Perceived effectiveness of environmental decision support systems in participatory planning: Evidence from small groups of end-users

The challenges associated with evaluating the effectiveness of environmental decision support systems (EDSS) based on the perceptions of only a small sample of end-users are well understood. Although methods adopted from Management Information Systems (MISs) evaluation research have benefited from relatively large (100+) sample sizes, permitting the use of multi-criteria analysis of users perceptions, there are few examples of methods for quantifying effectiveness based on smaller groups of end-users. Use of environmental decision support systems in Integrated Water Resources Management (IWRM) has become increasingly prevalent over the passed twenty years, where their potential for facilitating the participatory process has been recognised; however, few quantitative assessments have been reported. This paper reports the application of a quantitative approach to evaluating environmental decision support systems with small groups of end-users in two case studies where the objective was to facilitate the participatory decision-making process in water management projects. The first case study involved nine end-users applying and evaluating a Bayesian network-based tool to facilitate water demand management implementation in Sofia, the capital city of Bulgaria. The second involved eleven end-users applying and evaluating an integrated tool - the Integrated Solution Support System (I3S) - during a water stress mitigation project in a European context. End-users' perceptions of effectiveness were elicited and compared using statistical analysis. The results of the two case studies suggest that end-user's employment influences their perceptions of EDSS effectiveness. We also show how the applied evaluation method is flexible enough to assess different EDSS types from a range of dimensions.