Does explanation improve the acceptance of decision support for product release planning?

Objective: Decision support provided to users is often lack of acceptance. One of the reasons is a deficit in understanding where the suggestions come from and how they come. This essentially is not a technical problem, but a technology adoption problem. This situation was also analyzed as a result of former empirical studies conducted on ReleasePlannerTM, a decision support tool for planning product releases. To overcome this situation, three machine learning techniques have been applied to mine the tool's solutions, and the mining results are presented to the tool users as explanations. This paper presents the evaluation on the generated explanations as a means to improve the user acceptance of the tool. Method: A three-stage controlled experiment was designed and carried out with a group of ten graduate students at the University of Calgary and another group of five project managers from the IT industry. Two research goals were addressed to (i) evaluate the impact of the explanations generated from these three applied techniques, and (ii) compare some of the findings from this study with the ones from our previous experiments. Results: Our findings for the first research goal indicated that the explanations generated from the three techniques contributed to the improvement of the subjects' confidence in the tool solutions and trust of the tool, and therefore an overall better user acceptance of the tool. Meanwhile, no significant differences were found among the impacts of the three techniques. For the second research goal, we found that some of the findings from this study were consistent with the ones from our previous experiments.

[1]  David Maxwell Chickering,et al.  Dependency Networks for Inference, Collaborative Filtering, and Data Visualization , 2000, J. Mach. Learn. Res..

[2]  Wen Gao,et al.  Learning Contextual Dependency Network Models for Link-Based Classification , 2006, IEEE Transactions on Knowledge and Data Engineering.

[3]  R Likert,et al.  A TECHNIQUE FOR THE MEASUREMENT OF ATTITUDE SCALES , 1932 .

[4]  Pankaj Bhawnani,et al.  Intelligent Decision Support for Road Mapping A Technology Transfer Case Study with Siemens Corporate Technology , 2006 .

[5]  Gregor Snelting,et al.  Assessing Modular Structure of Legacy Code Based on Mathematical Concept Analysis , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[6]  Anders Wall,et al.  Towards a Capability Model for the Software Release Planning Process - Based on a Multiple Industrial Case Study , 2008, PROFES.

[7]  Sjaak Brinkkemper,et al.  Software product release planning through optimization and what-if analysis , 2008, Inf. Softw. Technol..

[8]  Günther Ruhe,et al.  Supporting Software Release Planning Decisions for Evolving Systems , 2005, 29th Annual IEEE/NASA Software Engineering Workshop.

[9]  Günther Ruhe,et al.  A family of empirical studies to compare informal and optimization-based planning of software releases , 2006, ISESE '06.

[10]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[11]  Günther Ruhe,et al.  Rough set-based data analysis in goal-oriented software measurement , 1996, Proceedings of the 3rd International Software Metrics Symposium.

[12]  Anders Wall,et al.  Key Aspects of Software Release Planning in Industry , 2008, 19th Australian Conference on Software Engineering (aswec 2008).

[13]  Thomas Zimmermann,et al.  Explaining Product Release Planning Results Using Concept Analysis , 2008, SEKE.

[14]  Larry Wasserman,et al.  All of Statistics: A Concise Course in Statistical Inference , 2004 .

[15]  Janusz Zalewski,et al.  Rough sets: Theoretical aspects of reasoning about data , 1996 .

[16]  Hans-Peter Kriegel,et al.  Ieee Transactions on Knowledge and Data Engineering Probabilistic Memory-based Collaborative Filtering , 2022 .

[17]  Rudolf Wille,et al.  Restructuring Lattice Theory: An Approach Based on Hierarchies of Concepts , 2009, ICFCA.

[18]  Colin J. Neill,et al.  Modeling uncertainty in software engineering using rough sets , 2005, Innovations in Systems and Software Engineering.

[19]  Joachim Karlsson,et al.  A Cost-Value Approach for Prioritizing Requirements , 1997, IEEE Softw..

[20]  Raymond McCall,et al.  Rationale Management in Software Engineering , 2006 .

[21]  Dan Klein,et al.  Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network , 2003, NAACL.

[22]  David A. Penny,et al.  An estimation-based management framework for enhancive maintenance in commercial software products , 2002, International Conference on Software Maintenance, 2002. Proceedings..

[23]  Günther Ruhe,et al.  Hybrid Intelligence in Software Release Planning , 2004, Int. J. Hybrid Intell. Syst..