An Empirical Study of Software Project Managers Using a Case-Based Reasoner

BACKGROUND -- whilst substantial effort has been invested in developing and evaluating knowledge-based techniques for project prediction, little is known about the interaction between them and expert users. OBJECTIVE - the aim is to explore the interaction of cognitive processes and personality of software project managers undertaking tool-supported estimation tasks such as effort and cost prediction. METHOD - we conducted personality profiling and observational studies using think-aloud protocols with five senior project managers using a case-based reasoning (CBR) tool to predict effort for real projects. RESULTS - we found pronounced differences between the participants in terms of individual differences, cognitive behaviour and estimation outcomes, although there was a general tendency for over-optimism and over-confidence. CONCLUSIONS - in order to improve task effectiveness in the workplace we need to understand the cognitive behaviour of software professionals in addition to conducting machine learning research.

[1]  Magne Jørgensen,et al.  A Systematic Review of Software Development Cost Estimation Studies , 2007, IEEE Transactions on Software Engineering.

[2]  William G. Huitt,et al.  Problem solving and decision making: Consideration of individual differences using the Myers-Briggs Type Indicator. , 1992 .

[3]  S. Epstein.  Integration of the cognitive and the psychodynamic unconscious. , 1994, The American psychologist.

[4]  Martin J. Shepperd,et al.  Estimating Software Project Effort Using Analogies , 1997, IEEE Trans. Software Eng..

[5]  Janet B W Williams,et al.  Diagnostic and Statistical Manual of Mental Disorders , 2013 .

[6]  Ingunn Myrtveit,et al.  A Controlled Experiment to Assess the Benefits of Estimating with Analogy and Regression Models , 1999, IEEE Trans. Software Eng..

[7]  Narasimhaiah Gorla,et al.  Who should work with whom?: building effective software project teams , 2004, CACM.

[8]  Ron Kohavi,et al.  Wrappers for feature selection , 1997 .

[9]  Rajkumar Roy,et al.  Expert Judgement in Cost Estimating: Modelling the Reasoning Process , 2001, Concurr. Eng. Res. Appl..

[10]  Magne Jørgensen,et al.  Identification of more risks can lead to increased over-optimism of and over-confidence in software development effort estimates , 2010, Inf. Softw. Technol..

[11]  Sharon C. Lyter,et al.  Diagnostic and Statistical Manual of Mental Disorders: Making it Work for Social Work , 2012 .

[12]  H. Eysenck.  Biological Basis of Personality , 1963, Nature.

[13]  Magne Jørgensen,et al.  A Systematic Review of Software Development Cost Estimation Studies , 2007 .

[14]  Bent Flyvbjerg,et al.  Curbing Optimism Bias and Strategic Misrepresentation in Planning: Reference Class Forecasting in Practice , 2008 .

[15]  Bracha Kramarski,et al.  Enhancing Mathematical Reasoning in the Classroom: The Effects of Cooperative Learning and Metacognitive Training , 2003 .

[16]  Dag I. K. Sjøberg,et al.  The impact of customer expectation on software development effort estimates , 2004 .

[17]  Martin Shepperd,et al.  Case-Based Reasoning and Software Engineering , 2003 .

[18]  Steve M. Easterbrook,et al.  Anchoring and adjustment in software estimation , 2005, ESEC/FSE-13.

[19]  Janet L. Kolodner,et al.  Case-Based Reasoning , 1989, IJCAI 1989.

[20]  Janet L. Kolodner,et al.  Case-Based Reasoning , 1988, IJCAI 1989.

[21]  B. Flyvbjerg,et al.  Megaprojects and Risk: An Anatomy of Ambition , 2003 .

[22]  Luiz Fernando Capretz.  Personality types in software engineering , 2003, Int. J. Hum. Comput. Stud..

[23]  Carolyn Mair,et al.  Personality and analogy-based project estimation , 2009 .

[24]  David E. Avison,et al.  Action research , 1999, CACM.

[25]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[26]  I. B. Myers.  Manual: A Guide to the Development and Use of the Myers-Briggs Type Indicator , 1985 .

[27]  Carolyn Mair,et al.  The consistency of empirical comparisons of regression and analogy-based software project cost prediction , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[28]  K. A. Ericsson,et al.  Verbal reports as data. , 1980 .

[29]  Magne Jørgensen,et al.  A review of studies on expert estimation of software development effort , 2004, J. Syst. Softw..

[30]  Carolyn Mair,et al.  A Literature Review of Expert Problem Solving using Analogy , 2009, EASE.

[31]  C. Carver,et al.  Perspectives on Personality , 1988 .