The links between human error diversity and software diversity: Implications for fault diversity seeking

Abstract Software diversity is known to improve fault tolerance in N-version software systems by independent development. As the leading cause of software faults, human error is considered an important factor in diversity seeking. However, there is little scientific research focusing on how to seek software fault diversity based on human error mechanisms. A literature review was conducted to extract factors that may differentiate people with respect to human error-proneness. In addition, we constructed a conceptual model of the links between human error diversity and software diversity. An experiment was designed to validate the hypotheses, in the form of a programming contest, accompanied by a survey of cognitive styles and personality traits. One hundred ninety-two programs were submitted for the identical problem, and 70 surveys were collected. Code inspection revealed 23 faults, of which 10 were coincident faults. The results show that personality traits seems not effective predictors for fault diversity as a whole model, whereas cognitive styles and program measurements moderately account for the variation of fault density. The results also show causal relations between performance levels and coincident faults: coincident faults are unlikely to occur at skill-based performance level; the coincident faults introduced in rule-based performances show a high probability of occurrence, and the coincident faults introduced in knowledge-based performances are shaped by the content and formats of the task itself. Based on these results, we have proposed a model to seek software diversity and prevent coincident faults.

[1]  Helen Sharp,et al.  The use of MBTI in Software Engineering , 2010, PPIG.

[2]  Gavriel Salvendy,et al.  Quantitative and qualitative differences between experts and novices in chunking computer software knowledge , 1994, Int. J. Hum. Comput. Interact..

[3]  Simon P. Davies.  The role of notation and knowledge representation in the determination of programming strategy: A framework for integrating models of programming behavior , 1991 .

[4]  James M. Bieman,et al.  Competencies of exceptional and nonexceptional software engineers , 1995, J. Syst. Softw..

[5]  B. Whitley,et al.  The relationship of psychological type to computer aptitude, attitudes, and behavior , 1996 .

[6]  Nancy G. Leveson,et al.  An experimental evaluation of the assumption of independence in multiversion programming , 1986, IEEE Transactions on Software Engineering.

[7]  Bill Curtis,et al.  Breakdowns and processes during the early activities of software design by professionals , 1987 .

[8]  Meine van der Meulen,et al.  The Effectiveness of Software Diversity in a Large Population of Programs , 2008, IEEE Transactions on Software Engineering.

[9]  O. Awodele,et al.  An Improved Assessment of Personality Traits in Software Engineering , 2007 .

[10]  Brad A. Myers,et al.  A framework and methodology for studying the causes of software errors in programming systems , 2005, J. Vis. Lang. Comput..

[11]  Lorenzo Strigini,et al.  Choosing Effective Methods for Design Diversity - How to Progress from Intuition to Science , 1999, SAFECOMP.

[12]  Norsaremah Salleh,et al.  An empirical study of the effects of personality in pair programming using the five-factor model , 2009, ESEM 2009.

[13]  Robert Feldt,et al.  Generating diverse software versions with genetic programming: and experimental study , 1998, IEE Proc. Softw..

[14]  A. Fuqun Huang,et al.  A Taxonomy System to Identify Human Error Causes for Software Defects , 2012 .

[15]  C. Bishop-Clark,et al.  Cognitive style, personality, and computer programming , 1995 .

[16]  Nancy J. Cooke,et al.  Effects of Computer Programming Experience on Network Representations of Abstract Programming Concepts , 1988, Int. J. Man Mach. Stud..

[17]  Françoise Détienne,et al.  Mental Representations Constructed by Experts and Novices in Object-Oriented Program Comprehension , 1997, INTERACT.

[18]  Luiz Fernando Capretz.  Implications of MBTI in software engineering education , 2002, SGCS.

[19]  J M Hoc,et al.  From human – machine interaction to human – machine cooperation , 2000, Ergonomics.

[20]  Nigel Cross,et al.  Expertise in Design: an overview , 2004 .

[21]  Bev Littlewood,et al.  Modeling software design diversity: a review , 2001, CSUR.

[22]  R. Felder,et al.  Learning and Teaching Styles in Engineering Education. , 1988 .

[23]  Luiz Carlos Begosso,et al.  A Study on Human Error in the Interaction with the Computer Systems , 2011, HCI.

[24]  Mary Ann Schroeder,et al.  Diagnosing and Dealing with Multicollinearity , 1990, Western journal of nursing research.

[25]  Bev Littlewood,et al.  Conceptual Modeling of Coincident Failures in Multiversion Software , 1989, IEEE Trans. Software Eng..

[26]  B. Adelson.  Problem solving and the development of abstract categories in programming languages , 1981, Memory & cognition.

[27]  Dave E. Eckhardt,et al.  A Theoretical Basis for the Analysis of Multiversion Software Subject to Coincident Errors , 1985, IEEE Transactions on Software Engineering.

[28]  C. P. Hansen,et al.  A causal model of the relationship among accidents, biodata, personality, and cognitive factors. , 1989 .

[29]  David Greathead.  MBTI Personality Type and Student Code Comprehension Skill , 2008, PPIG.

[30]  Raymonde Guindon,et al.  Knowledge Exploited by Experts during Software System Design , 1990, Int. J. Man Mach. Stud..

[31]  Lefteris Angelis,et al.  Towards individualized software engineering: empirical studies should collect psychometrics , 2008, CHASE.

[32]  Chiuhsiang Joe Lin,et al.  Optimizing human–system interface automation design based on a skill-rule-knowledge framework , 2010 .

[33]  Kieran Coyle,et al.  Multiple factors of cognitive failure and their relationships with stress vulnerability , 1990 .

[34]  Françoise Détienne,et al.  Software Design — Cognitive Aspects , 2001, Practitioner Series.

[35]  D. D. Wheeler,et al.  The Myers-Briggs Personality Type and Its Relationship to Computer Programming , 1994 .

[36]  I. B. Myers.  Manual: A Guide to the Development and Use of the Myers-Briggs Type Indicator , 1985 .

[37]  Matthias M. Müller,et al.  Do programmer pairs make different mistakes than solo programmers? , 2006, J. Syst. Softw..

[38]  Susan Wiedenbeck,et al.  The Initial Stage of Program Comprehension , 1991, Int. J. Man Mach. Stud..

[39]  Michael R. Lyu,et al.  In search of effective diversity: a six-language study of fault-tolerant flight control software , 1988, [1988] The Eighteenth International Symposium on Fault-Tolerant Computing. Digest of Papers.

[40]  Claes Wohlin,et al.  Experimentation in Software Engineering , 2012, Springer Berlin Heidelberg.

[41]  Kate Ehrlich,et al.  Tapping into tacit programming knowledge , 1982, CHI '82.

[42]  Luiz Fernando Capretz,et al.  Why do we need personality diversity in software engineering? , 2010, SOEN.

[43]  Shari Lawrence Pfleeger,et al.  Towards a Framework for Software Measurement Validation , 1995, IEEE Trans. Software Eng..

[44]  Kym Elizabeth Pocius.  Personality factors in human-computer interaction: A review of the literature , 1991 .

[45]  David Greathead,et al.  Does personality matter?: an analysis of code-review ability , 2007, CACM.

[46]  W. Schnotz,et al.  A Reconsideration of Cognitive Load Theory , 2007 .

[47]  Michael R. Lyu,et al.  Software diversity metrics and measurements , 1992, [1992] Proceedings. The Sixteenth Annual International Computer Software and Applications Conference.

[48]  G. Matthews,et al.  Personality correlates of driver stress , 1991 .

[49]  Françoise Détienne.  Design strategies and knowledge in object-oriented programming: effects of experience , 1995 .

[50]  Ian D. Benest,et al.  EXTROVERT OR INTROVERT? THE REAL PERSONALITIES OF COMPUTING STUDENTS , 2003 .

[51]  Françoise Détienne,et al.  Strategies in object-oriented design , 1996, ArXiv.

[52]  B. Adelson,et al.  The Role of Domain Expenence in Software Design , 1985, IEEE Transactions on Software Engineering.

[53]  Jan Maarten Schraagen,et al.  How Experts Solve a Novel Problem in Experimental Design , 1993, Cogn. Sci..

[54]  Luiz Fernando Capretz.  Personality types in software engineering , 2003, Int. J. Hum. Comput. Stud..

[55]  Dag I. K. Sjøberg,et al.  Effects of Personality on Pair Programming , 2010, IEEE Transactions on Software Engineering.

[56]  Ali Mosleh,et al.  Model-based human reliability analysis: prospects and requirements , 2004, Reliab. Eng. Syst. Saf..

[57]  J. Reitman,et al.  Knowledge organization and skill differences in computer programmers , 1981, Cognitive Psychology.

[58]  Erik Hollnagel,et al.  The Phenotype of Erroneous Actions , 1993, Int. J. Man Mach. Stud..

[59]  Elliot Soloway,et al.  A model of software design , 1986, Int. J. Intell. Syst..

[60]  Nicolas Marmaras,et al.  Supporting skill-, rule-, and knowledge-based behaviour through an ecological interface: an industry-scale application. , 2009 .

[61]  Robert S. Rist.  Knowledge Creation and Retrieval in Program Design: A Comparison of Novice and intermediate Student Programmers , 1991, Hum. Comput. Interact..

[62]  Ram Chillarege,et al.  Orthogonal defect classification , 1996 .

[63]  Juan de Lara,et al.  Empirical study of how personality, team processes and task characteristics relate to satisfaction and software quality , 2008, ESEM '08.

[64]  David F. McAllister,et al.  An Experimental Evaluation of Software Redundancy as a Strategy For Improving Reliability , 1991, IEEE Trans. Software Eng..

[65]  Robert S. Rist.  Plans in programming: definition, demonstration, and development , 1986 .

[66]  M. Chi,et al.  The Nature of Expertise , 1988 .

[67]  Lynda Thomas,et al.  Learning styles and performance in the introductory programming sequence , 2002, SIGCSE '02.

[68]  Ellen J. Bass,et al.  Generating phenotypical erroneous human behavior to evaluate human-automation interaction using model checking , 2012, Int. J. Hum. Comput. Stud..

[69]  Sallyann Bryant,et al.  Rating Expertise in Collaborative Software Development , 2005, PPIG.

[70]  Ramu Naidoo,et al.  Errors made by Students in a Computer Programming Course , 2011 .