Faith, hope, and love: an essay on software science's neglect of human factors

Research in the area of programming languages has different facets -- from formal reasoning about new programming language constructs (such as type soundness proofs for new type systems) over inventions of new abstractions, up to performance measurements of virtual machines. A closer look into the underlying research methods reveals a distressing characteristic of programming language research: developers, which are the main audience for new language constructs, are hardly considered in the research process. As a consequence, it is simply not possible to state whether a new construct that requires some kind of interaction with the developer has any positive impact on the construction of software. This paper argues for appropriate research methods in programming language research that rely on studies of developers -- and argues that the introduction of corresponding empirical methods not only requires a new understanding of research but also a different view on how to teach software science to students.

[1]  W. E. Hick.  Quarterly Journal of Experimental Psychology , 1948, Nature.

[2]  Claes Wohlin,et al.  Experimentation in Software Engineering , 2000, The Kluwer International Series in Software Engineering.

[3]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[4]  Herbert Keuth.  Karl Poppers „Logik der Forschung“ , 2019, Handbuch Karl Popper.

[5]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[6]  P. Fitts.  The information capacity of the human motor system in controlling the amplitude of movement. , 1954, Journal of experimental psychology.

[7]  B. A. Sheil,et al.  The Psychological Study of Programming , 1981, CSUR.

[8]  Paul Lukowicz,et al.  Experimental evaluation in computer science: A quantitative study , 1995, J. Syst. Softw..

[9]  Ralph E. Johnson,et al.  Documenting frameworks using patterns , 1992, OOPSLA '92.

[10]  Edsger W. Dijkstra,et al.  A Discipline of Programming , 1976 .

[11]  Peter J. Denning,et al.  Is computer science science? , 2005, CACM.

[12]  Leland Wilkinson,et al.  Statistical Methods in Psychology Journals Guidelines and Explanations , 2005 .

[13]  J. E. Sammet,et al.  Software psychology: human factors in computer and information systems , 1983, SGCH.

[14]  Victor R. Basili,et al.  The role of experimentation in software engineering: past, current, and future , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[15]  Jack Trout,et al.  The Power Of Simplicity , 1998 .

[16]  Edsger W. Dijkstra,et al.  The humble programmer , 1972, CACM.

[17]  Andreas Rausch,et al.  Componentware: Methodik des evolutionären Architekturentwurfs , 2001, Softwaretechnik-Trends.

[18]  Gonzalo Génova.  Is computer science truly scientific? , 2010, Commun. ACM.

[19]  Norman Meyrowitz,et al.  Conference proceedings on Object-oriented programming systems, languages, and applications , 1986, OOPSLA 1987.

[20]  Amela Karahasanovic,et al.  A survey of controlled experiments in software engineering , 2005, IEEE Transactions on Software Engineering.

[21]  Randall B. Smith,et al.  Self: The power of simplicity , 1987, OOPSLA 1987.

[22]  Amer Diwan,et al.  The DaCapo benchmarks: java benchmarking development and analysis , 2006, OOPSLA '06.

[23]  Walter F. Tichy,et al.  Should Computer Scientists Experiment More? , 1998, Computer.

[24]  J. Bortz,et al.  Forschungsmethoden und Evaluation für Human- und Sozialwissenschaftler , 2006 .

[25]  H. Hornich.  Logik der Forschung , 1936 .

[26]  Michael Eichberg,et al.  A Handbook of Software and Systems Engineering , 2009 .

[27]  Ralph Johnson.  Documenting frameworks using patterns , 1992, OOPSLA 1992.

[28]  Ben Shneiderman,et al.  Software psychology: Human factors in computer and information systems (Winthrop computer systems series) , 1980 .

[29]  Walter F. Tichy.  Die Bedeutung der Empirie fuer die Softwaretechnik , 2005, Software Engineering.

[30]  Gerald M. Weinberg,et al.  Psychology of computer programming , 1971 .

[31]  Mary Shaw,et al.  Computer Science: Reflections on the Field, Reflections from the Field , 2004 .

[32]  Natalia Juristo Juzgado,et al.  Basics of Software Engineering Experimentation , 2010, Springer US.

[33]  Gregor Snelting.  Paul Feyerabend und die Softwaretechnologie , 1998, Informatik-Spektrum.

[34]  Tore Dybå,et al.  The Future of Empirical Methods in Software Engineering Research , 2007, Future of Software Engineering (FOSE '07).

[35]  Daniela E. Damian,et al.  Selecting Empirical Methods for Software Engineering Research , 2008, Guide to Advanced Empirical Software Engineering.

[36]  Shari Lawrence Pfleeger,et al.  Soup or Art? The Role of Evidential Force in Empirical Software Engineering , 2005, IEEE Softw..

[37]  Maurice H. Halstead,et al.  Elements of software science (Operating and programming systems series) , 1977 .

[38]  Liming Zhu,et al.  Evaluating guidelines for empirical software engineering studies , 2006, ISESE '06.

[39]  Rachel Harrison,et al.  An exploratory study of the effect of aspect-oriented programming on maintainability , 2008, Software Quality Journal.

[40]  J. Lange,et al.  The power of simplicity , 2003, International journal of STD & AIDS.

[41]  Randall B. Smith,et al.  SELF: The power of simplicity , 1987, OOPSLA '87.

[42]  Ole Lehrmann Madsen,et al.  What Object-Oriented Programming May Be - and What It Does Not Have To Be , 1988, ECOOP.

[43]  Barry W. Boehm,et al.  Cost models for future software life cycle processes: COCOMO 2.0 , 1995, Ann. Softw. Eng..

[44]  Thomas Ledoux,et al.  Aspect-Oriented Software Development , 2003 .

[45]  Michael Philippsen,et al.  Two Controlled Experiments Assessing the Usefulness of Design Pattern Documentation in Program Maintenance , 2002, IEEE Trans. Software Eng..

[46]  Marvin V. Zelkowitz,et al.  Experimental Models for Validating Technology , 1998, Computer.

[47]  Andreas Speck.  Software Engineering (9) , 2006 .

[48]  Gregor Snelting.  Feyerabend – zwei Jahre später , 2001, Informatik-Spektrum.

[49]  Shari Lawrence Pfleeger,et al.  Software Metrics : A Rigorous and Practical Approach , 1998 .

[50]  Lutz Prechelt.  Kontrollierte Experimente in der Softwaretechnik , 2001 .