PLIERS: A Process that Integrates User-Centered Methods into Programming Language Design

Programming language design requires making many usability-related design decisions. However, existing HCI methods can be impractical to apply to programming languages: they have high iteration costs, programmers require significant learning time, and user performance has high variance. To address these problems, we adapted both formative and summative HCI methods to make them more suitable for programming language design. We integrated these methods into a new process, PLIERS, for designing programming languages in a user-centered way. We evaluated PLIERS by using it to design two new programming languages. Glacier extends Java to enable programmers to express immutability properties effectively and easily. Obsidian is a language for blockchains that includes verification of critical safety properties. Summative usability studies showed that programmers were able to program effectively in both languages after short training periods.

[1]  Jonathan Aldrich,et al.  An Empirical Study of Object Protocols in the Wild , 2011, ECOOP.

[2]  Margaret M. Burnett,et al.  Programming, Problem Solving, and Self-Awareness: Effects of Explicit Guidance , 2016, CHI.

[3]  Jakob Nielsen,et al.  A mathematical model of the finding of usability problems , 1993, INTERCHI.

[4]  K. Stewart,et al.  Drug Guru: a computer software program for drug design using medicinal chemistry rules. , 2006, Bioorganic & medicinal chemistry.

[5]  Andreas Stefik,et al.  An empirical comparison of the accuracy rates of novices using the quorum, perl, and randomo programming languages , 2011, PLATEAU '11.

[6]  Kate Ehrlich,et al.  Empirical Studies of Programming Knowledge , 1984, IEEE Transactions on Software Engineering.

[7]  Ben Shneiderman,et al.  Empirical studies of programmers: the territory, paths, and destination , 1986 .

[8]  Brad A. Myers,et al.  Usability Methods for Designing Programming Languages for Software Engineers , 2019 .

[9]  Stefan Hanenberg,et al.  How do API documentation and static typing affect API usability? , 2014, ICSE.

[10]  Bill Curtis,et al.  Breakdowns and processes during the early activities of software design by professionals , 1987 .

[11]  Thomas D. LaToza,et al.  Programmers Are Users Too: Human-Centered Methods for Improving Programming Tools , 2016, Computer.

[12]  Susan Elliott Sim,et al.  Case studies for software engineers , 2006, ICSE '06.

[13]  James D. Herbsleb,et al.  Searching the State Space: A Qualitative Study of API Protocol Usability , 2015, 2015 IEEE 23rd International Conference on Program Comprehension.

[14]  N. Pennington.  Stimulus structures and mental representations in expert comprehension of computer programs , 1987, Cognitive Psychology.

[15]  Brad A. Myers,et al.  A Pilot Study of the Safety and Usability of the Obsidian Blockchain Programming Language , 2019, PLATEAU@UIST.

[16]  David Radcliffe,et al.  Impact of CAD tools on creative problem solving in engineering design , 2009, Comput. Aided Des..

[17]  Alan F. Blackwell,et al.  The practices of programming , 2016, 2016 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[18]  Caitlin Kelleher,et al.  Lowering the barriers to programming: A taxonomy of programming environments and languages for novice programmers , 2005, CSUR.

[19]  Éric Tanter,et al.  First-class state change in plaid , 2011, OOPSLA '11.

[20]  Sophia Drossopoulou,et al.  Flint for Safer Smart Contracts , 2019, ArXiv.

[21]  Stefan Hanenberg,et al.  The Programming Language Wars: Questions and Responsibilities for the Programming Language Community , 2014, Onward!.

[22]  Bill Curtis,et al.  Communication breakdowns and boundary spanning activities on large programming projects , 1987 .

[23]  Stefan Hanenberg,et al.  Methodological Irregularities in Programming-Language Research , 2017, Computer.

[24]  Vladimir Tosic,et al.  Guidelines for industrially-based multiple case studies in software engineering , 2009, 2009 Third International Conference on Research Challenges in Information Science.

[25]  Anneliese Amschler Andrews,et al.  Program understanding behavior during corrective maintenance of large-scale software , 1999, Int. J. Hum. Comput. Stud..

[26]  Brad A. Myers,et al.  Natural programming languages and environments , 2004, Commun. ACM.

[27]  Willemien Visser.  Strategies in programming programmable controllers: a field study on a professional programmer , 1987 .

[28]  Emil Sekerinski,et al.  A Study of The Fragile Base Class Problem , 1998, ECOOP.

[29]  William J. Knottenbelt,et al.  Towards Safer Smart Contracts: A Survey of Languages and Verification Methods , 2018, ArXiv.

[30]  Arvind Satyanarayan,et al.  Reactive Vega: A Streaming Dataflow Architecture for Declarative Interactive Visualization , 2016, IEEE Transactions on Visualization and Computer Graphics.

[31]  Alan L. Porter,et al.  New Methodological Developments in FTA , 2008 .

[32]  Shriram Krishnamurthi,et al.  Can we crowdsource language design? , 2017, Onward!.

[33]  Margaret M. Burnett,et al.  A practical guide to controlled experiments of software engineering tools with human participants , 2013, Empirical Software Engineering.

[34]  Elaine Shi,et al.  Step by Step Towards Creating a Safe Smart Contract: Lessons and Insights from a Cryptocurrency Lab , 2016, Financial Cryptography Workshops.

[35]  Lance A. Miller.  Programming by Non-Programmers , 1974, Int. J. Man Mach. Stud..

[36]  Joshua J. Bloch,et al.  Effective java™, second edition , 2008 .

[37]  Lutz Prechelt.  The Mythical 10x Programmer , 2019, Rethinking Productivity in Software Engineering.

[38]  Jan Gulliksen,et al.  Key principles for user-centred systems design , 2003, Behav. Inf. Technol..

[39]  Leonardo Alt,et al.  SMT-Based Verification of Solidity Smart Contracts , 2018, ISoLA.

[40]  Brad A. Myers,et al.  Glacier: Transitive Class Immutability for Java , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[41]  Jonathan Aldrich,et al.  Modular typestate checking of aliased objects , 2007, OOPSLA.

[42]  Harry Hochheiser,et al.  Research Methods for Human-Computer Interaction , 2008 .

[43]  Westley Weimer,et al.  Benefits and barriers of user evaluation in software engineering research , 2011, OOPSLA '11.

[44]  Joseph F. Dumas,et al.  A Practical Guide to Usability Testing , 1993 .

[45]  Joyce J. Elam,et al.  A methodology for studying software design teams: an investigation of conflict behaviors in the requirements definition phase , 1987 .

[46]  William R. Nichols.  The End to the Myth of Individual Programmer Productivity , 2019, IEEE Software.

[47]  Chris Speed,et al.  Making Sense of Blockchain Applications: A Typology for HCI , 2018, CHI.

[48]  L. Given,et al.  The SAGE encyclopedia of qualitative research methods , 2011 .

[49]  Eric Rosenbaum,et al.  Scratch: programming for all , 2009, Commun. ACM.

[50]  Forrest Shull,et al.  Exploring Language Support for Immutability , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[51]  Stefan Hanenberg,et al.  An Empirical Study on the Impact of C++ Lambdas and Programmer Experience , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[52]  Michael D. Ernst,et al.  Object and reference immutability using Java generics , 2007, ESEC-FSE '07.

[53]  Ilya Sergey,et al.  Scilla: a Smart Contract Intermediate-Level LAnguage , 2018, ArXiv.

[54]  Brad A. Myers,et al.  Interdisciplinary programming language design , 2018, Onward!.

[55]  Mary Shaw,et al.  The state of the art in end-user software engineering , 2011, ACM Comput. Surv..

[56]  Arne Jönsson,et al.  Wizard of Oz studies -- why and how , 1993, Knowl. Based Syst..

[57]  Jonathan Aldrich,et al.  PLURAL: checking protocol compliance under aliasing , 2008, ICSE Companion '08.

[58]  Brad A. Myers,et al.  Using HCI techniques to design a more usable programming system , 2002, Proceedings IEEE 2002 Symposia on Human Centric Computing Languages and Environments.

[59]  Benjamin C. Pierce,et al.  Types and programming languages: the next generation , 2003, 18th Annual IEEE Symposium of Logic in Computer Science, 2003. Proceedings..

[60]  Nikhil Swamy,et al.  Formal Verification of Smart Contracts: Short Paper , 2016, PLAS@CCS.

[61]  Brad A. Myers,et al.  Obsidian: Typestate and Assets for Safer Blockchain Programming , 2019, ACM Trans. Program. Lang. Syst..

[62]  Allen Newell,et al.  The Prospects for Psychological Science in Human-Computer Interaction , 1985, Hum. Comput. Interact..

[63]  Jan Pries-Heje,et al.  A Comprehensive Framework for Evaluation in Design Science Research 1 , 2022 .

[64]  Jonathan Aldrich,et al.  Typestate-oriented programming , 2009, OOPSLA Companion.

[65]  Jeffrey Heer,et al.  Protovis: A Graphical Toolkit for Visualization , 2009, IEEE Transactions on Visualization and Computer Graphics.

[66]  Maurice Herlihy,et al.  Blockchains from a distributed computing perspective , 2019, Commun. ACM.

[67]  Robert L. Glass,et al.  Loyal Opposition - Frequently Forgotten Fundamental Facts about Software Engineering , 2001, IEEE Softw..

[68]  Simon L. Peyton Jones,et al.  A user-centred approach to functions in Excel , 2003, ICFP '03.