Towards Providing On-Demand Expert Support for Software Developers

Software development is an expert task that requires complex reasoning and the ability to recall language or API-specific details. In practice, developers often seek support from IDE tools, Web resources, or other developers to help fill in gaps in their knowledge on-demand. In this paper, we present two studies that seek to inform the design of future systems that use remote experts to support developers on demand. The first explores what types of questions developers would ask a hypothetical assistant capable of answering any question they pose. The second study explores the interactions between developers and remote experts in supporting roles. Our results suggest eight key system features needed for on-demand remote developer assistants to be effective, which has implications for future human-powered development tools.

[1]  Eytan Adar,et al.  SHOCK: communicating with computational messages and automatic private profiles , 2003, WWW '03.

[2]  Rita Shewbridge,et al.  Everyday making: identifying future uses for 3D printing in the home , 2014, Conference on Designing Interactive Systems.

[3]  Laurie Williams,et al.  The costs and benefits of pair programming , 2001 .

[4]  Yu-An Sun,et al.  Human OCR: Insights from a Complex Human Computation Process , 2011 .

[5]  Rob Miller,et al.  VizWiz: nearly real-time answers to visual questions , 2010, UIST.

[6]  Gail C. Murphy,et al.  Asking and Answering Questions during a Programming Change Task , 2008, IEEE Transactions on Software Engineering.

[7]  Fred P. Brooks,et al.  The Mythical Man-Month , 1975, Reliable Software.

[8]  Jeffrey P. Bigham,et al.  VizWiz: nearly real-time answers to visual questions , 2010, W4A.

[9]  Scott P. Robertson,et al.  Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , 1991 .

[10]  Damon Horowitz,et al.  The anatomy of a large-scale social search engine , 2010, WWW '10.

[11]  Laurie A. Williams,et al.  Pair Programming Illuminated , 2002 .

[12]  André van der Hoek,et al.  Microtask programming: building software with a crowd , 2014, UIST.

[13]  Robert DeLine,et al.  Information Needs in Collocated Software Development Teams , 2007, 29th International Conference on Software Engineering (ICSE'07).

[14]  Michael S. Bernstein,et al.  Apparition: Crowdsourced User Interfaces that Come to Life as You Sketch Them , 2015, CHI.

[15]  P. David Stotts,et al.  Exploring the Efficacy of Distributed Pair Programming , 2002, XP/Agile Universe.

[16]  Philip R. Cohen.  The role of natural language in a multimodal interface , 1992, UIST '92.

[17]  David R. Barstow,et al.  Proceedings of the 25th International Conference on Software Engineering , 1978, ICSE.

[18]  Jacob Aristotle,et al.  Stack Overflow , 2012 .

[19]  Rob Miller,et al.  Real-time collaborative coding in a web IDE , 2011, UIST.

[20]  Lydia B. Chilton,et al.  TurKit: human computation algorithms on mechanical turk , 2010, UIST.

[21]  Peter B. Henderson.  Software engineering education , 2003, SOEN.

[22]  Michael S. Bernstein,et al.  Crowds in two seconds: enabling realtime crowd-powered interfaces , 2011, UIST.

[23]  Estelle Brodman,et al.  Managing the Flow of Technology: Technology Transfer and the Dissemination of Technological Information Within the R&D Organization (Book Review) , 1978 .

[24]  Michael S. Bernstein,et al.  Expert crowdsourcing with flash teams , 2014, UIST.

[25]  James D. Herbsleb,et al.  Object-oriented analysis and design in software project teams , 1995 .

[26]  Philip J. Guo,et al.  Two studies of opportunistic programming: interleaving web foraging, learning, and writing code , 2009, CHI.

[27]  Philip J. Guo,et al.  Codechella: Multi-user program visualizations for real-time tutoring and collaborative learning , 2015, 2015 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[28]  Philip J. Guo.  Codeopticon: Real-Time, One-To-Many Human Tutoring for Computer Programming , 2015, UIST.

[29]  Jeffrey Nichols,et al.  Chorus: a crowd-powered conversational assistant , 2013, UIST.

[30]  Mark S. Ackerman,et al.  Answer Garden 2: merging organizational memory with collaborative help , 1996, CSCW '96.

[31]  Mark S. Ackerman,et al.  Expertise recommender: a flexible recommendation system and architecture , 2000, CSCW '00.

[32]  Chanchal Kumar Roy,et al.  Answering questions about unanswered questions of Stack Overflow , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[33]  Jaime Teevan,et al.  Information extraction and manipulation threats in crowd-powered systems , 2014, CSCW.

[34]  Alexandre Bergel,et al.  Asking and Answering Questions during a Programming Change Task in Pharo Language , 2014, PLATEAU.

[35]  Lutz Prechelt,et al.  Distributed-pair programming can work well and is not just distributed pair-programming , 2013, ICSE Companion.

[36]  Evangelos E. Milios,et al.  Finding expert users in community question answering , 2012, WWW.

[37]  Judith S. Olson,et al.  Distance Matters , 2000, Hum. Comput. Interact..

[38]  Harlan D. Mills.  Software engineering education , 1980 .

[39]  Audris Mockus,et al.  Expertise Browser: a quantitative approach to identifying expertise , 2002, Proceedings of the 24th International Conference on Software Engineering. ICSE 2002.

[40]  Lena Mamykina,et al.  Design lessons from the fastest q&a site in the west , 2011, CHI.

[41]  Thomas J. Allen,et al.  Managing the flow of technology: technology transfer and the dissemination of technological informat , 1977 .

[42]  Philip J. Guo,et al.  Writing Code to Prototype, Ideate, and Discover , 2009, IEEE Software.

[43]  Arie van Deursen,et al.  Supporting Developers' Coordination in the IDE , 2015, CSCW.

[44]  Maria Soledad Pera,et al.  A community question-answering refinement system , 2011, HT '11.

[45]  Danai Koutra,et al.  Glance: rapidly coding behavioral video with the crowd , 2014, UIST.

[46]  Igor Steinmacher,et al.  Barriers Faced by Newcomers to Open Source Projects: A Systematic Review , 2014, OSS.

[47]  Martin P. Robillard,et al.  Recommendation Systems for Software Engineering , 2010, IEEE Software.

[48]  James D. Herbsleb,et al.  Object-Oriented Analysis and Design in Software Project Teams , 1995, Hum. Comput. Interact..

[49]  Brad A. Myers,et al.  Six Learning Barriers in End-User Programming Systems , 2004, 2004 IEEE Symposium on Visual Languages - Human Centric Computing.

[50]  Scott R. Klemmer,et al.  Example-centric programming: integrating web search into the development environment , 2010, CHI.

[51]  Scott R. Klemmer,et al.  What would other programmers do: suggesting solutions to error messages , 2010, CHI.