How Users Interpret Bugs in Trigger-Action Programming

Trigger-action programming (TAP) is a programming model enabling users to connect services and devices by writing if-then rules. As such systems are deployed in increasingly complex scenarios, users must be able to identify programming bugs and reason about how to fix them. We first systematize the temporal paradigms through which TAP systems could express rules. We then identify ten classes of TAP programming bugs related to control flow, timing, and inaccurate user expectations. We report on a 153-participant online study where participants were assigned to a temporal paradigm and shown a series of pre-written TAP rules. Half of the rules exhibited bugs from our ten bug classes. For most of the bug classes, we found that the presence of a bug made it harder for participants to correctly predict the behavior of the rule. Our findings suggest directions for better supporting end-user programmers.

[1]  Shan Lu,et al.  TaxDC: A Taxonomy of Non-Deterministic Concurrency Bugs in Datacenter Distributed Systems , 2016, ASPLOS.

[2]  Kaisa Väänänen,et al.  Evolution towards smart home environments: empirical evaluation of three user interfaces , 2004, Personal and Ubiquitous Computing.

[3]  Ying Zhang,et al.  An empirical characterization of IFTTT: ecosystem, usage, and performance , 2017, Internet Measurement Conference.

[4]  Lujo Bauer,et al.  Some Recipes Can Do More Than Spoil Your Appetite: Analyzing the Security and Privacy Risks of IFTTT Recipes , 2017, WWW.

[5]  Amos Azaria,et al.  InstructableCrowd: Creating IF-THEN Rules via Conversations with the Crowd , 2016, CHI Extended Abstracts.

[6]  Tom Rodden,et al.  "Playing with the Bits" User-Configuration of Ubiquitous Domestic Environments , 2003, UbiComp.

[7]  Lujo Bauer,et al.  Access Control for Home Data Sharing: Attitudes, Needs and Practices , 2010, CHI 2010.

[8]  S. Frederick.  Journal of Economic Perspectives—Volume 19, Number 4—Fall 2005—Pages 25–42 Cognitive Reflection and Decision Making , 2022 .

[9]  Shan Lu,et al.  AutoTap: Synthesizing and Repairing Trigger-Action Programs Using LTL Properties , 2019, 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE).

[10]  Blase Ur,et al.  Practical trigger-action programming in the smart home , 2014, CHI.

[11]  Brad A. Myers,et al.  Using HCI techniques to design a more usable programming system , 2002, Proceedings IEEE 2002 Symposia on Human Centric Computing Languages and Environments.

[12]  Michael Weber,et al.  Exploring End User Programming Needs in Home Automation , 2017, ACM Trans. Comput. Hum. Interact..

[13]  Daniela Fogli,et al.  Rule-based tools for the configuration of ambient intelligence systems: a comparative user study , 2017, Multimedia Tools and Applications.

[14]  Mark W. Newman,et al.  Devices and Data and Agents, Oh My , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[15]  Monica S. Lam,et al.  Almond: The Architecture of an Open, Crowdsourced, Privacy-Preserving, Programmable Virtual Assistant , 2017, WWW.

[16]  HENRY LIEBERMAN,et al.  End-User Development: An Emerging Paradigm , 2006, End User Development.

[17]  Fabio Paternò,et al.  Personalization of Context-Dependent Applications Through Trigger-Action Rules , 2017, ACM Trans. Comput. Hum. Interact..

[18]  W. Keith Edwards,et al.  At Home with Ubiquitous Computing: Seven Challenges , 2001, UbiComp.

[19]  Mary Shaw,et al.  The state of the art in end-user software engineering , 2011, ACM Comput. Surv..

[20]  John Zimmerman,et al.  Principles of Smart Home Control , 2006, UbiComp.

[21]  Yngve Dahl,et al.  End-User Composition Interfaces for Smart Environments: A Preliminary Study of Usability Factors , 2011, HCI.

[22]  Elliot Soloway,et al.  Preprogramming Knowledge: A Major Source of Misconceptions in Novice Programmers , 1985, Hum. Comput. Interact..

[23]  Atul Prakash,et al.  Decentralized Action Integrity for Trigger-Action IoT Platforms , 2018, NDSS.

[24]  Roy D. Pea,et al.  Language-Independent Conceptual “Bugs” in Novice Programming , 1986 .

[25]  Margaret M. Burnett,et al.  A Debugging Perspective on End-User Mashup Programming , 2010, 2010 IEEE Symposium on Visual Languages and Human-Centric Computing.

[26]  Daniel C. Dennett,et al.  Brainstorms: Philosophical Essays on Mind and Psychology , 1981 .

[27]  Maya Cakmak,et al.  RoboFlow: A flow-based visual programming language for mobile manipulation tasks , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[28]  Margaret M. Burnett,et al.  Impact of interruption style on end-user debugging , 2004, CHI.

[29]  Qi Wang,et al.  Fear and Logging in the Internet of Things , 2018, NDSS.

[30]  Mordechai Ben-Ari,et al.  Habits of programming in scratch , 2011, ITiCSE '11.

[31]  Mark W. Newman,et al.  Providing an Integrated User Experience of Networked Media, Devices, and Services through End-User Composition , 2009, Pervasive.

[32]  Fulvio Corno,et al.  HomeRules: A Tangible End-User Programming Interface for Smart Homes , 2015, CHI Extended Abstracts.

[33]  Eric Cheung,et al.  Toward user-centric feature composition for the Internet of Things , 2015, ArXiv.

[34]  Pablo A. Haya,et al.  Towards a Ubiquitous End-User Programming System for Smart Spaces , 2010, J. Univers. Comput. Sci..

[35]  Youn-Kyung Lim,et al.  User experience in do-it-yourself-style smart homes , 2015, UbiComp.

[36]  Donatella Sciuto,et al.  BuildingRules: a trigger-action based system to manage complex commercial buildings , 2015, UbiComp/ISWC Adjunct.

[37]  David Kim,et al.  Integrating the Smart Home into the Digital Calendar , 2016, CHI.

[38]  Jiyun Lee,et al.  Trigger-Action Programming in the Wild: An Analysis of 200,000 IFTTT Recipes , 2016, CHI.

[39]  Tanakorn Leesatapornwongsa,et al.  What Bugs Live in the Cloud? A Study of 3000+ Issues in Cloud Systems , 2014, SoCC.

[40]  Brad A. Myers,et al.  Designing the whyline: a debugging interface for asking questions about program behavior , 2004, CHI.

[41]  Atul Prakash,et al.  IFTTT vs. Zapier: A Comparative Study of Trigger-Action Programming Frameworks , 2017, ArXiv.

[42]  Timothy Sohn,et al.  iCAP: Interactive Prototyping of Context-Aware Applications , 2006, Pervasive.

[43]  Jeffrey Nichols,et al.  End-user programming of mashups with vegemite , 2009, IUI.

[44]  Pamela Zave,et al.  Locked or Not?: Mental Models of IoT Feature Interaction , 2017, CHI.

[45]  Eben M. Haber,et al.  CoScripter: automating & sharing how-to knowledge in the enterprise , 2008, CHI.

[46]  Elliot Soloway,et al.  Uncovering principles of novice programming , 1983, POPL '83.

[47]  Zhao Li,et al.  Systematically Debugging IoT Control System Correctness for Building Automation , 2016, BuildSys@SenSys.

[48]  Maya Cakmak,et al.  Supporting mental model accuracy in trigger-action programming , 2015, UbiComp.