How Experienced Designers of Enterprise Applications Engage AI as a Design Material

HCI research has explored AI as a design material, suggesting that designers can envision AI’s design opportunities to improve UX. Recent research claimed that enterprise applications offer an opportunity for AI innovation at the user experience level. We conducted design workshops to explore the practices of experienced designers who work on cross-functional AI teams in the enterprise. We discussed how designers successfully work with and struggle with AI. Our findings revealed that designers can innovate at the system and service levels. We also discovered that making a case for an AI feature’s return on investment is a barrier for designers when they propose AI concepts and ideas. Our discussions produced novel insights on designers’ role on AI teams, and the boundary objects they used for collaborating with data scientists. We discuss the implications of these findings as opportunities for future research aiming to empower designers in working with data and AI.

[1]  Michael A. Madaio,et al.  Stakeholder Participation in AI: Beyond "Add Diverse Stakeholders and Stir" , 2021, ArXiv.

[2]  Yuan Lu,et al.  ENGAGING END USERS IN AN AI-ENABLED SMART SERVICE DESIGN - THE APPLICATION OF THE SMART SERVICE BLUEPRINT SCAPE (SSBS) FRAMEWORK , 2021, Proceedings of the Design Society.

[3]  Volker Wulf,et al.  Alexa, We Need to Talk: A Data Literacy Approach on Voice Assistants , 2021, Conference on Designing Interactive Systems.

[4]  William Odom,et al.  A Design Inquiry into Introspective AI: Surfacing Opportunities, Issues, and Paradoxes , 2021, Conference on Designing Interactive Systems.

[5]  Virpi Roto,et al.  The Overlaps and Boundaries Between Service Design and User Experience Design , 2021, Conference on Designing Interactive Systems.

[6]  Mark Evans,et al.  Design Heuristics for Artificial Intelligence: Inspirational Design Stimuli for Supporting UX Designers in Generating AI-Powered Ideas , 2021, CHI Extended Abstracts.

[7]  Carrie J. Cai,et al.  Onboarding Materials as Cross-functional Boundary Objects for Developing AI Assistants , 2021, CHI Extended Abstracts.

[8]  Adam Fourney,et al.  Planning for Natural Language Failures with the AI Playbook , 2021, CHI.

[9]  Eytan Adar,et al.  Towards A Process Model for Co-Creating AI Experiences , 2021, Conference on Designing Interactive Systems.

[10]  Abeba Birhane,et al.  Algorithmic injustice: a relational ethics approach , 2021, Patterns.

[11]  Soya Park,et al.  How AI Developers Overcome Communication Challenges in a Multidisciplinary Team , 2021, Proc. ACM Hum. Comput. Interact..

[12]  Arne Berger,et al.  Machine Learning Uncertainty as a Design Material: A Post-Phenomenological Inquiry , 2021, CHI.

[13]  Alex Kass,et al.  UX designers pushing AI in the enterprise , 2020, Interactions.

[14]  Elisa Giaccardi,et al.  Technology and More-Than-Human Design , 2020, Design Issues.

[15]  M. Søndergaard,et al.  More-Than-Human Design and AI: In Conversation with Agents , 2020, Conference on Designing Interactive Systems.

[16]  Laura Forlano,et al.  Participation Is not a Design Fix for Machine Learning , 2020, EAAMO.

[17]  Michael Carl Tschantz,et al.  Human-Centered Approaches to Fair and Responsible AI , 2020, CHI Extended Abstracts.

[18]  Colin M. Gray,et al.  Dimensions of UX Practice that Shape Ethical Awareness , 2020, CHI.

[19]  Hanna M. Wallach,et al.  Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI , 2020, CHI.

[20]  Graham Dove,et al.  Monsters, Metaphors, and Machine Learning , 2020, CHI.

[21]  Paul Coulton,et al.  Researching AI Legibility through Design , 2020, CHI.

[22]  Brian Magerko,et al.  What is AI Literacy? Competencies and Design Considerations , 2020, CHI.

[23]  Qian Yang,et al.  Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design , 2020, CHI.

[24]  Katerina Gorkovenko,et al.  Exploring The Future of Data-Driven Product Design , 2020, CHI.

[25]  Amy X. Zhang,et al.  How do Data Science Workers Collaborate? Roles, Workflows, and Tools , 2020, Proc. ACM Hum. Comput. Interact..

[26]  Q. Liao,et al.  Questioning the AI: Informing Design Practices for Explainable AI User Experiences , 2020, CHI.

[27]  Zachary C. Lipton,et al.  Algorithmic Fairness from a Non-ideal Perspective , 2020, AIES.

[28]  Greg Walsh,et al.  AI + Co-Design: Developing a Novel Computer-supported Approach to Inclusive Design , 2019, CSCW Companion.

[29]  Kush R. Varshney,et al.  How Data ScientistsWork Together With Domain Experts in Scientific Collaborations , 2019, Proc. ACM Hum. Comput. Interact..

[30]  Elisa Giaccardi,et al.  Designing and Prototyping from the Perspective of AI in the Wild , 2019, Conference on Designing Interactive Systems.

[31]  Karey Helms,et al.  Do You Have to Pee?: A Design Space for Intimate and Somatic Data , 2019, Conference on Designing Interactive Systems.

[32]  Qian Yang,et al.  Sketching NLP: A Case Study of Exploring the Right Things To Design with Language Intelligence , 2019, CHI.

[33]  Douglas Eck,et al.  Identifying the Intersections: User Experience + Research Scientist Collaboration in a Generative Machine Learning Interface , 2019, CHI Extended Abstracts.

[34]  Paul N. Bennett,et al.  Will You Accept an Imperfect AI?: Exploring Designs for Adjusting End-user Expectations of AI Systems , 2019, CHI.

[35]  Paul N. Bennett,et al.  Guidelines for Human-AI Interaction , 2019, CHI.

[36]  Alec Scharff,et al.  Triptech: A Method for Evaluating Early Design Concepts , 2019, CHI Extended Abstracts.

[37]  Miroslav Dudík,et al.  Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need? , 2018, CHI.

[38]  Steven J. Jackson,et al.  Trust in Data Science , 2018, Proc. ACM Hum. Comput. Interact..

[39]  Wendy Ju,et al.  Cybernetics and the design of the user experience of AI systems , 2018, Interactions.

[40]  Philip van Allen,et al.  Prototyping ways of prototyping AI , 2018, Interactions.

[41]  Jason Shun Wong,et al.  Design and fiction , 2018, Interactions.

[42]  Henriette Cramer,et al.  Assessing and Addressing Algorithmic Bias - But Before We Get There , 2018, AAAI Spring Symposia.

[43]  Benjamin R. Cowan,et al.  Design guidelines for hands-free speech interaction , 2018, MobileHCI Adjunct.

[44]  Jodi Forlizzi,et al.  Moving beyond user-centered design , 2018, Interactions.

[45]  Daria Loi,et al.  PD manifesto for AI futures , 2018, PDC.

[46]  T. Jylkäs,et al.  AI Assistants as Non-human Actors in Service Design , 2018 .

[47]  Gerd Kortuem,et al.  Design Enquiry Through Data: Appropriating a Data Science Workflow for the Design Process , 2018 .

[48]  John Zimmerman,et al.  Investigating How Experienced UX Designers Effectively Work with Machine Learning , 2018, Conference on Designing Interactive Systems.

[49]  Lone Koefoed Hansen,et al.  Intimate Futures: Staying with the Trouble of Digital Personal Assistants through Design Fiction , 2018, Conference on Designing Interactive Systems.

[50]  Allison Woodruff,et al.  A Qualitative Exploration of Perceptions of Algorithmic Fairness , 2018, CHI.

[51]  John Zimmerman,et al.  Mapping Machine Learning Advances from HCI Research to Reveal Starting Places for Design Innovation , 2018, CHI.

[52]  William Odom,et al.  On the Design of OLO Radio: Investigating Metadata as a Design Material , 2018, CHI.

[53]  Michael Veale,et al.  Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making , 2018, CHI.

[54]  Jun Zhao,et al.  'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions , 2018, CHI.

[55]  Min Kyung Lee.  Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management , 2018, Big Data Soc..

[56]  Deborah E. White,et al.  Thematic Analysis , 2017 .

[57]  Lars Erik Holmquist,et al.  Intelligence on tap , 2017, Interactions.

[58]  Kim Halskov,et al.  UX Design Innovation: Challenges for Working with Machine Learning as a Design Material , 2017, CHI.

[59]  Elizabeth F. Churchill,et al.  Designing with Data: Improving the User Experience with A/B Testing , 2017 .

[60]  John Zimmerman,et al.  Planning Adaptive Mobile Experiences When Wireframing , 2016, Conference on Designing Interactive Systems.

[61]  Caroline Hummels,et al.  Connected Baby Bottle: A Design Case Study Towards a Framework for Data-Enabled Design , 2016, Conference on Designing Interactive Systems.

[62]  J. Oberlander,et al.  Designing from, with and by Data: Introducing the ablative framework , 2016 .

[63]  Carl F. DiSalvo,et al.  Data, Design and Civics: An Exploratory Study of Civic Tech , 2016, CHI.

[64]  Jacki O'Neill,et al.  Algorithms at Work: Empirical Diversity, Analytic Vocabularies, Design Implications , 2016, CSCW Companion.

[65]  Daniela Karin Rosner,et al.  Out of Time, Out of Place: Reflections on Design Workshops as a Research Method , 2016, CSCW.

[66]  Phoebe Sengers,et al.  Expanding and Refining Design and Criticality in HCI , 2015, CHI.

[67]  Sara Jones,et al.  Using data to stimulate creative thinking in the design of new products and services , 2014, Conference on Designing Interactive Systems.

[68]  Connor Upton,et al.  Greybox scheduling: designing a joint cognitive system for sustainable manufacturing , 2014, CHI Extended Abstracts.

[69]  John Zimmerman,et al.  Swarthmore College , 2012 .

[70]  John Zimmerman,et al.  How to support designers in getting hold of the immaterial material of software , 2010, CHI.

[71]  Charlotte P. Lee,et al.  Boundary Negotiating Artifacts: Unbinding the Routine of Boundary Objects and Embracing Chaos in Collaborative Work , 2007, Computer Supported Cooperative Work (CSCW).

[72]  John Zimmerman,et al.  Research through design as a method for interaction design research in HCI , 2007, CHI.

[73]  Neil A. M. Maiden,et al.  Towards a Framework for Integrating Agile Development and User-Centred Design , 2006, XP.

[74]  Leonard J. Bass,et al.  Identifying gaps between HCI, software engineering, and design, and boundary objects to bridge them , 2004, CHI EA '04.

[75]  Daniel Fallman,et al.  Design-oriented human-computer interaction , 2003, CHI '03.

[76]  Sara A. Bly,et al.  Design through matchmaking: technology in search of users , 1999, INTR.

[77]  Susan Leigh Star,et al.  Institutional Ecology, `Translations' and Boundary Objects: Amateurs and Professionals in Berkeley's Museum of Vertebrate Zoology, 1907-39 , 1989 .

[78]  Daniele Regoli,et al.  Design Methods for Artificial Intelligence Fairness and Transparency , 2021, IUI Workshops.

[79]  Fabien Girardin,et al.  When User Experience Designers Partner with Data Scientists , 2017, AAAI Spring Symposia.

[80]  Patrick Hebron,et al.  Machine Learning for Designers , 2016 .

[81]  Hugh Dubberly,et al.  Cybernetics and Design: Conversations for Action , 2015, Cybern. Hum. Knowing.

[82]  Holger Rhinow,et al.  Design Prototypes as Boundary Objects in Innovation Processes , 2012 .

[83]  Perry R. Cook,et al.  A Meta-Instrument for Interactive, On-the-Fly Machine Learning , 2009, NIME.

[84]  Ezio Manzini,et al.  New Design Knowledge , 2008 .

[85]  方华.  google,我,萨娜 , 2006 .

[86]  A. Jameson.  Adaptive interfaces and agents , 2002 .