Paper2Wire – A Case Study of User-Centred Development of Machine Learning Tools for UX Designers

This paper reflects on a case study of a user-centred concept development process for a Machine Learning (ML) based design tool, conducted at an industry partner. The resulting concept uses ML to match graphical user interface elements in sketches on paper to their digital counterparts to create consistent wireframes. A user study (N=20) with a working prototype shows that this concept is preferred by designers, compared to the previous manual procedure. Reflecting on our process and findings we discuss lessons learned for developing ML tools that respect practitioners' needs and practices.

[1]  Antonios Liapis,et al.  Mixed-Initiative Creative Interfaces , 2017, CHI Extended Abstracts.

[2]  Kaj Grønbæk,et al.  Cooperative Prototyping: Users and Designers in Mutual Activity , 1990, Int. J. Man Mach. Stud..

[3]  Albrecht Schmidt.  Technologies to Amplify the Mind , 2017, Computer.

[4]  Andreas Sonderegger,et al.  The influence of prototype fidelity and aesthetics of design in usability tests: effects on user behaviour, subjective evaluation and emotion. , 2009, Applied ergonomics.

[5]  N. Hoffart.  Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory , 2000 .

[6]  Ben Shneiderman,et al.  User interfaces for creativity support tools , 1999, Creativity & Cognition.

[7]  John Zimmerman,et al.  Mapping Machine Learning Advances from HCI Research to Reveal Starting Places for Design Innovation , 2018, CHI.

[8]  Eric Horvitz,et al.  Principles of mixed-initiative user interfaces , 1999, CHI '99.

[9]  Antti Oulasvirta,et al.  User Interface Design with Combinatorial Optimization , 2017, Computer.

[10]  James A. Landay,et al.  Sketching Interfaces: Toward More Human Interface Design , 2001, Computer.

[11]  Tony Beltramelli,et al.  pix2code: Generating Code from a Graphical User Interface Screenshot , 2017, EICS.

[12]  Mira Dontcheva,et al.  Rewire: Interface Design Assistance from Examples , 2018, CHI.

[13]  Kim Halskov,et al.  UX Design Innovation: Challenges for Working with Machine Learning as a Design Material , 2017, CHI.

[14]  Tong Lu,et al.  Sketching Interfaces , 2014, Handbook of Document Image Processing and Recognition.

[15]  Mark W. Newman,et al.  Sitemaps, storyboards, and specifications: a sketch of Web site design practice , 2000, DIS '00.

[16]  Jakob Nielsen,et al.  Comparative design review: an exercise in parallel design , 1993, INTERCHI.

[17]  Antti Oulasvirta,et al.  Sketchplore: Sketch and Explore with a Layout Optimiser , 2016, Conference on Designing Interactive Systems.

[18]  Levent Burak Kara,et al.  An image-based, trainable symbol recognizer for hand-drawn sketches , 2005, Comput. Graph..

[19]  A. Royalty,et al.  Destination, Imagination and the Fires Within: Design Thinking in a Middle School Classroom , 2010 .

[20]  Antti Oulasvirta,et al.  Aalto Interface Metrics (AIM): A Service and Codebase for Computational GUI Evaluation , 2018, UIST.

[21]  Yang Liu,et al.  From UI Design Image to GUI Skeleton: A Neural Machine Translator to Bootstrap Mobile GUI Implementation , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[22]  James A. Landay,et al.  High-Fidelity or Low-Fidelity, Paper or Computer? Choosing Attributes when Testing Web Prototypes , 2002 .

[23]  Qian Yang,et al.  The Role of Design in Creating Machine-Learning-Enhanced User Experience , 2017, AAAI Spring Symposia.

[24]  Janin Koch,et al.  Design implications for Designing with a Collaborative AI , 2017, AAAI Spring Symposia.

[25]  W. Mackay,et al.  Prototyping tools and techniques , 2002 .

[26]  Anselm L. Strauss,et al.  Basics of qualitative research : techniques and procedures for developing grounded theory , 1998 .

[27]  Martin Schrepp,et al.  Construction and Evaluation of a User Experience Questionnaire , 2008, USAB.