The usefulness of usability and user experience evaluation methods on an e-Learning platform development from a developer's perspective: A case study

The development of a web platform is a complex and interdisciplinary task, where people with different roles such as project manager, designer or developer participate. Different usability and User Experience evaluation methods can be used in each stage of the development life cycle, but not all of them have the same influence in the software development and in the final product or system. This article presents the study of the impact of these methods applied in the context of an e-Learning platform development. The results show that the impact has been strong from a developer's perspective. Developer team members considered that usability and User Experience evaluation allowed them mainly to identify design mistakes, improve the platform's usability and understand the end users and their needs in a better way. Interviews with potential users, clickmaps and scrollmaps were rated as the most useful methods. Finally, these methods were considered unanimously very useful in the context of the entire software development, only comparable to SCRUM meetings and overcoming the rest of involved factors.

[1]  Timo Jokela.  The Early Phases of UX: Why They are Important (More than Evaluation), and What They are , 2012, I-UxSED.

[2]  Lim Tek Yong.  User experience evaluation methods for mobile devices , 2013, Third International Conference on Innovative Computing Technology (INTECH 2013).

[3]  Kasper Hornbæk,et al.  Exploring the Value of Usability Feedback Formats , 2009, Int. J. Hum. Comput. Interact..

[4]  Evangelos Bekiaris,et al.  A UCD Approach towards the Design, Development and Assessment of Accessible Applications in a Large Scale European Integrated Project , 2009, HCI.

[5]  Carmelo Ardito,et al.  Are Software Companies Aware of UX? , 2012, I-UxSED.

[6]  John M. Carroll,et al.  Five reasons for scenario-based design , 2000, Interact. Comput..

[7]  Linda Rising,et al.  The Scrum Software Development Process for Small Teams , 2000, IEEE Softw..

[8]  Joseph S. Dumas,et al.  Oracle Corporation , 1994 .

[9]  Aldo Gordillo,et al.  Virtual Science Hub: An Open Source Platform to Enrich Science Teaching , 2014 .

[10]  Evangelos Karapanos,et al.  Reconstructing Experiences through Sketching , 2009, ArXiv.

[11]  Silvia Abrahão,et al.  Interplay between User Experience Evaluation and System Development: State of the Art , 2012 .

[12]  Evangelos Karapanos,et al.  User experience over time , 2008, CHI Extended Abstracts.

[13]  Juan Quemada,et al.  Facilitating the creation of K-12 interactive learning objects using a multi device web tool , 2012, 2012 Frontiers in Education Conference Proceedings.

[14]  David Thomas,et al.  Agile web development with Rails - a pragmatic guide , 2005 .

[15]  Michail N. Giannakos,et al.  The Evaluation of an e-Learning Web-based Platform , 2010, CSEDU.

[16]  Mike Kuniavsky,et al.  Observing the User Experience: A Practitioner's Guide to User Research (Second Edition) , 2013, IEEE Transactions on Professional Communication.

[17]  Angelika Foerster.  Designing Pleasurable Products An Introduction To The New Human Factors , 2016 .

[18]  Virpi Roto,et al.  USER EXPERIENCE WHITE PAPER Bringing clarity to the concept of user experience , 2011 .

[19]  Kristina Höök,et al.  The sensual evaluation instrument: developing an affective evaluation tool , 2006, CHI.

[20]  Joseph Krajcik,et al.  Learner-Centered Design: Reflections and New Directions , 2001 .

[21]  Robert C. Williges,et al.  Criteria For Evaluating Usability Evaluation Methods , 2003, Int. J. Hum. Comput. Interact..

[22]  M. W. van Someren,et al.  The think aloud method: a practical approach to modelling cognitive processes , 1994 .

[23]  Daniel Gallego,et al.  A web tool to create educational content with gaming visualization , 2012, 2012 Frontiers in Education Conference Proceedings.

[24]  Maarten van Someren,et al.  The Think Aloud Method: A Practical Guide to Modelling Cognitive Processes , 1994 .

[25]  Oscar Mauricio Serrano Jaimes,et al.  EVALUACION DE LA USABILIDAD EN SITIOS WEB, BASADA EN EL ESTANDAR ISO 9241-11 (International Standard (1998) Ergonomic requirements For office work with visual display terminals (VDTs)-Parts II: Guidance on usability , 2012 .

[26]  Aldo Gordillo,et al.  ENHANCING K-12 SCIENCE EDUCATION THROUGH A MULTI-DEVICE WEB TOOL TO FACILITATE CONTENT INTEGRATION AND E-INFRASTRUCTURE ACCESS , 2013 .

[27]  Pieter Desmet,et al.  Measuring Emotion: Development and Application of an Instrument to Measure Emotional Responses to Products , 2005, Funology.

[28]  Craig Larman,et al.  Agile and Iterative Development: A Manager's Guide , 2003 .

[29]  Ergonomic requirements for office work with visual display terminals ( VDTs ) — Part 11 : Guidance on usability , 1998 .

[30]  Austin Henderson,et al.  What you see, some of what's in the future, and how we go about doing it: HI at Apple Computer , 1995, CHI 95 Conference Companion.

[31]  Kuldeep Kelkar,et al.  When user experience met agile: a case study , 2009, CHI Extended Abstracts.

[32]  Virpi Roto,et al.  Understanding, scoping and defining user experience: a survey approach , 2009, CHI.

[33]  Dennis R. Wixon,et al.  Changing the Rules: A Pragmatic Approach to Product Development , 1993, Digit. Tech. J..

[34]  Danilo Caivano,et al.  Usability evaluation: a survey of software development organizations , 2011, SEKE.

[35]  Constantine Stephanidis,et al.  Universal Access in Human-Computer Interaction , 2011 .

[36]  Teresa Roselli,et al.  An approach to usability evaluation of e-learning applications , 2006, Universal Access in the Information Society.

[37]  John Zimmerman,et al.  User experience over time: an initial framework , 2009, CHI.

[38]  Virpi Roto,et al.  User experience evaluation methods: current state and development needs , 2010, NordiCHI.

[39]  Mike Kuniavsky,et al.  Observing the User Experience: A Practitioner's Guide to User Research (Morgan Kaufmann Series in Interactive Technologies) (The Morgan Kaufmann Series in Interactive Technologies) , 2003 .

[40]  Virpi Roto,et al.  10373 Abstracts Collection - Demarcating User eXperience , 2010, Demarcating User eXperience.

[41]  Michael J. Muller,et al.  Participatory design , 1993, CACM.

[42]  Gregory D. Abowd,et al.  Human-Computer Interaction (3rd Edition) , 2003 .