Measuring the User Engagement Process

Engagement is a quality of user experience that facilitates more enriching interactions with computer applications. It is defined by a core set of attributes: aesthetic appeal, novelty, involvement, focused attention, perceived usability, and endurability. The ability to engage users influences the products they purchase, the websites they use, and the decisions they make regarding what they will use in future and what they will recommend to others. Engagement is clearly an important component of user experience, but like other components, it is somewhat intangible, and thus difficult to measure and evaluate. This workshop paper outlines previous research that has focused on the evaluation of engagement as an outcome of experience. We propose that focusing on measuring the process of engagement is the crucial next step. To assess whether or not users are engaged while using an application and what aspects of the system engage them, we must employ mixed methodologies to capture the cognitive, affective, and behavioural components of the experience. But which methods are most appropriate, and how can they be used in concert? Addressing these questions will allow us to understand the nature of engagement and inform design.

[1]  John C. McCarthy,et al.  Technology as experience , 2004, INTR.

[2]  C. Bartneck,et al.  Measuring the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots , 2008, HRI 2008.

[3]  Diane H. Sonnenwald,et al.  Investigating Information Seeking Behavior Using the Concept of Information Horizons , 2001 .

[4]  Martin Oliver,et al.  Activity theory and learning from digital games: implications for game design , 2006 .

[5]  Caroline Hummels,et al.  Let's Make Things Engaging , 2005, Funology.

[6]  Regan L. Mandryk,et al.  A continuous and objective evaluation of emotional experience with interactive play environments , 2006, CHI.

[7]  James D. Hollan,et al.  Direct Manipulation Interfaces , 1985, Hum. Comput. Interact..

[8]  Richard Hull,et al.  Designing Engaging Experiences with Children and Artists , 2005, Funology.

[9]  Minoru Nakayama,et al.  The response of eye-movement and pupil size to audio instruction while viewing a moving target , 2000, ETRA.

[10]  Jane Webster,et al.  Audience engagement in multimedia presentations , 1997, DATB.

[11]  Peter Chapman,et al.  Engagement in multimedia training systems , 1999, Proceedings of the 32nd Annual Hawaii International Conference on Systems Sciences. 1999. HICSS-32. Abstracts and CD-ROM of Full Papers.

[12]  Erik Champion,et al.  Applying game design theory to virtual heritage environments , 2003, GRAPHITE '03.

[13]  Karon E. MacLean,et al.  Exploring affective design for physical controls , 2007, CHI.

[14]  Richard David Jacques,et al.  The nature of engagement and its role in hypermedia evaluation and design , 1996 .

[15]  Tom Carey,et al.  Engagement as a Design Concept for Multimedia , 1995 .

[16]  Cristina Conati,et al.  Data-Driven Refinement of a Probabilistic Model of User Affect , 2005, User Modeling.

[17]  Paul Cairns,et al.  Engagement with an Interactive Museum Exhibit , 2005, BCS HCI.

[18]  Pekka Makkonen,et al.  Does collaborative hypertext support better engagement in learning of the basics in informatics? , 1997, ITiCSE '97.