Data logging plus e-diary: towards an online evaluation approach of mobile service field trial

Many field study methods such as usability test in fields, contextual inquiry, and ethnographic interview can be applied to evaluate user experience of concepts in trials; however, most of such traditional field study methods suffer from weaknesses like resource demanding, time consuming, lack of measurement for longitudinal usage or lack of channels to collect user feedbacks in real time. In this paper, we propose an approach to combine data logging with e-diary for user experience evaluation in field trials. An evaluation case on a mobile service concept is also presented to show this approach.

[1]  Mark Guzdial,et al.  Analyzing and Visualizing Log Files: A Computational Science of Usability , 1994 .

[2]  David F. Redmiles,et al.  Extracting usability information from user interface events , 2000, CSUR.

[3]  Connor Graham,et al.  A Review of Mobile HCI Research Methods , 2003, Mobile HCI.

[4]  Lars Bo Larsen,et al.  Evaluating the usefulness of mobile services based on captured usage data from longitudinal field trials , 2007, Mobility '07.

[5]  N. Bolger,et al.  Diary methods: capturing life as it is lived. , 2003, Annual review of psychology.

[6]  Mary Czerwinski,et al.  A diary study of task switching and interruptions , 2004, CHI.

[7]  Fabio Paternò,et al.  Automatic Support for Usability Evaluation , 1998, IEEE Trans. Software Eng..

[8]  Stephanie Rosenbaum,et al.  Alternative methods for field usability research , 2003, SIGDOC '03.

[9]  Robert E. Kraut,et al.  HomeNet: a field trial of residential Internet services , 1996, CHI.

[10]  Kasper Løvborg Jensen RECON: capturing mobile and ubiquitous interaction in real contexts , 2009, Mobile HCI.

[11]  Joseph B. Walther,et al.  The Value of Web Log Data in Use-Based Design and Testing , 2006, J. Comput. Mediat. Commun..

[12]  Marti A. Hearst,et al.  The state of the art in automating usability evaluation of user interfaces , 2001, CSUR.

[13]  Leysia Palen,et al.  Voice-mail diary studies for naturalistic data capture under mobile conditions , 2002, CSCW '02.

[14]  Manfred Tscheligi,et al.  Experiencing real-world interaction: results from a NFC user experience field trial , 2007, Mobile HCI.

[15]  James D. Hollan,et al.  A diary study of mobile information needs , 2008, CHI.