An Experimental Usability Test for different Destination Recommender Systems

The present paper outlines the experimental evaluation of travel recommendation systems. First, theoretical concepts concentrating on the influencing factors for human-computer interaction, system usage and satisfaction are reviewed. An introduction of various methods dealing with usability evaluation is given and an overview of different “standard” survey instruments is provided. Second, a case study, the assessment of a travel recommender system currently under development, is presented. The evaluation considers aspects such as design and layout, functionality or ease of use. These measures obtained by a user questionnaire are combined with user interaction logging data. Different variants of the travel recommendation system and a baseline system were used for the assessment. This promising approach complements subjective ratings by objective tracking data to obtain a more thorough picture of the system’s weaknesses. Finally, findings are presented and an explanatory model for user/system satisfaction is proposed.

[1]  Kent L. Norman,et al.  Development of an instrument measuring user satisfaction of the human-computer interface , 1988, CHI '88.

[2]  Sirpa Riihiaho,et al.  Experiences with Usability Evaluation Methods , 2000 .

[3]  Werner Winiwarter,et al.  PROVIDING MULTILINGUAL NATURAL LANGUAGE ACCESS TO TOURISM INFORMATION , 2001 .

[4]  Paul R. Cohen,et al.  The evolution and performance of the GRANT System , 1987, IEEE Expert.

[5]  Helmut Berger,et al.  Activation on the Move: Querying Tourism Information via Spreading Activation , 2003, DEXA.

[6]  James R. Lewis,et al.  IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use , 1995, Int. J. Hum. Comput. Interact..

[7]  Gitte Lindgaard.  Usability testing and system evaluation - a guide for designing useful computer systems , 1994, Chapman and Hall computing series.

[8]  Fabio Crestani,et al.  Application of Spreading Activation Techniques in Information Retrieval , 1997, Artificial Intelligence Review.

[9]  Fabio Crestani,et al.  Searching the web by constrained spreading activation , 2000, Inf. Process. Manag..

[10]  Helmut Berger,et al.  A Natural Language Query Interface for Tourism Information , 2003, ENTER.

[11]  P. Pavlou,et al.  Consumer Acceptance of Electronic Commerce: Integrating Trust and Risk with the Technology Acceptance Model , 2003 .

[12]  Werner Retschitzegger,et al.  Beyond Traditional Tourism Information Systems - TIScover , 1998, J. Inf. Technol. Tour..

[13]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[14]  V. Hirvisalo.  HELSINKI UNIVERSITY OF TECHNOLOGY Laboratory of Information Processing Science , 2022 .

[15]  Albert L. Lederer,et al.  The technology acceptance model and the World Wide Web , 2000, Decis. Support Syst..

[16]  Donna L. Hoffman,et al.  Measuring the Customer Experience in Online Environments: A Structural Modeling Approach , 2000 .

[17]  Paul A. Pavlou,et al.  Consumer Acceptance of Electronic Commerce: Integrating Trust and Risk with the Technology Acceptance Model , 2003, Int. J. Electron. Commer..

[18]  Miyoung Jeong,et al.  Adaptation of an information quality framework to measure customers’ behavioral intentions to use lodging Web sites , 2001 .

[19]  Helmut Berger,et al.  What customers really want to know from tourism information systems but never dared to ask , 2002 .

[20]  Wilbert O. Galitz,et al.  The Essential Guide to User Interface Design , 2002 .

[21]  Francesco Ricci,et al.  DIETORECS: Travel Advisory for Multiple Decision Styles , 2003, ENTER.

[22]  Jakob Nielsen,et al.  Noncommand user interfaces , 1993, CACM.