Effective use of human physiological metrics to evaluate website usability: An empirical investigation from China

Purpose The evaluation of website usability is the precondition and a critical step for website design and optimization. The purpose of this paper is to investigate and provide empirical evidence of the interrelationships between human physiological metrics and website usability. This study examines how eye-movement metrics and heart rate variability (HRV) evaluate website usability, and then affect users’ online surfing behavior. Design/methodology/approach A physiological measurement experiment is designed to collect participants’ physiological metrics. This paper proposes an objective measurement model for website usability, and partial least squares is used to analyze the measurement and structural models, based on data collected from 200 participants who had experienced online surfing at least four times. Findings The analysis supports partially or fully 28 of the 31 hypotheses formulated. The study reveals that human physiological metrics (i.e. fixation duration, fixation count, blink count, HRV) have a strong explanatory ability for website usability. Research limitations/implications Data for this study were collected only from mainland China. Therefore, participants may have been influenced by Chinese cultures. The generalizability of this study may be enhanced by collecting data from more diverse samples and validating the model on different cultures. Originality/value This study contributes significantly to the industry by providing empirical evidence of the interrelationship between human physiological metrics and website usability. The findings also provide managers with valuable insight into better understanding of the nature of these interrelationships.

[1]  Marisa E. Campbell,et al.  Conference preview: HCI 2000: usability or else! , 2000, INTR.

[2]  Kursat Cagiltay,et al.  Analyzing Turkish E-government Websites by Eye Tracking , 2013, 2013 Joint Conference of the 23rd International Workshop on Software Measurement and the 8th International Conference on Software Process and Product Measurement.

[3]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[4]  Nazmun Nahar,et al.  Interactions between contingency, organizational IT factors, and ERP success , 2009, Ind. Manag. Data Syst..

[5]  Alain Abran,et al.  Usability Meanings and Interpretations in ISO Standards , 2003, Software Quality Journal.

[6]  Albert Huang.  A RESEARCH TAXONOMY FOR E-COMMERCESYSTEM USABILITY , 2002 .

[7]  Stefan Steinmetz,et al.  Eye Tracking Evaluation Of Chinese Web Sites For The Chinese Market , 2011 .

[8]  Hsiu-Fen Lin,et al.  Contextual factors affecting knowledge management diffusion in SMEs , 2014, Ind. Manag. Data Syst..

[9]  Udo Konradt,et al.  Usability in online shops: scale construction, validation and the influence on the buyers’ intention and decision , 2003, Behav. Inf. Technol..

[10]  Oscar Mauricio Serrano Jaimes,et al.  EVALUACION DE LA USABILIDAD EN SITIOS WEB, BASADA EN EL ESTANDAR ISO 9241-11 (International Standard (1998) Ergonomic requirements For office work with visual display terminals (VDTs)-Parts II: Guidance on usability , 2012 .

[11]  İsmail Tonbuloğlu,et al.  Using Eye Tracking Method and Video Record in Usability Test of Educational Softwares and Gender Effects , 2013 .

[12]  Jia Tan,et al.  FOUUX : A Framework for Usability & User Experience , 2009 .

[13]  Chin-Lung Hsu,et al.  Acceptance of blog usage: The roles of technology acceptance, social influence and knowledge sharing motivation , 2008, Inf. Manag..

[14]  Joseph H. Goldberg,et al.  Eye Movements and Interface Component Grouping: An Evaluation Method , 1998 .

[15]  James R. Lewis,et al.  IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use , 1995, Int. J. Hum. Comput. Interact..

[16]  Wynne W. Chin,et al.  A Partial Least Squares Latent Variable Modeling Approach for Measuring Interaction Effects: Results from a Monte Carlo Simulation Study and an Electronic - Mail Emotion/Adoption Study , 2003, Inf. Syst. Res..

[17]  Judy H. Jeng.  What is usability in the context of the digital library and how can it be measured , 2005 .

[18]  Jae-Hyeon Ahn,et al.  Management of trust in the e-marketplace: the role of the buyer's experience in building trust , 2007, J. Inf. Technol..

[19]  Marshall Scott Poole,et al.  Affect in Web Interfaces: A Study of the Impacts of Web Page Visual Complexity and Order , 2010, MIS Q..

[20]  Rex B. Kline,et al.  Usability measurement and metrics: A consolidated model , 2006, Software Quality Journal.

[21]  C. Fornell,et al.  Evaluating structural equation models with unobservable variables and measurement error. , 1981 .

[22]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[23]  Rik Pieters,et al.  Attention Capture and Transfer in Advertising: Brand, Pictorial, and Text-Size Effects , 2004 .

[24]  D. A. Kenny,et al.  The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. , 1986, Journal of personality and social psychology.

[25]  Stephen Doherty,et al.  Eye tracking as an MT evaluation technique , 2010, Machine Translation.

[26]  Chiuhsiang Joe Lin,et al.  Evaluating camouflage design using eye movement data. , 2014, Applied ergonomics.

[27]  Thomas S. Tullis,et al.  Designing Noticeable Bricklets by Tracking Users' Eye Movements , 2012, 2012 45th Hawaii International Conference on System Sciences.

[28]  Robert F. Otondo,et al.  Reexamining Technology Acceptance in Online Task Behaviours , 2006, Electron. Mark..

[29]  Ergonomic requirements for office work with visual display terminals ( VDTs ) — Part 11 : Guidance on usability , 1998 .

[30]  Emilio Insfrán,et al.  A Systematic Review of Usability Evaluation in Web Development , 2008, WISE Workshops.

[31]  Chun-Heng Ho,et al.  Can pupil size be measured to assess design products , 2014 .

[32]  Thomas S. Tullis,et al.  Visual Hierarchy and Viewing Behavior: An Eye Tracking Study , 2011, HCI.

[33]  Wynne W. Chin.  The partial least squares approach for structural equation modeling. , 1998 .

[34]  Vincent G. Duffy,et al.  Evaluating the industrial ergonomics of service quality for online recruitment websites , 2005 .

[35]  Detmar W. Straub,et al.  Common Beliefs and Reality About PLS , 2014 .

[36]  Xianjun Qi,et al.  Members' satisfaction and continuance intention: a socio-technical perspective , 2015, Ind. Manag. Data Syst..

[37]  Mary Corbett,et al.  SUMI: the Software Usability Measurement Inventory , 1993, Br. J. Educ. Technol..

[38]  Roel Vertegaal,et al.  Explaining effects of eye gaze on mediated group conversations:: amount or synchronization? , 2002, CSCW '02.

[39]  Edward Cutrell,et al.  What are you looking for?: an eye-tracking study of information usage in web search , 2007, CHI.

[40]  M. Sarstedt,et al.  A new criterion for assessing discriminant validity in variance-based structural equation modeling , 2015 .

[41]  Tao Lin,et al.  Evaluating usability based on multimodal information: an empirical study , 2006, ICMI '06.

[42]  Vincent G. Duffy,et al.  Total quality management: An empirical test for mediation effect , 2001 .

[43]  Jan-Benedict E. M. Steenkamp,et al.  The use of LISREL in validating marketing constructs. , 1991 .

[44]  Jurek Kirakowski,et al.  Measuring the Usability of Web Sites , 1998 .

[45]  Linden J. Ball,et al.  Cueing retrospective verbal reports in usability testing through eye-movement replay , 2007, BCS HCI.

[46]  Paul M. Fitts,et al.  Eye movements of aircraft pilots during instrument-landing approaches. , 1950 .

[47]  Jieun Kim,et al.  Personality and its effects on learning performance: Design guidelines for an adaptive e-learning system based on a user model , 2013 .

[48]  Sri Hastuti Kurniawan,et al.  Increasing the Usability of Online Information for Older Users: A Case Study in Participatory Design , 2000, Int. J. Hum. Comput. Interact..

[49]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[50]  Alan Bryman,et al.  Understanding research for social policy and social work: themes, methods and approaches, 2nd edition , 2012 .

[51]  Cathleen Wharton,et al.  Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces , 1990, CHI '90.

[52]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[53]  Jörg Henseler,et al.  Consistent Partial Least Squares Path Modeling , 2015, MIS Q..

[54]  Jaeki Song,et al.  A Theoretical Approach to Web Design in E-Commerce: A Belief Reinforcement Model , 2005, Manag. Sci..

[55]  Jörg Henseler,et al.  Consistent and asymptotically normal PLS estimators for linear structural equations , 2014 .

[56]  Detmar W. Straub,et al.  The Relative Importance of Perceived Ease of Use in IS Adoption: A Study of E-Commerce Adoption , 2000, J. Assoc. Inf. Syst..

[57]  John G. Lynch,et al.  Reconsidering Baron and Kenny: Myths and Truths about Mediation Analysis , 2010 .

[58]  Steve Muylle,et al.  The conceptualization and empirical validation of web site user satisfaction , 2004, Inf. Manag..

[59]  Kent L. Norman,et al.  Development of an instrument measuring user satisfaction of the human-computer interface , 1988, CHI '88.

[60]  Jakob Nielsen,et al.  Heuristic evaluation of user interfaces , 1990, CHI '90.

[61]  Byung-Kwan Lee,et al.  The effect of information overload on consumer choice quality in an on-line environment , 2004 .

[62]  M. Just,et al.  Eye fixations and cognitive processes , 1976, Cognitive Psychology.

[63]  Silvia Mara Abrahão,et al.  Usability evaluation methods for the web: A systematic mapping study , 2011, Inf. Softw. Technol..

[64]  Joseph H. Goldberg,et al.  Computer interface evaluation using eye movements: methods and constructs , 1999 .

[65]  Linden J. Ball,et al.  Eye Tracking in Human-Computer Interaction and Usability Research : Current Status and Future Prospects , 2004 .

[66]  Christian Nitzl,et al.  Partial Least Squares Structural Equation Modelling (PLS-SEM) in Management Accounting Research: Directions for Future Theory Development , 2016 .

[67]  Jorgen P. Bansler,et al.  Corporate Intranet Implementation: Managing Emergent Technologies and Organizational Practices , 2000, J. Assoc. Inf. Syst..