Integration of Eye-Tracking Based Studies into e-Commerce Websites Evaluation Process with eQual and TOPSIS Methods

The evaluation of the e-commerce websites’ quality, usability and user experience is an important research task. Various survey-based evaluation methods are available, eQual being one of them. However, these methods are not free of disadvantages. In this paper, a novel approach is presented, where a single usability evaluation model is created on the basis of the eQual survey criteria and the perceptual evaluation data from eye tracking devices, thus extending the statistical survey model with objective gaze measurements and Multi-Criteria Decision Analysis (MCDA) methodology foundations. The combined data is processed with the crisp and fuzzy variants of the TOPSIS method to evaluate the websites. An empirical verification is performed and the results are presented. The results showed the benefits of the author’s proposed approach as well as the wide possibilities of interpretation of the obtained solutions.

[1]  Richard T. Vidgen,et al.  WebQual: An Exploration of Web-Site Quality , 2000, ECIS.

[2]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[3]  Ingoo Han,et al.  The impact of the online and offline features on the user acceptance of Internet shopping malls , 2004, Electron. Commer. Res. Appl..

[4]  Kun Chang Lee Ubiquitous decision support systems and their impact on users’ online decision-making , 2013 .

[5]  V. Zeithaml,et al.  E-S-QUAL A Multiple-Item Scale for Assessing Electronic Service Quality , 2004 .

[6]  Jinghua Huang,et al.  An e-commerce performance assessment model: Its development and an initial test on e-commerce applications in the retail sector of China , 2009, Inf. Manag..

[7]  Patrice Perny,et al.  Aiding Decisions with Multiple Criteria , 2002 .

[8]  Aïda Valls,et al.  ELECTRE-III-H: An outranking-based decision aiding method for hierarchically structured criteria , 2015, Expert Syst. Appl..

[9]  Daniel Cunliffe,et al.  Developing usable Web sites - a review and model , 2000, Internet Res..

[10]  Hokey Min International Supplier Selection , 1994 .

[11]  Pavel Štrach,et al.  Adaptation Needed: Eye-Tracking Study of Cross-Cultural Differences in Perception of B2B Websites , 2017 .

[12]  Da Ruan,et al.  A fuzzy preference-ranking model for a quality evaluation of hospital web sites: Research Articles , 2006 .

[13]  Jakob Nielsen,et al.  Chapter 4 – The Usability Engineering Lifecycle , 1993 .

[14]  E. Stanley Lee,et al.  An extension of TOPSIS for group decision making , 2007, Math. Comput. Model..

[15]  A. Milani,et al.  The effect of normalization norms in multiple attribute decision making models: a case study in gear material selection , 2005 .

[16]  A. Parasuraman,et al.  SERVQUAL: A multiple-item scale for measuring consumer perceptions of service quality. , 1988 .

[17]  David M. Nichols,et al.  Exploring Usability Discussions in Open Source Development , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[18]  Tolga Kaya,et al.  Multi-attribute Evaluation of Website Quality in E-business Using an Integrated Fuzzy AHPTOPSIS Methodology , 2010, Int. J. Comput. Intell. Syst..

[19]  Richard T. Vidgen,et al.  Measuring Web site quality improvements: a case study of the forum on strategic management knowledge exchange , 2003, Ind. Manag. Data Syst..

[20]  Kyeong Kang,et al.  Assessing Web Content Accessibility of E-Commerce Websites for People with Disabilities , 2016, ISD.

[21]  S. Akinci,et al.  Re-assessment of E-S-Qual and E-RecS-Qual in a pure service setting , 2010 .

[22]  Ching-Lai Hwang,et al.  A new approach for multiple objective decision making , 1993, Comput. Oper. Res..

[23]  Richard T. Vidgen,et al.  Data triangulation and web quality metrics: A case study in e-government , 2006, Inf. Manag..

[24]  Ching-Hsue Cheng,et al.  Fuzzy hierarchical TOPSIS for supplier selection , 2009, Appl. Soft Comput..

[25]  Leo Lentz,et al.  Website Evaluation Questionnaire: Development of a Research-Based Tool for Evaluating Informational Websites , 2007, EGOV.

[26]  J. Jacoby,et al.  Is There an Optimal Number of Alternatives for Likert Scale Items? Study I: Reliability and Validity , 1971 .

[27]  Alexandre N. Tuch,et al.  Location matters, especially for non-salient features-An eye-tracking study on the effects of web object placement on different types of websites , 2013, Int. J. Hum. Comput. Stud..

[28]  Lixuan Zhang,et al.  An Eyetracking Study on Online Hotel Decision Making: The Effects of Images and umber of Options , 2016 .

[29]  Harold W. Webb,et al.  SiteQual: an integrated measure of Web site quality , 2004, J. Enterp. Inf. Manag..

[30]  Li Chen,et al.  Eye-Tracking Study of User Behavior in Recommender Interfaces , 2010, UMAP.

[31]  Adel Guitouni,et al.  Tentative guidelines to help choosing an appropriate MCDA method , 1998, Eur. J. Oper. Res..

[32]  Golam Kabir,et al.  COMPARATIVE ANALYSIS OF TOPSIS AND FUZZY TOPSIS FOR THE EVALUATION OF TRAVEL WEBSITE SERVICE QUALITY , 2012 .

[33]  Kursat Cagiltay,et al.  Analyzing Turkish E-government Websites by Eye Tracking , 2013, 2013 Joint Conference of the 23rd International Workshop on Software Measurement and the 8th International Conference on Software Process and Product Measurement.

[34]  Madan M. Gupta,et al.  Introduction to Fuzzy Arithmetic , 1991 .

[35]  Stuart J. Barnes,et al.  The E-Qual Approach to the Assessment of E-commerce Quality: A Longitudinal Study of Internet Bookstores , 2004 .

[36]  Hsiu-Fen Lin,et al.  An application of fuzzy AHP for evaluating course website quality , 2010, Comput. Educ..

[37]  Artur Strzelecki,et al.  The use of management dashboard in monitoring the efficiency of the internet advertising campaigns illustrated on the example of Google Analytics , 2016 .

[38]  Øystein Nytrø,et al.  Usability Evaluation of Clinical Guidelines on the Web Using Eye-Tracker , 2016, MIE.

[39]  Jaroslaw Jankowski,et al.  PEQUAL - E-commerce websites quality evaluation methodology , 2016, 2016 Federated Conference on Computer Science and Information Systems (FedCSIS).

[40]  Nils Magne Larsen,et al.  Consumer attention to price in social commerce: Eye tracking patterns in retail clothing☆ , 2016 .

[41]  R. Słowiński,et al.  Aiding decisions with multiple criteria: essays in honor of Bernard Roy , 2002 .

[42]  Da Ruan,et al.  A fuzzy preference‐ranking model for a quality evaluation of hospital web sites , 2006, Int. J. Intell. Syst..

[43]  Gustavo Rossi,et al.  Assessing refactorings for usability in e-commerce applications , 2016, Empirical Software Engineering.

[44]  Matthias Ehrgott,et al.  Multiple criteria decision analysis: state of the art surveys , 2005 .

[45]  Anabela Pereira Tereso,et al.  A Multicriteria Decision Aid Software Application for selecting MCDA Software using AHP , 2010 .

[46]  Kyeong Kang,et al.  E-Commerce Web Accessibility for People with Disabilities , 2017 .

[47]  Zheng Zhou,et al.  Development and validation of an instrument to measure user perceived service quality of information presenting Web portals , 2005, Inf. Manag..

[48]  Chen-Tung Chen,et al.  Extensions of the TOPSIS for group decision-making under fuzzy environment , 2000, Fuzzy Sets Syst..

[49]  S French,et al.  Multicriteria Methodology for Decision Aiding , 1996 .

[50]  V. Zeithaml,et al.  A Dynamic Process Model of Service Quality: From Expectations to Behavioral Intentions , 1993 .

[51]  Richard T. Vidgen,et al.  Data Triangulation in Action: Using Comment Analysis to Refine Web Quality Metrics , 2005, ECIS.

[52]  Younghwa Lee,et al.  Investigating the effect of website quality on e-business success: An analytic hierarchy process (AHP) approach , 2006, Decis. Support Syst..

[53]  Helene Gelderblom,et al.  The value of comparative usability and UX evaluation for e-commerce organisations , 2016, CONF-IRM.

[54]  Richard T. Vidgen,et al.  An Evaluation of Cyber-Bookshops: The WebQual Method , 2001, Int. J. Electron. Commer..

[55]  Chia-Chi Sun,et al.  Using fuzzy TOPSIS method for evaluating the competitive advantages of shopping websites , 2009, Expert Syst. Appl..

[56]  Francisco Muñoz-Leiva,et al.  What type of online advertising is most effective for eTourism 2.0? An eye tracking study based on the characteristics of tourists , 2015, Comput. Hum. Behav..

[57]  S. Treu,et al.  Need for multi-aspect measures to support evaluation of complex human-computer interfaces , 1998, Proceedings Fourth Annual Symposium on Human Interaction with Complex Systems.

[58]  Linda A. Webb,et al.  Business to Consumer Electronic Commerce Website Quality: Integrating Information and Service Dimensions , 2001 .

[59]  Kun Chang Lee,et al.  Exploring the effect of the human brand on consumers' decision quality in online shopping: An eye‐tracking approach , 2013 .

[60]  Linden J. Ball,et al.  An Eye Movement Analysis of Web Page Usability , 2002 .