UNSCALE: A Fuzzy-based Multi-criteria Usability Evaluation Framework for Measuring and Evaluating Library Websites

ABSTRACT Usability evaluation of websites has been one of the most important activities to determine user acceptance and to enrich the overall quality of the website. This research, drawing upon above concept, investigates the issues of website usability and proposes a fuzzy-based framework for measuring and evaluating the usability of websites, particularly library websites. The findings of the usability criteria elicitation, expert reviews, survey, and statistical analysis resulted in a comprehensive list of seven usability dimensions along with 20 measuring items which formed the basis of the evaluation framework abbreviate UNSCALE. The inputs of experts in the related field were used to probe weights of key dimensions and measuring items with the motive of examining their level of contribution to the usability index system. The study employed fuzzy theories along with the extent analysis, fuzzy analytic hierarchy process method with the expectation of scrutinizing the relative weightings. The framework was developed including eight cyclical steps for measuring and evaluating website usability by employing the fuzzy comprehensive evaluation method. It can be used to evaluate the overall usability score and also scores regarding each evaluation dimension. The framework was tested for its applicability and practicality on university library websites in Sri Lanka, a developing country with fast-growing internet accessibility. The findings of this research indicate that the proposed framework is useful for librarians in developing more useful library websites. On the other hand, it will provide vital insights to better appraise the outcomes of website usability for the scholars and researchers.

[1]  H Liu,et al.  Determine weights of evaluation indices for E-Commerce websites ranking based on fuzzy AHP , 2015 .

[2]  Melody Y. Ivory-Ndiaye.  An Empirical Approach to Automated Web Site Evaluation , 2003, J. Digit. Inf. Manag..

[3]  B. Teimourpour,et al.  Webometrics-based Analysis and Ranking of Iranian Hospital Websites , 2013 .

[4]  Soohyung Joo,et al.  Measuring the usability of academic digital libraries: Instrument development and validation , 2011, Electron. Libr..

[5]  Paul H. P. Yeow,et al.  Crucial web usability factors of 36 industries for students: a large-scale empirical study , 2011, Electron. Commer. Res..

[6]  Richard T. Vidgen,et al.  An Integrative Approach to the Assessment of E-Commerce Quality , 2002, J. Electron. Commer. Res..

[7]  Simone D. J. Barbosa,et al.  Introduction to Human-Computer Interaction , 2018, CHI Extended Abstracts.

[8]  Ankur Pant.  Usability evaluation of an academic library website: Experience with the Central Science Library, University of Delhi , 2015, Electron. Libr..

[9]  Feng Li,et al.  Evaluating the Usability and Content Usefulness of Web Sites: A Benchmarking Approach , 2005, J. Electron. Commer. Organ..

[10]  Alan G. Downe,et al.  Hybrid Usability Methodology: Integrating Heuristic Evaluation with Laboratory Testing across the Software Development Lifecycle , 2013, 2013 10th International Conference on Information Technology: New Generations.

[11]  C K Kwong,et al.  Determining the Importance Weights for the Customer Requirements in QFD Using a Fuzzy AHP with an Extent Analysis Approach , 2003 .

[12]  N. S. Harinarayana,et al.  An analysis of usability features of library web sites , 2008 .

[13]  Amit Kumar,et al.  Commentary on "Evaluating the criteria for human resource for science and technology (HRST) based on an integrated fuzzy AHP and fuzzy DEMATEL approach" , 2017, Appl. Soft Comput..

[14]  Victor B. Kreng,et al.  Evaluation of knowledge portal development tools using a fuzzy AHP approach: The case of Taiwanese stone industry , 2007, Eur. J. Oper. Res..

[15]  William Ho,et al.  Integrated analytic hierarchy process and its applications - A literature review , 2008, Eur. J. Oper. Res..

[16]  Elaine G. Toms,et al.  Usability of the Academic Library Web Site: Implications for Design , 2001 .

[17]  J. Nazari,et al.  TOPSIS approach to prioritize critical success factors of TQM , 2016 .

[18]  Sarah J Hammill.  Usability Testing at Florida International University Libraries: What We Learned , 2003 .

[19]  Kumru Didem Atalay,et al.  Multi‐Criteria Usability Evaluation of Electronic Devices in a Fuzzy Environment , 2014 .

[20]  Yong-Mi Kim,et al.  Users' perceptions of university library websites: A unifying view , 2011 .

[21]  Kelsey L. Johnson,et al.  Benchmarking: a tool for Web site evaluation and improvement , 1999, Internet Res..

[22]  Steve Hiller,et al.  Usability Testing, Interface Design, and Portals , 2005 .

[23]  Zhu Zhu,et al.  Usability Evaluation of Independent-Sales B2C Fashion Website Based on Consumer's Perspective * , 2013 .

[24]  Chin-Feng Lai,et al.  An evaluation model for digital libraries' user interfaces using fuzzy AHP , 2014, Electron. Libr..

[25]  Tom Stewart,et al.  Usability evaluation , 2009, Behav. Inf. Technol..

[26]  Parminder Kaur,et al.  Analysis of website usability evaluation methods , 2016, 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom).

[27]  Dhanapal Durai Dominic,et al.  Quality Evaluation of E-government Website Using Web Diagnostic Tools: Asian Case , 2009, 2009 International Conference on Information Management and Engineering.

[28]  Hepu Deng,et al.  Multicriteria analysis with fuzzy pairwise comparison , 1999, FUZZ-IEEE'99. 1999 IEEE International Fuzzy Systems. Conference Proceedings (Cat. No.99CH36315).

[30]  Ying-Chyi Chou,et al.  Evaluating the criteria for human resource for science and technology (HRST) based on an integrated fuzzy AHP and fuzzy DEMATEL approach , 2012, Appl. Soft Comput..

[31]  Hugh Preston,et al.  Usability analysis for redesign of a Caribbean academic library web site: a case study , 2009, OCLC Syst. Serv..

[32]  Nikola Kasabov,et al.  Foundations Of Neural Networks, Fuzzy Systems, And Knowledge Engineering [Books in Brief] , 1996, IEEE Transactions on Neural Networks.

[33]  Golam Kabir,et al.  COMPARATIVE ANALYSIS OF AHP AND FUZZY AHP M ODELS FOR M ULTICRITERIA INVENTORY CLASSIFICATION , 2011 .

[34]  Steven Buchanan,et al.  Evaluating the usability and usefulness of a digital library , 2009 .

[35]  Younghwa Lee,et al.  Understanding of website usability: Specifying and measuring constructs and their relationships , 2012, Decis. Support Syst..

[36]  Rajib Mall,et al.  Quality assurance of academic websites using usability testing: an experimental study with AHP , 2017, Int. J. Syst. Assur. Eng. Manag..

[37]  W. Pedrycz,et al.  A fuzzy extension of Saaty's priority theory , 1983 .

[38]  Ken Kwong-Kay Wong,et al.  Partial Least Squares Structural Equation Modeling (PLS-SEM) Techniques Using SmartPLS , 2013 .

[39]  Rex B. Kline,et al.  Usability measurement and metrics: A consolidated model , 2006, Software Quality Journal.

[40]  Soohyung Joo,et al.  A Usability Evaluation Model for Academic Library Websites: Efficiency, Effectiveness and Learnability , 2011 .

[41]  Jeng-Fung Chen,et al.  Evaluating teaching performance based on fuzzy AHP and comprehensive evaluation approach , 2015, Appl. Soft Comput..

[42]  Joseph F. Hair,et al.  Partial Least Squares : The Better Approach to Structural Equation Modeling ? , 2012 .

[43]  D. Singer.  A fuzzy set approach to fault tree and reliability analysis , 1990 .

[44]  Renuka Nagpal,et al.  FAHP Approach to Rank Educational Websites on Usability , 2015 .

[45]  Haifeng Li,et al.  Brain oscillations mechanism for cognitive control process , 2015 .

[46]  J. Buckley,et al.  Fuzzy hierarchical analysis , 1999, FUZZ-IEEE'99. 1999 IEEE International Fuzzy Systems. Conference Proceedings (Cat. No.99CH36315).

[47]  Álvaro Rocha,et al.  Framework for a Global Quality Evaluation of a Website , 2012, Online Inf. Rev..

[48]  Da Ruan,et al.  A fuzzy multi-criteria decision approach for software development strategy selection , 2004, Int. J. Gen. Syst..

[49]  Mike Thelwall,et al.  A modeling approach to uncover hyperlink patterns: the case of Canadian universities , 2005, Inf. Process. Manag..

[50]  Judy Jeng,et al.  Usability Assessment of Academic Digital Libraries: Effectiveness, Efficiency, Satisfaction, and Learnability , 2005 .

[51]  Temel Öncan,et al.  Determining key capabilities in technology management using fuzzy analytic hierarchy process: A case study of Turkey , 2006, Inf. Sci..

[52]  Renuka Nagpal,et al.  Rank University Websites Using Fuzzy AHP and Fuzzy TOPSIS Approach on Usability , 2015 .

[53]  Shirley Ann Becker,et al.  A Global Perspective on Web Site Usability , 2001, IEEE Softw..

[54]  Marko Sarstedt,et al.  PLS-SEM: Indeed a Silver Bullet , 2011 .

[55]  D. Chang.  Applications of the extent analysis method on fuzzy AHP , 1996 .

[56]  Saadettin Erhan Kesen,et al.  A fuzzy AHP approach to personnel selection problem , 2009, Appl. Soft Comput..

[57]  Nadjla Hariri,et al.  Determining evaluation criteria for digital libraries' user interface: a review , 2011, Electron. Libr..

[58]  Henry C. W. Lau,et al.  A fuzzy analytic hierarchy process approach in modular product design , 2001, Expert Syst. J. Knowl. Eng..

[59]  Rob Kling,et al.  Digital library design for organizational usability , 1994, SIGO.

[60]  Thomas L. Saaty,et al.  How to Make a Decision: The Analytic Hierarchy Process , 1990 .

[61]  Mahmud Mohd. Jusan,et al.  Priority of structural housing attribute preferences: identifying customer perception , 2015 .

[62]  Htw Chur.  BibEval – A framework for usability evaluations of online library services , 2014 .

[63]  Ludmil Mikhailov,et al.  Evaluation of services using a fuzzy analytic hierarchy process , 2004, Appl. Soft Comput..

[64]  P. Dhanapal Durai Dominic,et al.  Performance measure of airline websites using analytical hierarchy process & fuzzy analytical hierarchy process , 2014, 2014 IEEE International Conference on Control System, Computing and Engineering (ICCSCE 2014).

[65]  Mohammad Reza Sadeghi,et al.  Identifying and prioritizing of effective constructs in readiness of knowledge management implementation by using fuzzy analytic hierarchy process (AHP) , 2013 .

[66]  Yakup Akgül.  Quality evaluation of E-government websites of Turkey , 2016, 2016 11th Iberian Conference on Information Systems and Technologies (CISTI).

[67]  Cheng-Ru Wu,et al.  Integrating fuzzy theory and hierarchy concepts to evaluate software quality , 2008, Software Quality Journal.

[68]  Osman Taylan,et al.  Construction projects selection and risk assessment by fuzzy AHP and fuzzy TOPSIS methodologies , 2014, Appl. Soft Comput..

[69]  Maryellen Allen,et al.  A Survey of the Usability of Digital Reference Services on Academic Health Science Library Web Sites , 2006 .

[70]  Da Ruan,et al.  Fuzzy group decision making for selection among computer integrated manufacturing systems , 2003, Comput. Ind..

[71]  J. Kalra,et al.  Evaluation indicators of library websites of selected research institutions in India , 2011 .

[72]  Mohammad Ali Soukhakian,et al.  Using Analytic Network Process in a Group Decision-Making for Supplier Selection , 2012, Informatica.

[73]  Ying-Feng Kuo,et al.  Using the Fuzzy Synthetic Decision Approach to Assess the Performance of University Teachers in Taiwan , 2002 .

[74]  Nosheen Fatima Warraich,et al.  Usability evaluation of an academic library website: A case of the University of the Punjab , 2016 .