Tools for the evaluation of web accessibility: A systematic literature mapping

In recent years, different tools have been proposed to automate the evaluation of the web contents accessibility criteria proposed by the World Wide Web Consortium (W3C). These tools can verify that a website complies with web accessibility standards such as WCAG, but the results of the evaluation may depend on the tool used. This study identifies and characterizes web accessibility assessment tools through a systematic literature mapping. A total of 50 articles were analyzed. We report the accessibility criteria evaluated by each tool as well as the main challenges related to the evaluations.

[1]  A Juhana,et al.  An Analysis of Website Accessibility in Higher Education in Indonesia Based on WCAG 2.0 Guidelines , 2018 .

[2]  Tuan-Dat Trinh,et al.  Checking and Correcting the Source Code of Web Pages for Accessibility , 2012, 2012 IEEE RIVF International Conference on Computing & Communication Technologies, Research, Innovation, and Vision for the Future.

[3]  Juan Manuel Fernández,et al.  A New Approach to the Automatic Web Accessibility , 2008, ICEIS.

[4]  H. D. Rombach,et al.  The Goal Question Metric Approach , 1994 .

[5]  M. Bhuiyan,et al.  Accessibility analysis and evaluation of Bangladesh government websites , 2012, 2012 International Conference on Informatics, Electronics & Vision (ICIEV).

[6]  Paula Gomes Mian,et al.  Systematic Review in Software Engineering , 2005 .

[7]  Patricia Acosta-Vargas,et al.  Challenges to Assess Accessibility in Higher Education Websites: A Comparative Study of Latin America Universities , 2018, IEEE Access.

[8]  Carlos Delgado Kloos,et al.  Web Accessibility Evaluation Tools: A Survey and Some Improvements , 2006, Electron. Notes Theor. Comput. Sci..

[9]  Reiner R. Dumke,et al.  A Test Procedure for Checking the WCAG 2.0 Guidelines , 2016, HCI.

[10]  Wan Abdul Rahim Wan Mohd Isa,et al.  Accessibility evaluation using Web Content Accessibility Guidelines (WCAG) 2.0 , 2016, 2016 4th International Conference on User Science and Engineering (i-USEr).

[11]  Ridi Ferdiana,et al.  Accessibility evaluation using WCAG 2.0 guidelines webometrics based assessment criteria (case study: Sebelas Maret University) , 2014, 2014 International Conference on ICT For Smart Society (ICISS).

[12]  Pei-Luen Patrick.  Evaluation of web accessibility in China: changes from 2009 to 2013 , 2016 .

[13]  Solomon Adelowo Adepoju,et al.  Usability evaluation of academic websites using automated tools , 2014, 2014 3rd International Conference on User Science and Engineering (i-USEr).

[14]  Silvia Baldiris,et al.  Accessibility evaluation improvement using Case Based Reasoning , 2012, 2012 Frontiers in Education Conference Proceedings.

[15]  Winfred Yaokumah,et al.  Accessibility, Quality and Performance of Government Portals and Ministry Web Sites: A View Using Diagnostic Tools , 2015, 2015 Annual Global Online Conference on Information and Computer Technology (GOCICT).

[16]  Markel Vigo,et al.  Benchmarking web accessibility evaluation tools: measuring the harm of sole reliance on automated tests , 2013, W4A.

[17]  Amjad Mehmood,et al.  Web Accessibility for Disabled: A Case Study of Government Websites in Pakistan , 2012, 2012 10th International Conference on Frontiers of Information Technology.

[18]  Rafael Leonardo Ochoa,et al.  Evaluation of accessibility in Mexican cybermedia , 2018, Universal Access in the Information Society.

[19]  Hend S. Al-Khalifa,et al.  A pilot study for evaluating Arabic websites using automated WCAG 2.0 evaluation tools , 2011, 2011 International Conference on Innovations in Information Technology.

[20]  K. S. Kuppusamy,et al.  Accessibility of Indian universities' homepages: An exploratory study , 2016, J. King Saud Univ. Comput. Inf. Sci..

[21]  Kaan Uyar,et al.  Accessibility of the cyprus Island municipal websites , 2017, 2017 9th International Conference on Computational Intelligence and Communication Networks (CICN).

[22]  Rik Panganiban,et al.  The World Summit on the Information Society , 2004 .

[23]  Patricia Acosta-Vargas,et al.  Framework for Accessibility Evaluation of Hospital Websites , 2018, 2018 International Conference on eDemocracy & eGovernment (ICEDEG).

[24]  William Akotam Agangiba,et al.  Web Accessibility for the Visually Impaired: A Case of Higher Education Institutions' Websites in Ghana , 2017, ICWL.

[25]  Yavuz Inal,et al.  Usability and accessibility evaluation of Libyan government websites , 2017, Universal Access in the Information Society.

[26]  Rita Ismailova,et al.  Universities of the Kyrgyz Republic on the Web: accessibility and usability , 2017, Universal Access in the Information Society.

[27]  Reiner R. Dumke,et al.  Measuring the Accessability Based on Web Content Accessibility Guidelines , 2016, 2016 Joint Conference of the International Workshop on Software Measurement and the International Conference on Software Process and Product Measurement (IWSM-MENSURA).

[28]  Michele Kirchner.  Evaluation, repair, and transformation of Web pages for Web content accessibility. Review of some available tools , 2002, Proceedings. Fourth International Workshop on Web Site Evolution.

[29]  Rosli Mohamad,et al.  Evaluating Accessibility of Malaysian Ministries Websites using WCAG 2.0 and Section 508 Guideline , 2016 .

[30]  Aleksander Bai,et al.  Empowering Agile Project Members with Accessibility Testing Tools: A Case Study , 2019, XP.

[31]  Hiroyuki Seki,et al.  Accessibility verification of WWW documents by an automatic guideline verification tool , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[32]  Sara Paiva,et al.  Accessibility analysis of higher education institution websites of Portugal , 2019, Universal Access in the Information Society.

[33]  I Gusti Bagus Ngurah Eka Darmaputra,et al.  Evaluating the accessibility of provinces' e-government websites in Indonesia , 2017, 2017 5th International Conference on Cyber and IT Service Management (CITSM).

[34]  Pei-Luen Patrick Rau,et al.  Evaluation of web accessibility in China: changes from 2009 to 2013 , 2014, Universal Access in the Information Society.

[35]  Yahya M. Tashtoush,et al.  The Arabian E-government websites accessibility: A case study , 2016, 2016 7th International Conference on Information and Communication Systems (ICICS).

[36]  Aidi Ahmi,et al.  Evaluating accessibility of Malaysian public universities websites using AChecker and WAVE , 2016 .

[37]  Yavuz Inal,et al.  Effect of human development level of countries on the web accessibility and quality in use of their municipality websites , 2020, J. Ambient Intell. Humaniz. Comput..

[38]  Luis Salvador-Ullauri,et al.  Quality evaluation of government websites , 2017, 2017 Fourth International Conference on eDemocracy & eGovernment (ICEDEG).

[39]  Subarna Shakya,et al.  Evaluating the web accessibility of websites of the central government of Nepal , 2007, ICEGOV '07.

[40]  K. S. Kuppusamy,et al.  Web accessibility investigation and identification of major issues of higher education websites with statistical measures: A case study of college websites , 2019, J. King Saud Univ. Comput. Inf. Sci..

[41]  Kamen Spassov,et al.  Evaluating Metropolitan Assembly Web Sites in Ghana: Accessibility, Compatibility and Usability , 2018, Webology.

[42]  Morten Tollefsen,et al.  A practitioner's approach to using WCAG evaluation tools , 2017, 2017 6th International Conference on Information and Communication Technology and Accessibility (ICTA).

[43]  Yavuz Inal,et al.  Accessibility evaluation of top university websites: a comparative study of Kyrgyzstan, Azerbaijan, Kazakhstan and Turkey , 2018, Universal Access in the Information Society.

[44]  Swikruti U. Dongaonkar,et al.  Accessibility Analyzer: Tool for New Adaptations in Government Web Applications to Improve Accessibility , 2017, 2017 International Conference on Computing, Communication, Control and Automation (ICCUBEA).

[45]  Jorge Pereira,et al.  Portuguese Web Accessibility Snapshot - Status of the Portuguese Websites Regarding Accessibility Levels , 2010, ICEIS.

[46]  Bayan Abu Shawar.  Evaluating Web Accessibility of Educational Websites , 2015, iJET.

[47]  Gregg C. Vanderheiden,et al.  Web Content Accessibility Guidelines (WCAG) 2.0 , 2008 .

[48]  Dimitrios Tzovaras,et al.  An analysis of personalized web accessibility , 2014, W4A.

[49]  Dhanapal Durai Dominic,et al.  Website accessibility performance evaluation in Malaysia , 2008, 2008 International Symposium on Information Technology.

[50]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[51]  Rached Zantout,et al.  Toward better web accessibility , 2015, 2015 5th International Conference on Information & Communication Technology and Accessibility (ICTA).

[52]  Muhammad Zuhri Catur Candra,et al.  Automatically: An Automated Refactoring Method and Tool for Improving Web Accessibility , 2018, 2018 5th International Conference on Data and Software Engineering (ICoDSE).

[53]  Yakup AkgÜL,et al.  Web Content Accessibility of Municipal Web Sites in Turkey , 2016 .

[54]  Kai Petersen,et al.  Guidelines for conducting systematic mapping studies in software engineering: An update , 2015, Inf. Softw. Technol..

[55]  S. Luján-Mora,et al.  Accessibility of eGovernment Services in Latin America , 2018, 2018 International Conference on eDemocracy & eGovernment (ICEDEG).

[56]  C. Michael Sperberg-McQueen,et al.  World Wide Web Consortium , 2009, Encyclopedia of Database Systems.

[57]  Sergio Luján Mora,et al.  Introduction to Web Accessibility , 2011 .

[58]  Patricia Acosta-Vargas,et al.  Towards Web Accessibility in Telerehabilitation Platforms , 2018, 2018 IEEE Third Ecuador Technical Chapters Meeting (ETCM).

[59]  Helen Petrie,et al.  Towards a unified definition of web accessibility , 2015, W4A.

[60]  Pinar Onay Durdu,et al.  Evaluation of Accessibility of University Websites: A Case from Turkey , 2017, HCI.

[61]  Ajay Rana,et al.  A Practitioner’s Approach to Assess the WCAG 2.0 Website Accessibility Challenges , 2019, 2019 Amity International Conference on Artificial Intelligence (AICAI).

[62]  S. Gopinath,et al.  Usability and accessibility analysis of selected government websites in Sri Lanka , 2016, 2016 IEEE Region 10 Symposium (TENSYMP).

[63]  Luis Salvador-Ullauri,et al.  Toward a Combined Method for Evaluation of Web Accessibility , 2018, ICITS.