Web Accessibility Evaluation Tools: A Survey and Some Improvements

Web Content Accessibility Guidelines (WCAG) from W3C consist of a set of 65 checkpoints or specifications that Web pages should accomplish in order to be accessible to people with disabilities or using alternative browsers. Many of these 65 checkpoints can only be checked by a human operator, thus implying a very high cost for full evaluation. However, some checkpoints can be automatically evaluated, thus spotting accessibility barriers in a very effective manner. Well known tools like Bobby, Tawdis or WebXACT evaluate Web accessibility by using a mixture of automated, manual and semiautomated evaluations. However, the automation degree of these Web evaluations is not the same for each of these tools. Since WCAG are not written in a formalized manner, these evaluators may have different ''interpretations'' of what these rules mean. As a result, different evaluation results might be obtained for a single page depending on which evaluation tool is being used. Taking into account that a fully automated Web accessibility evaluation is not feasible because of the nature of WCAG, this paper evaluates the WCAG automation coverage of some well known Web accessibility evaluation tools spotting their weaknesses and differences. We also provide a formalized specification for those checkpoints where these differences have been detected, thus challenging nowaday's tools for more automated coverage.