A Cost-benefit Evaluation of Accessibility Testing in Agile Software Development

Accessibility testing in software development is testing the software to ensure that it is usable by as many people as possible, independent of their capabilities. Few guidelines exist on how to include accessibility testing in an agile process, and how to select testing methods from a cost-benefit point of view. The end result is that many development teams do not include accessibility testing, since they do not know how to prioritize the different testing methods within a tight budget. In this paper, we present an evaluation of four accessibility testing methods that fits in an agile software development process. We discuss the cost of each method with regards to resources and knowledge requirements, and based on a cost-benefit analysis, we present the optimal combinations of these methods in terms of cost and issues discovered. Finally, we describe how accessibility testing methods can be incorporated into an agile process by using the agile accessibility spiral. Keywords—Accessibility testing; Agile software development; Cost-benefit analysis; Usability.

[1]  Trenton Schulz,et al.  Creating Personas with Disabilities , 2012, ICCHP.

[2]  Marilyn Tremaine,et al.  Cost/benefit analysis for incorporating human factors in the software lifecycle , 1988, CACM.

[3]  José Antonio Pow-Sang,et al.  A Systematic Mapping Review of Usability Evaluation Methods for Software Development Process , 2016 .

[4]  Maria Cecília Calani Baranauskas,et al.  An Agile Process Model for Inclusive Software Development , 2009, ICEIS.

[5]  Helen Petrie,et al.  The Evaluation of Accessibility, Usability, and User Experience , 2009, The Universal Access Handbook.

[6]  Randolph G. Bias,et al.  Cost-Justifying Usability: An Update for the Internet Age , 2005 .

[7]  P. John Clarkson,et al.  Simulation in user-centred design: helping designers to empathise with atypical users , 2012 .

[8]  長瀬 修.  わかりやすい障害者の権利条約 : 知的障害のある人の権利のために = Convention on the rights of persons with disabilities , 2009 .

[9]  D. Scott McCrickard,et al.  Towards Extreme(ly) Usable Software: Exploring Tensions Between Usability and Agile Software Development , 2007, Agile 2007 (AGILE 2007).

[10]  Pj Clarkson,et al.  Inclusive design toolkit , 2007 .

[11]  Torgeir Dingsøyr,et al.  A decade of agile methodologies: Towards explaining agile software development , 2012, J. Syst. Softw..

[12]  Gregg C. Vanderheiden,et al.  Accessible design and testing in the application development process: considerations for an integrated approach , 2008, Universal Access in the Information Society.

[13]  Gregg C. Vanderheiden,et al.  Web content accessibility guidelines 1.0 , 2001, INTR.

[14]  A. Boardman,et al.  Cost-Benefit Analysis: Concepts and Practice , 1996 .

[15]  Lin Chou Cheng,et al.  A Reference to Usability Inspection Methods , 2015 .

[16]  Lourdes Moreno,et al.  Toward an Integration of Web Accessibility into Testing Processes , 2013, DSAI.

[17]  Error Cost Escalation Through the Project Life Cycle , 2010 .

[18]  Leaf Van Boven,et al.  Stumbling in Their Shoes , 2015 .

[19]  Joseph F. Dumas,et al.  A Practical Guide to Usability Testing , 1993 .

[20]  Paul A. Cairns,et al.  A systematic literature review for agile development processes and user centred design integration , 2014, EASE '14.

[21]  Kristin Skeide Fuglerud,et al.  An evaluation of web-based voting usability and accessibility , 2011, Universal Access in the Information Society.

[22]  Des Greer,et al.  The Maturation of Agile Software Development Principles and Practice: Observations on Successive Industrial Studies in 2010 and 2012 , 2013, 2013 20th IEEE International Conference and Workshops on Engineering of Computer Based Systems (ECBS).

[23]  André Pimenta Freire,et al.  Guidelines are only half of the story: accessibility problems encountered by blind users on the web , 2012, CHI.