Crowdsourcing Accessibility: Human-Powered Access Technologies

People with disabilities have always engaged the people around them inorder to circumvent inaccessible situations, allowing them to live moreindependently and get things done in their everyday lives. Increasingconnectivity is allowing this approach to be extended to wherever andwhenever it is needed. Technology can leverage this human work forceto accomplish tasks beyond the capabilities of computers, increasinghow accessible the world is for people with disabilities. This articleoutlines the growth of online human support, outlines a number ofprojects in this space, and presents a set of challenges and opportunitiesfor this work going forward.

[1]  Andrew Gorman,et al.  Increasing Workplace Independence for People with Cognitive Disabilities by Leveraging Distributed Cognition among Caregivers and Clients , 2004, Computer Supported Cooperative Work (CSCW).

[2]  Amy Hurst,et al.  Designing an accessible clothing tag system for people with vision impairments , 2013, ASSETS.

[3]  Luís Carriço,et al.  Friendsourcing the unmet needs of people with dementia , 2014, W4A.

[4]  Amy Hurst,et al.  Preliminary Investigation of the Limitations Fashion Presents to Those with Vision Impairments , 2013 .

[5]  Ben Shneiderman,et al.  The limits of speech recognition , 2000, CACM.

[6]  Hironobu Takagi,et al.  Exploratory Analysis of Collaborative Web Accessibility Improvement , 2010, TACC.

[7]  Constantine Stephanidis,et al.  Universal access in the information society , 1999, HCI.

[8]  Panagiotis G. Ipeirotis.  Demographics of Mechanical Turk , 2010 .

[9]  Benjamin B. Bederson,et al.  Human computation: a survey and taxonomy of a growing field , 2011, CHI.

[10]  Michael S. Bernstein,et al.  We Are Dynamo: Overcoming Stalling and Friction in Collective Action for Crowd Workers , 2015, CHI.

[11]  James Davis,et al.  Evaluating and improving the usability of Mechanical Turk for low-income workers in India , 2010, ACM DEV '10.

[12]  Michael S. Bernstein,et al.  Crowds in two seconds: enabling realtime crowd-powered interfaces , 2011, UIST.

[13]  Ephraim P. Glinert,et al.  Improving GUI accessibility for people with low vision , 1995, CHI '95.

[14]  Eric Horvitz,et al.  Crowdsourcing the acquisition of natural language corpora: Methods and observations , 2012, 2012 IEEE Spoken Language Technology Workshop (SLT).

[15]  Jeffrey P. Bigham,et al.  Evaluating existing audio CAPTCHAs and an interface optimized for non-visual use , 2009, CHI.

[16]  Meredith Ringel Morris,et al.  Investigating the appropriateness of social network question asking as a resource for blind users , 2013, CSCW.

[17]  John J. Horton.  Online Labor Markets , 2010 .

[18]  Michael S. Bernstein,et al.  The future of crowd work , 2013, CSCW.

[19]  Gregg Vanderheiden,et al.  Fundamental principles and priority setting for universal usability , 2000, CUU '00.

[20]  Vicki L. Hanson,et al.  General writing guidelines for technology and people with disabilities , 2008, ASAC.

[21]  Erich Elsen,et al.  Deep Speech: Scaling up end-to-end speech recognition , 2014, ArXiv.

[22]  Hironobu Takagi,et al.  Social accessibility: achieving accessibility through collaborative metadata authoring , 2008, Assets '08.

[23]  Matt Post,et al.  The Language Demographics of Amazon Mechanical Turk , 2014, TACL.

[24]  Elena M. Umland,et al.  A Common Language for Interprofessional Education: The World Health Organization's International Classification of Functioning, Disability and Health (ICF) , 2013 .

[25]  Panagiotis G. Ipeirotis,et al.  Quality management on Amazon Mechanical Turk , 2010, HCOMP '10.

[26]  Lorrie Faith Cranor,et al.  Are your participants gaming the system?: screening mechanical turk workers , 2010, CHI.

[27]  Roberta L. Klatzky,et al.  Navigation System for the Blind: Auditory Display Modes and Guidance , 1998, Presence.

[28]  Oded Nov,et al.  Technology-Mediated Citizen Science Participation: A Motivational Model , 2011, ICWSM.

[29]  N. Watson,et al.  The social model of disability: An outdated ideology? , 2001 .

[30]  Amy Hurst,et al.  "Pray before you step out": describing personal and situational blind navigation behaviors , 2013, ASSETS.

[31]  Michael S. Bernstein.  Crowd-Powered Systems , 2012, KI - Künstliche Intelligenz.

[32]  Meredith Ringel Morris,et al.  Gauging Receptiveness to Social Microvolunteering , 2015, CHI.

[33]  Jeffrey P. Bigham,et al.  VizWiz: nearly real-time answers to visual questions , 2010, W4A.

[34]  Aaron Allen,et al.  What Frustrates Screen Reader Users on the Web: A Study of 100 Blind Users , 2007, Int. J. Hum. Comput. Interact..

[35]  Richard E. Ladner,et al.  Automated tactile graphics translation: in the field , 2007, Assets '07.

[36]  Aaron D. Shaw,et al.  Social desirability bias and self-reports of motivation: a study of amazon mechanical turk in the US and India , 2012, CHI.

[37]  Jeffrey P. Bigham.  Increasing web accessibility by automatically judging alternative text quality , 2007, IUI '07.

[38]  Richard E. Ladner,et al.  Enhancing independence and safety for blind and deaf-blind public transit riders , 2011, CHI.

[39]  Xin Li,et al.  Using Social Psychology to Motivate Contributions to Online Communities , 2005, J. Comput. Mediat. Commun..

[40]  Walter S. Lasecki,et al.  Real-time captioning by groups of non-experts , 2012, UIST.

[41]  Haofeng Zhou,et al.  Crowdsourcing Transcription Beyond Mechanical Turk , 2013, AAAI 2013.

[42]  Aniket Kittur,et al.  An Assessment of Intrinsic and Extrinsic Motivation on Task Performance in Crowdsourcing Markets , 2011, ICWSM.

[43]  C. Barnes,et al.  Independent Futures: Creating user-led disability services in a disabling society , 2006 .

[44]  Meredith Ringel Morris,et al.  Accessible Crowdwork?: Understanding the Value in and Challenge of Microtask Employment for People with Disabilities , 2015, CSCW.

[45]  M. Six Silberman,et al.  Turkopticon: interrupting worker invisibility in amazon mechanical turk , 2013, CHI.

[46]  Martin Wattenberg,et al.  Proceedings of the 40th Hawaii International Conference on System Sciences- 2007 Talk Before You Type: Coordination in Wikipedia , 2022 .

[47]  Hironobu Takagi,et al.  Collaborative web accessibility improvement: challenges and possibilities , 2009, Assets '09.

[48]  Jonathan Lazar,et al.  Accessibility of U.S. federal government home pages: Section 508 compliance and site accessibility statements , 2011, Gov. Inf. Q..

[49]  W. Gerrey,et al.  Remote reading systems for the blind: A potential application of virtual presence , 1992, 1992 14th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[50]  Peter Gregor,et al.  User-Sensitive Inclusive Design , 2011, Universal Access in the Information Society.

[51]  M F Story,et al.  Maximizing usability: the principles of universal design. , 1998, Assistive technology : the official journal of RESNA.

[52]  Jeffrey P. Bigham.  Making the web easier to see with opportunistic accessibility improvement , 2014, UIST.

[53]  Anne Pemberton.  Marcia J. Scherer: Living in the State of Stuck: How Technology Impacts the Lives of Persons with Disabilities , 1997 .

[54]  Jonathan Lazar,et al.  Developing usable CAPTCHAs for blind users , 2007, Assets '07.

[55]  Scott R. Klemmer,et al.  Shepherding the crowd yields better work , 2012, CSCW.

[56]  C. Lintott,et al.  Galaxy Zoo: Exploring the Motivations of Citizen Science Volunteers. , 2009, 0909.2925.

[57]  Michael S. Bernstein,et al.  Human-Computer Interaction and Collective Intelligence , 2014 .

[58]  Richard E. Ladner,et al.  Freedom to roam: a study of mobile device adoption and accessibility for people with visual and motor disabilities , 2009, Assets '09.

[59]  Gregory D. Abowd,et al.  Designing a social network to support the independence of young adults with autism , 2012, CSCW.

[60]  Aaron Steinfeld,et al.  Helping visually impaired users properly aim a camera , 2012, ASSETS '12.

[61]  Michael S. Bernstein,et al.  Soylent: a word processor with a crowd inside , 2010, UIST.

[62]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[63]  Masayuki Kobayashi,et al.  Support Technique for Real-Time Captionist to Use Speech Recognition Software , 2008, ICCHP.

[64]  Jeffrey P. Bigham,et al.  Supporting blind photography , 2011, ASSETS.

[65]  Walter S. Lasecki,et al.  Helping students keep up with real-time captions by pausing and highlighting , 2014, W4A.

[66]  Walter S. Lasecki,et al.  Architecting Real-Time Crowd-Powered Systems , 2014, Hum. Comput..

[67]  Daniel Gildea,et al.  Text Alignment for Real-Time Crowd Captioning , 2013, NAACL.

[68]  Tara Matthews,et al.  Scribe4Me: Evaluating a Mobile Sound Transcription Tool for the Deaf , 2006, UbiComp.

[69]  Gillian R. Hayes,et al.  Disability studies as a source of critical inquiry for the field of assistive technology , 2010, ASSETS '10.

[70]  Jan C. Galvin,et al.  Evaluating, Selecting, and Using Appropriate Assistive Technology , 2004 .

[71]  Richard E. Ladner,et al.  Asl-stem forum: enabling sign language to grow through online collaboration , 2010, CHI.

[72]  Walter S. Lasecki,et al.  Warping time for more effective real-time crowdsourcing , 2013, CHI.

[73]  Bill Tomlinson,et al.  Who are the crowdworkers?: shifting demographics in mechanical turk , 2010, CHI Extended Abstracts.

[74]  H. Stonnington.  Nothing About Us Without Us: Disability Oppression and Empowerment , 1998 .

[75]  Mark Pilgrim.  Greasemonkey hacks - tips and tools for remixing the web with Firefox , 2005 .

[76]  Jacki O'Neill,et al.  Being a turker , 2014, CSCW.

[77]  Michael S. Bernstein,et al.  Apparition: Crowdsourced User Interfaces that Come to Life as You Sketch Them , 2015, CHI.

[78]  John Langford,et al.  Telling humans and computers apart automatically , 2004, CACM.

[79]  Richard E. Ladner,et al.  Automating tactile graphics translation , 2005, Assets '05.

[80]  Jonathan Lazar,et al.  Improving web accessibility: a study of webmaster perceptions , 2004, Comput. Hum. Behav..

[81]  Danai Koutra,et al.  Glance: rapidly coding behavioral video with the crowd , 2014, UIST.

[82]  Harry Levitt,et al.  REAL-TIME SPEECH-TO-TEXT SERVICES , 1999 .

[83]  Krzysztof Z. Gajos,et al.  Ability-Based Design: Concept, Principles and Examples , 2011, TACC.

[84]  Erin Brady,et al.  Visual challenges in the everyday lives of blind people , 2013, CHI.

[85]  Dhruv Jain,et al.  Path-guided indoor navigation for the visually impaired using minimal building retrofitting , 2014, ASSETS.

[86]  Manuel Blum,et al.  Improving accessibility of the web with a computer game , 2006, CHI.

[87]  Jeffrey P. Bigham,et al.  Crowdsourcing subjective fashion advice using VizWiz: challenges and opportunities , 2012, ASSETS '12.

[88]  Richard E. Ladner,et al.  Accessmonkey: a collaborative scripting framework for web users and developers , 2007, W4A '07.

[89]  Bambang Parmanto,et al.  Accessibility of Internet websites through time , 2003, ASSETS.

[90]  Rob Miller,et al.  Real-time crowd control of existing interfaces , 2011, UIST.

[91]  T. V. Raman,et al.  AxsJAX: a talking translation bot using google IM: bringing web-2.0 applications to life , 2008, W4A '08.

[92]  Richard E. Ladner,et al.  The design of human-powered access technology , 2011, ASSETS.

[93]  M. Marschark,et al.  Benefits of sign language interpreting and text alternatives for deaf students' classroom learning. , 2006, Journal of deaf studies and deaf education.

[94]  Lydia B. Chilton,et al.  Task search in a human computation market , 2010, HCOMP '10.

[95]  Richard E. Ladner,et al.  ClassInFocus: enabling improved visual attention strategies for deaf and hard of hearing students , 2009, Assets '09.

[96]  Dana Chandler,et al.  Breaking Monotony with Meaning: Motivation in Crowdsourcing Markets , 2012, ArXiv.

[97]  Lydia B. Chilton,et al.  TurKit: human computation algorithms on mechanical turk , 2010, UIST.

[98]  Harry Levitt,et al.  Real-Time Speech-to-Text Services. [A Report of the] National Task Force on Quality of Services in the Postsecondary Education of Deaf and Hard of Hearing Students. , 1999 .