A Multimodal End-2-End Approach to Accessible Computing

Research in intelligent interactive systems can offer valuable assistance to elderly and disabled populations by helping them to achieve greater levels of engagement with the world. Many users find it difficult to use existing interaction devices, either for physical or age-related impairments. However, research on intelligent voice recognition, adaptable pointing, browsing and navigation, and affect and gesture recognition can hugely benefit such users. Additionally, systems and services developed for elderly or disabled people often find useful applications for their able-bodied counterparts. A Multimodal End-2-End Approach to Accessible Computing illustrates the state-of-the-art of technology and presents a vision for accessibility in the near future. It considers challenges faced by accessibility practitioners at research institutes, industries and legislative institutions throughout the world, and explores the different phases of delivering accessible products and services through design, development, deployment and maintenance. A collection of eminent researchers cover topics on developing and standardizing user models for inclusive design, adaptable multimodal system development for digital TV and ubiquitous devices. With a foreword from the BBCs Head of Technology and organiser of the Switchover Help Scheme, and an End Note from the chairman of the ITU-Ts Focus Group on Audiovisual Media Accessibility (presenting a vision for accessible computing), this book will be an invaluable resource for researchers and practitioners.

[1]  T. Koda,et al.  Agents with faces: the effect of personification , 1996, Proceedings 5th IEEE International Workshop on Robot and Human Communication. RO-MAN'96 TSUKUBA.

[2]  Gloria Mark Designing Believable Interaction by Applying Social Conventions , 1999, Appl. Artif. Intell..

[3]  Akikazu Takeuchi,et al.  Situated facial displays: towards social interaction , 1995, CHI '95.

[4]  R. Klein,et al.  Prevalence of hearing loss in older adults in Beaver Dam, Wisconsin. The Epidemiology of Hearing Loss Study. , 1998, American journal of epidemiology.

[5]  Günter Wallner,et al.  HOMIE: an artificial companion for elderly people , 2005, CHI Extended Abstracts.

[6]  Julita Vassileva,et al.  Affective pedagogical agents and user persuasion , 2001, HCI.

[7]  M. Holden,et al.  Virtual Environment Training: A New Tool for Neurorehabilitation , 2002 .

[8]  J. Kulik,et al.  Stress, affiliation, and emotional contagion. , 1997, Journal of personality and social psychology.

[9]  Ian Sommerville,et al.  Designing technology with older people , 2007, Universal Access in the Information Society.

[10]  Jeremy N. Bailenson,et al.  A meta-analysis of the impact of the inclusion and realism of human-like faces on user experiences in interfaces , 2007, CHI.

[11]  K. Teigen Yerkes-Dodson: A Law for all Seasons , 1994 .

[12]  Lee Sproull,et al.  Using a human face in an interface , 1994, CHI '94.

[13]  Jonathan Klein,et al.  This computer responds to user frustration: Theory, design, and results , 2002, Interact. Comput..

[14]  Anke M. Brock,et al.  Usage of multimodal maps for blind people: why and how , 2010, ITS '10.

[15]  C. Kidd Engagement in Long-Term Human-Robot Interaction , 2007 .

[16]  Mark A. Neerincx,et al.  Incorporating guidelines for health assistance into a socially intelligent robot , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[17]  A. Rigaud,et al.  Cognitive stimulation intervention for elders with mild cognitive impairment compared with normal aged subjects: preliminary results , 2007, Aging clinical and experimental research.

[18]  C. Perry Clinical Features , 2004, Bristol medico-chirurgical journal.

[19]  Li Gong Is happy better than sad even if they are both non-adaptive? Effects of emotional expressions of talking-head interface agents , 2007, Int. J. Hum. Comput. Stud..

[20]  Sally Hong,et al.  The Social Life of Avatars: Presence and Interaction in Shared Virtual Environments , 2002, J. Educ. Technol. Soc..

[21]  D. Feil-Seifer,et al.  Defining socially assistive robotics , 2005, 9th International Conference on Rehabilitation Robotics, 2005. ICORR 2005..

[22]  Teddy McCalley,et al.  The Pet Plant: Developing an Inanimate Emotionally Interactive Tool for the Elderly , 2007, PERSUASIVE.

[23]  J. Holmes,et al.  Aging differently: Physical limitations among adults aged 50 years and over: United States, 2001-2007. , 2009, NCHS data brief.

[24]  I. V. Ramakrishnan,et al.  Bridging the Web Accessibility Divide , 2009, WWV.

[25]  Jörn Ostermann,et al.  User evaluation: Synthetic talking faces for interactive services , 1999, The Visual Computer.

[26]  Junji Maeda,et al.  Accessibility designer: visualizing usability for the blind , 2004, Assets '04.

[27]  Sharon L. Oviatt,et al.  Multimodal interfaces for dynamic interactive maps , 1996, CHI.

[28]  Takayuki Kanda,et al.  Interactive Robots as Social Partners and Peer Tutors for Children: A Field Trial , 2004, Hum. Comput. Interact..

[29]  Lee Sproull,et al.  Cooperating with life-like interface agents , 1999 .

[30]  C. Miller,et al.  Results from a Field Study : The Need for an Emotional Relationship between the Elderly and their Assistive Technologies , 2005 .

[31]  Takanori Shibata,et al.  Statistical analysis and comparison of questionnaire results of subjective evaluations of seal robot in Japan and UK , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[32]  Shaun W. Lawson,et al.  The Impact of Owner Age on Companionship with Virtual Pets , 2007, ECIS.

[33]  James C. Lester,et al.  Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments , 2000 .

[34]  Paul Blenkhorn,et al.  Java-Powered Braille Slate Talker , 2004, ICCHP.

[35]  Stefan Kopp,et al.  Know Your Users! Empirical Results for Tailoring an Agent's Nonverbal Behavior to Different User Groups , 2010, IVA.

[36]  C. Perkins,et al.  Real World Map Reading Strategies , 2003 .

[37]  Anind K. Dey,et al.  Lifelogging memory appliance for people with episodic memory impairment , 2008, UbiComp.

[38]  Nils Ulltveit-Moe,et al.  Interpreting Results from Large Scale Automatic Evaluation of Web Accessibility , 2006, ICCHP.

[39]  David Melzer,et al.  Physical and cognitive function , 2003 .

[40]  Yvonne de Kort,et al.  Virtual Fitness: Stimulating Exercise Behavior through Media Technology , 2006, PRESENCE: Teleoperators and Virtual Environments.

[41]  Xiaoli Li,et al.  Eliminating noisy information in Web pages for data mining , 2003, KDD '03.

[42]  Nicole C. Krämer,et al.  “Theory of companions” What can theoretical models contribute to applications and understanding of human-robot interaction? , 2011 .

[43]  Maja J. Mataric,et al.  Encouraging physical therapy compliance with a hands-Off mobile robot , 2006, HRI '06.

[44]  Kim Marriott,et al.  GraVVITAS: Generic Multi-touch Presentation of Accessible Graphics , 2011, INTERACT.

[45]  J G Clark,et al.  Uses and abuses of hearing loss classification. , 1981, ASHA.

[46]  B. Salzman Gait and balance disorders in older adults. , 2010, American family physician.

[47]  Kerstin Fischer,et al.  Social interaction with robots and agents: Where do we stand, where do we go? , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[48]  I. V. Ramakrishnan,et al.  Combating information overload in non-visual web access using context , 2007, IUI '07.

[49]  J. Kleim Neural plasticity and neurorehabilitation: teaching the new brain old tricks. , 2011, Journal of communication disorders.

[50]  James C. Lester,et al.  Deictic and emotive communication in animated pedagogical agents , 2001 .

[51]  Brian Scassellati,et al.  How Social Robots Will Help Us to Diagnose, Treat, and Understand Autism , 2005, ISRR.

[52]  P. Raina,et al.  Influence of Companion Animals on the Physical and Psychological Health of Older People: An Analysis of a One‐Year Longitudinal Study , 1999, Journal of the American Geriatrics Society.

[53]  Jacob O. Wobbrock,et al.  Slide rule: making mobile touch screens accessible to blind people using multi-touch interaction techniques , 2008, Assets '08.

[54]  Richard E. Ladner,et al.  Access overlays: improving non-visual access to large touch screens for blind users , 2011, UIST.

[55]  Richard E. Ladner,et al.  Hunting for headings: sighted labeling vs. automatic classification of headings , 2008, Assets '08.

[56]  M. Rieger,et al.  Enhancing spatial learning and mobility training of visually impaired people—a technical paper on the Internet‐based tactile and audio‐tactile mapping , 2003 .

[57]  B. Dudgeon,et al.  Virtual cues and functional mobility of people with Parkinson's disease: a single-subject pilot study. , 2007, Journal of rehabilitation research and development.

[58]  Enrico Pontelli,et al.  Towards the creation of accessibility agents for non-visual navigation of the web , 2002 .

[59]  John A. Gardner,et al.  Scientific diagrams made easy with IVEO , 2006 .

[60]  Illah R. Nourbakhsh,et al.  A survey of socially interactive robots , 2003, Robotics Auton. Syst..

[61]  Dennis Reidsma,et al.  Towards a Reactive Virtual Trainer , 2006, IVA.

[62]  R. Dan Jacobson Navigating maps with little or no sight: An audio-tactile approach , 1998 .

[63]  K. Majamaa,et al.  Self-reported hearing problems among older adults: prevalence and comparison to measured hearing impairment. , 2011, Journal of the American Academy of Audiology.

[64]  Ben J. A. Krse,et al.  Studying the acceptance of a robotic agent by elderly users , 2006 .

[65]  Mel Slater,et al.  Building Expression into Virtual Characters , 2006, Eurographics.

[66]  Stephanie Rosenthal,et al.  Designing robots for long-term social interaction , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[67]  T. Salthouse Selective review of cognitive aging , 2010, Journal of the International Neuropsychological Society.

[68]  K. Ball,et al.  Long-term effects of cognitive training on everyday functional outcomes in older adults. , 2006, JAMA.

[69]  Jan O. Borchers,et al.  Tactile motion instructions for physical activities , 2009, CHI.

[70]  Mitsuru Ishizuka,et al.  Using human physiology to evaluate subtle expressivity of a virtual quizmaster in a mathematical game , 2005, Int. J. Hum. Comput. Stud..

[71]  C. Barnes,et al.  Neural plasticity in the ageing brain , 2006, Nature Reviews Neuroscience.

[72]  G. Bente,et al.  Personalizing e-Learning. The Social Effects of Pedagogical Agents , 2010 .

[73]  Izak Benbasat,et al.  Online Consumer Trust and Live Help Interfaces: The Effects of Text-to-Speech Voice and Three-Dimensional Avatars , 2005, Int. J. Hum. Comput. Interact..

[74]  Maria del Puy Carretero,et al.  Elderly Users in Ambient Intelligence: Does an Avatar Improve the Interaction? , 2006, Universal Access in Ambient Intelligence Environments.

[75]  Byron Reeves,et al.  The effects of animated characters on anxiety, task performance, and evaluations of user interfaces , 2000, CHI.

[76]  Nicole C. Krämer,et al.  How Our Personality Shapes Our Interactions with Virtual Characters - Implications for Research and Development , 2010, IVA.

[77]  Panos Markopoulos,et al.  GentleGuide: An Exploration of Haptic Output for Indoors Pedestrian Guidance , 2003, Mobile HCI.

[78]  Birgit Graf,et al.  Robotic home assistant Care-O-bot: past-present-future , 2002, Proceedings. 11th IEEE International Workshop on Robot and Human Interactive Communication.

[79]  Ashok Samal,et al.  A system for recognizing a large class of engineering drawings , 1995, Proceedings of 3rd International Conference on Document Analysis and Recognition.

[80]  T. Kamarck,et al.  Social support reduces cardiovascular reactivity to psychological challenge: a laboratory model. , 1990, Psychosomatic medicine.

[81]  Andrew Jensen,et al.  Using Embodied Pedagogical Agents and Direct Instruction to Improve Learning Outcomes for Young Children with Learning Disabilities , 2012 .

[82]  P. Greenwood Functional plasticity in cognitive aging: review and hypothesis. , 2007, Neuropsychology.

[83]  Rubén San-Segundo-Hernández,et al.  Advanced speech communication system for deaf people , 2010, INTERSPEECH.

[84]  Masaharu Yoshioka IR Interface for Contrasting Multiple News Sites , 2008, AIRS.

[85]  James C. Lester,et al.  The Case for Social Agency in Computer-Based Teaching: Do Students Learn More Deeply When They Interact With Animated Pedagogical Agents? , 2001 .

[86]  R. Zajonc SOCIAL FACILITATION. , 1965, Science.

[87]  Cynthia Breazeal,et al.  TIKL: Development of a Wearable Vibrotactile Feedback Suit for Improved Human Motor Learning , 2007, IEEE Transactions on Robotics.

[88]  Ben J. A. Kröse,et al.  Assessing Acceptance of Assistive Social Agent Technology by Older Adults: the Almere Model , 2010, Int. J. Soc. Robotics.

[89]  James C. Lester,et al.  The persona effect: affective impact of animated pedagogical agents , 1997, CHI.

[90]  S. Ferris,et al.  Age‐associated memory impairment: Proposed diagnostic criteria and measures of clinical change — report of a national institute of mental health work group , 1986 .

[91]  Mel Slater,et al.  The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment , 2003, CHI '03.

[92]  Yeliz Yesilada,et al.  Testability and validity of WCAG 2.0: the expertise effect , 2010, ASSETS '10.

[93]  Sascha Fagel Emotional McGurk Effect , 2006 .

[94]  Christoph Schlieder,et al.  TeDUB: A System for Presenting and Exploring Technical Drawings for Blind People , 2002, ICCHP.

[95]  I. V. Ramakrishnan,et al.  OntoMiner: Bootstrapping and Populating Ontologies from Domain-Specific Web Sites , 2003, IEEE Intell. Syst..

[96]  M. Lindstrom,et al.  Social capital, trust in the health-care system and self-rated health: the role of access to health care in a population-based study. , 2007, Social science & medicine.

[97]  Kerstin Dautenhahn,et al.  Applying Mobile Robot Technology to the Rehabilitation of Autistic Children , 1999 .

[98]  Martha E. Pollack,et al.  Autominder: an intelligent cognitive orthotic system for people with memory impairment , 2003, Robotics Auton. Syst..

[99]  Andreas Stainer-Hochgatterer,et al.  Avatars in Assistive Homes for the Elderly , 2008, USAB.

[100]  R. Weale The Eye within the Framework of Human Senescence: Biological Decline and Morbidity , 1998, Ophthalmic Research.

[101]  Erwin Schoonderwaldt,et al.  Good vibrations: Guiding body movements with vibrotactile feedback , 2009 .

[102]  M. Mather The Role of Cognitive Control in Older Adults' Emotional Well-Being , 2008 .

[103]  J. Vormbrock,et al.  Cardiovascular effects of human-pet dog interactions , 1988, Journal of Behavioral Medicine.

[104]  J. Cummings,et al.  Mild cognitive impairment (MCI) represents early-stage Alzheimer's disease. , 2005, Journal of Alzheimer's disease : JAD.

[105]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[106]  Jeffrey Nichols,et al.  Trailblazer: enabling blind users to blaze trails through the web , 2009, IUI.

[107]  Clifford Nass,et al.  The media equation - how people treat computers, television, and new media like real people and places , 1996 .

[108]  Sina Bahram,et al.  CAVIAR: a vibrotactile device for accessible reaching , 2012, IUI '12.

[109]  J. Bóveda,et al.  Prevalence and Correlates of Cognitive Impairment in Adults with Subjective Memory Complaints in Primary Care Centres , 2012, Dementia and Geriatric Cognitive Disorders.

[110]  Jeannine S. Schiller,et al.  Summary health statistics for U.S. adults: National Health Interview Survey, 2002. , 2004, Vital and health statistics. Series 10, Data from the National Health Survey.

[111]  M. Angermeyer,et al.  Mild cognitive impairment 1 – a review of prevalence, incidence and outcome according to current approaches , 2002, Acta psychiatrica Scandinavica.

[112]  C. Nass,et al.  Truth is beauty: researching embodied conversational agents , 2001 .

[113]  J. Millán-Calenti,et al.  [Accesibility and usability of computer-based cognitive stimulation: Telecognitio]. , 2010, Revista espanola de geriatria y gerontologia.

[114]  Eben M. Haber,et al.  CoScripter: automating & sharing how-to knowledge in the enterprise , 2008, CHI.

[115]  Sina Bahram,et al.  Prediction of web page accessibility based on structural and textual features , 2011, W4A.

[116]  D. Papalia Adult Development and Aging , 1995 .

[117]  Takanori Shibata,et al.  Living With Seal Robots—Its Sociopsychological and Physiological Influences on the Elderly at a Care House , 2007, IEEE Transactions on Robotics.

[118]  Cynthia Breazeal,et al.  The huggable: a therapeutic robotic companion for relational, affective touch , 2006, CCNC 2006. 2006 3rd IEEE Consumer Communications and Networking Conference, 2006..

[119]  D. Turner,et al.  Companion Animals in Human Health , 1997 .

[120]  Renata Pontin de Mattos Fortes,et al.  An evaluation of web accessibility metrics based on their attributes , 2008, SIGDOC '08.

[121]  Ben J. A. Kröse,et al.  The influence of social presence on enjoyment and intention to use of a robot and screen agent by elderly users , 2008, RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication.

[122]  Richard E. Ladner,et al.  WebInSight:: making web images accessible , 2006, Assets '06.

[123]  DAVID R. HILL,et al.  Substitution for a restricted visual channel in multimodal computer-human dialogue , 1988, IEEE Trans. Syst. Man Cybern..

[124]  Kate S. Hone,et al.  Empathic agents to reduce user frustration: The effects of varying agent characteristics , 2006, Interact. Comput..

[125]  Ben J. A. Kröse,et al.  Enjoyment, intention to use and actual use of a conversational robot by elderly people , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[126]  Gregg C. Vanderheiden,et al.  Use of Audio-Haptic Interface Techniques to Allow Nonvisual Access to Touchscreen Appliances , 1996 .

[127]  Frank Biocca,et al.  The Effect of the Agency and Anthropomorphism on Users' Sense of Telepresence, Copresence, and Social Presence in Virtual Environments , 2003, Presence: Teleoperators & Virtual Environments.

[128]  Jonathan Klein,et al.  Computers that recognise and respond to user emotion: theoretical and practical implications , 2002, Interact. Comput..

[129]  M. Swerts,et al.  Hearing and seeing beats : The influence of visual beats on the production and perception of prominence , 2006 .

[130]  Ashok Samal,et al.  Isolating symbols from connection lines in a class of engineering drawings , 1994, Pattern Recognit..

[131]  R. Kessler,et al.  Perceived support, received support, and adjustment to stressful life events. , 1986, Journal of health and social behavior.

[132]  Michael Lewis,et al.  Attentional Effect of Animated Character , 2003, INTERACT.