Exploring the Design Space of AAC Awareness Displays

Augmentative and alternative communication (AAC) devices are a critical technology for people with disabilities that affect their speech. One challenge with AAC systems is their inability to portray aspects of nonverbal communication that typically accent, complement, regulate, or substitute for verbal speech. In this paper, we explore the design space of awareness displays that can supplement AAC devices, considering their output features and their effects on the perceptions of interlocutors. Through designing prototypes and getting feedback on our designs from people with ALS, their primary caregivers, and other communication partners, we consider (1) the consistent tensions that arose between abstractness and clarity in meaning for these designs and (2) the ways in which these designs can further mark users as "other." Overall, we contribute a generative understanding of designing AAC awareness displays to augment and contextualize communication.

[1]  William W. Gaver.  What should we expect from research through design? , 2012, CHI.

[2]  Peter Washington,et al.  A Wearable Social Interaction Aid for Children with Autism , 2016, CHI Extended Abstracts.

[3]  Shelly Farnham,et al.  Structured online interactions: improving the decision-making of small discussion groups , 2000, CSCW '00.

[4]  Meredith Ringel Morris,et al.  "At times avuncular and cantankerous, with the reflexes of a mongoose": Understanding Self-Expression through Augmentative and Alternative Communication Devices , 2017, CSCW.

[5]  A. Ortony,et al.  What's basic about basic emotions? , 1990, Psychological review.

[6]  Katya Hill,et al.  Advances in augmentative and alternative communication as quality-of-life technology. , 2010, Physical medicine and rehabilitation clinics of North America.

[7]  D. Leder.  The Absent Body , 1990 .

[8]  R. Plutchik.  Emotion, a psychoevolutionary synthesis , 1980 .

[9]  Mary Czerwinski,et al.  Actuating mood: design of the textile mirror , 2013, TEI '13.

[10]  Ruli Manurung,et al.  Evaluating the STANDUP Pun Generating Software with Children with Cerebral Palsy , 2009, TACC.

[11]  Thad Starner,et al.  MoodLens: towards improving nonverbal emotional expression with an in-lens fiber optic display , 2016, SEMWEB.

[12]  Michael Oliver,et al.  Understanding Disability: From Theory to Practice , 1995 .

[13]  P. Enderby,et al.  Barriers and facilitators to the use of high-technology augmentative and alternative communication devices: a systematic review and qualitative synthesis. , 2012, International journal of language & communication disorders.

[14]  Scott Counts,et al.  Exploring wearable ambient displays for social awareness , 2006, CHI EA '06.

[15]  Maria da Graça Campos Pimentel,et al.  Filteryedping: A Dwell-Free Eye Typing Technique , 2015, CHI Extended Abstracts.

[16]  Margrit Betke,et al.  EyeSwipe: Dwell-free Text Entry Using Gaze Paths , 2016, CHI.

[17]  Poika Isokoski,et al.  Now Dasher! Dash away!: longitudinal study of fast text entry by Eye Gaze , 2008, ETRA.

[18]  Jacob O. Wobbrock,et al.  In the shadow of misperception: assistive technology use and social interactions , 2011, CHI.

[19]  D. Beukelman,et al.  Augmentative & Alternative Communication: Supporting Children & Adults With Complex Communication Needs , 2006 .

[20]  S. King,et al.  Speech synthesis technologies for individuals with vocal disabilities: Voice banking and reconstruction , 2012 .

[21]  Meredith Ringel Morris,et al.  AACrobat: Using Mobile Devices to Lower Communication Barriers and Provide Autonomy with Gaze-Based AAC , 2017, CSCW.

[22]  Sethuraman Panchanathan,et al.  Affective Haptics for Enhancing Access to Social Interactions for Individuals Who are Blind , 2014, HCI.

[23]  Norman Alm,et al.  Using NLG to Help Language-Impaired Users Tell Stories and Participate in Social Dialogues , 2009, ENLG.

[24]  Mohammed Yeasin,et al.  EmoAssist: emotion enabled assistive tool to enhance dyadic conversation for the blind , 2016, Multimedia Tools and Applications.

[25]  Yun Yang,et al.  Conversational Awareness in Text-Based Computer Mediated Communication , 2009, Awareness Systems.

[26]  Gökhan Mura.  WEARABLE TECHNOLOGIES FOR EMOTION COMMUNICATION , 2008 .

[27]  Per Ola Kristensson,et al.  The potential of dwell-free eye-typing for fast assistive gaze communication , 2012, ETRA.

[28]  Sethuraman Panchanathan,et al.  A Wizard of Oz Study Exploring How Agreement/Disagreement Nonverbal Cues Enhance Social Interactions for Individuals Who Are Blind , 2014, HCI.

[29]  Mary Wickenden.  Whose Voice is That?: Issues of Identity, Voice and Representation Arising in an Ethnographic Study of the Lives of Disabled Teenagers who use Augmentative and Alternative Communication (AAC) , 2011 .

[30]  BRUCE WISENBURN,et al.  Participant Evaluations of Rate and Communication Efficacy of an AAC Application Using Natural Language Processing , 2009, Augmentative and alternative communication.

[31]  P. Square,et al.  A Set of Observational Measures for Rating Support and Participation in Conversation Between Adults with Aphasia and Their Conversation Partners , 2004, Topics in stroke rehabilitation.

[32]  Meredith Ringel Morris,et al.  SWARM: An Actuated Wearable for Mediating Affect , 2015, TEI.

[33]  Jeffery D. Higginbotham,et al.  Design Meets Disability , 2009 .

[34]  John Todman,et al.  Rate and quality of conversations using a text-storage AAC system: Single-case training study , 2000 .

[35]  John C. Tang.  Approaching and leave-taking: Negotiating contact in computer-mediated communication , 2007, TCHI.

[36]  Simon King,et al.  Towards Personalised Synthesised Voices for Individuals with Vocal Disabilities: Voice Banking and Reconstruction , 2013, SLPAT.

[37]  Christopher Frauenberger,et al.  Disability and Technology: A Critical Realist Perspective , 2015, ASSETS.

[38]  John C. Tang,et al.  Motion and Attention in a Kinetic Videoconferencing Proxy , 2011, INTERACT.

[39]  C. Guimond.  Amyotrophic Lateral Sclerosis: A Guide for Patients and Families , 2001 .

[40]  Simon King,et al.  音声の障害患者のための音声合成枝術 : Voice banking and reconstruction , 2011 .

[41]  Sethuraman Panchanathan,et al.  Assistive Technologies as Effective Mediators in Interpersonal Social Interactions for Persons with Visual Disability , 2010, ICCHP.

[42]  Mark Billinghurst,et al.  CoSense: Creating Shared Emotional Experiences , 2015, CHI Extended Abstracts.

[43]  Deborah I. Fels,et al.  Emotive captioning , 2007, CIE.

[44]  Joan Murphy.  "I Prefer Contact This Close": Perceptions of AAC by People with Motor Neurone Disease and their Communication Partners , 2004 .

[45]  Selma Sabanovic,et al.  Happy Moves, Sad Grooves: Using Theories of Biological Motion and Affect to Design Shape-Changing Interfaces , 2016, Conference on Designing Interactive Systems.

[46]  Martin Tomitsch,et al.  Designing a personal visualization projection of online social identity , 2011, CHI EA '11.

[47]  Carl Gutwin,et al.  Heuristic Evaluation of Groupware Based on the Mechanics of Collaboration , 2001, EHCI.

[48]  Pamela A McCombe,et al.  Effects of gender in amyotrophic lateral sclerosis. , 2010, Gender medicine.

[49]  David R. Beukelman,et al.  Eye-gaze access to AAC technology for people with amyotrophic lateral sclerosis , 2010 .

[50]  Antonio Nardone,et al.  Eye tracking communication devices in amyotrophic lateral sclerosis: Impact on disability and quality of life , 2013, Amyotrophic lateral sclerosis & frontotemporal degeneration.

[51]  T. Shakespeare,et al.  Cultural Representation of Disabled People: Dustbins for Disavowal? , 1994 .

[52]  Bilge Mutlu,et al.  In-body experiences: embodiment, control, and trust in robot-mediated communication , 2013, CHI.

[53]  Jessica L. Feuston.  Expressiv: An Expressive System for Individuals with Facial Paralysis , 2015 .

[54]  A. Osborn.  Applied imagination : principles and procedures of creative problem-solving , 1957 .

[55]  Oleg Spakov,et al.  Fast gaze typing with an adjustable dwell time , 2009, CHI.

[56]  Rosalind W. Picard,et al.  Affective Objects , 1999 .

[57]  T. Munsat,et al.  Amyotrophic Lateral Sclerosis: A Guide for Patients and Families , 2001 .

[58]  J. Russell,et al.  The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology , 2005, Development and Psychopathology.

[59]  Rosalee Wolfe,et al.  Combining emotion and facial nonmanual signals in synthesized american sign language , 2012, ASSETS '12.

[60]  Yun Yang,et al.  Supporting awareness in instant messaging: an empirical study and mechanism design , 2005, OZCHI.

[61]  Gillian R. Hayes,et al.  SayWAT: Augmenting Face-to-Face Conversations for Adults with Autism , 2016, CHI.

[62]  Youngwoo Park,et al.  Wrigglo: shape-changing peripheral for interpersonal mobile communication , 2014, CHI.

[63]  Mary Czerwinski,et al.  MoodWings: a wearable biofeedback device for real-time stress intervention , 2013, PETRA '13.

[64]  Richard E. Ladner,et al.  Freedom to roam: a study of mobile device adoption and accessibility for people with visual and motor disabilities , 2009, Assets '09.

[65]  Ellen Isaacs,et al.  Hubbub: a sound-enhanced mobile instant messenger that supports awareness and opportunistic interactions , 2002, CHI.

[66]  Halley Profita,et al.  The AT Effect: How Disability Affects the Perceived Social Acceptability of Head-Mounted Display Use , 2016, CHI.

[67]  N. Hoffart.  Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory , 2000 .

[68]  J. J. Higgins,et al.  The aligned rank transform for nonparametric factorial analyses using only anova procedures , 2011, CHI.

[69]  Leila Takayama,et al.  "Now, i have a body": uses and social norms for mobile remote presence in the workplace , 2011, CHI.