Fluent: An AI Augmented Writing Tool for People who Stutter

Stuttering is a speech disorder which impacts the personal and professional lives of millions of people worldwide. To save themselves from stigma and discrimination, people who stutter (PWS) may adopt different strategies to conceal their stuttering. One of the common strategies is word substitution where an individual avoids saying a word they might stutter on and use an alternative instead. This process itself can cause stress and add more burden. In this work, we present Fluent, an AI augmented writing tool which assists PWS in writing scripts which they can speak more fluently. Fluent embodies a novel active learning based method of identifying words an individual might struggle pronouncing. Such words are highlighted in the interface. On hovering over any such word, Fluent presents a set of alternative words which have similar meaning but are easier to speak. The user is free to accept or ignore these suggestions. Based on such user interaction (feedback), Fluent continuously evolves its classifier to better suit the personalized needs of each user. We evaluated our tool by measuring its ability to identify difficult words for 10 simulated users. We found that our tool can identify difficult words with a mean accuracy of over 80% in under 20 interactions and it keeps improving with more feedback. Our tool can be beneficial for certain important life situations like giving a talk, presentation, etc. The source code for this tool has been made publicly accessible at github.com/bhavyaghai/Fluent.

[1]  Wieslawa Kuniszyk-Józkowiak,et al.  Hierarchical ANN system for stuttering identification , 2013, Comput. Speech Lang..

[2]  Diyi Yang,et al.  RECAST: Enabling User Recourse and Interpretability of Toxicity Detection Models with Interactive Visualization , 2021, Proc. ACM Hum. Comput. Interact..

[3]  Seth E. Tichenor,et al.  Group Experiences and Individual Differences in Stuttering. , 2019, Journal of speech, language, and hearing research : JSLHR.

[4]  Xian Li,et al.  Design and Evaluation of a Social Media Writing Support Tool for People with Dyslexia , 2019, CHI.

[5]  Aslam Muhammad,et al.  An Intelligent and Interactive Interface to Support Symmetrical Collaborative Educational Writing among Visually Impaired and Sighted Users , 2019, Symmetry.

[6]  William A. Gale,et al.  A sequential algorithm for training text classifiers , 1994, SIGIR '94.

[7]  H. Levitt,et al.  Coping responses by adults who stutter: part I. Protecting the self and others. , 2009, Journal of fluency disorders.

[8]  J. Yaruss,et al.  Assessing quality of life in stuttering treatment outcomes research. , 2010, Journal of fluency disorders.

[9]  Michael Frueh,et al.  Stuttering An Integrated Approach To Its Nature And Treatment , 2016 .

[10]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[11]  Nader Jafarnia Dabanloo,et al.  An Automatic Prolongation Detection Approach in Continuous Speech With Robustness Against Speaking Rate Variations , 2017, Journal of medical signals and sensors.

[12]  Michael Bloodgood,et al.  Stopping Active Learning Based on Predicted Change of F Measure for Text Classification , 2019, 2019 IEEE 13th International Conference on Semantic Computing (ICSC).

[13]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[14]  Sanjoy Dasgupta,et al.  Hierarchical sampling for active learning , 2008, ICML '08.

[15]  J. Douglass,et al.  Speech-Language Therapy Experiences Across the Life Span of an Individual Who Covertly Stutters: A Thematic Analysis , 2020, Perspectives of the ASHA Special Interest Groups.

[16]  M. Garcia-Barrera,et al.  Anticipation in stuttering: A theoretical model of the nature of stutter prediction. , 2015, Journal of fluency disorders.

[17]  Ali Etemad,et al.  FluentNet: End-to-End Detection of Speech Disfluency with Deep Learning , 2020, ArXiv.

[18]  Phil D. Green,et al.  Sequence labeling to detect stuttering events in read speech , 2020, Comput. Speech Lang..

[19]  G. Andrews,et al.  Stuttering: a review of research findings and theories circa 1982. , 1983, The Journal of speech and hearing disorders.

[20]  Fabio Fassetti,et al.  Learning and Detecting Stuttering Disorders , 2019, AIAI.

[21]  Michael P. Boyle.  Enacted stigma and felt stigma experienced by adults who stutter. , 2018, Journal of communication disorders.

[22]  Laksnoria Laksnoria Karyuatry,et al.  Grammarly as a Tool to Improve Students’ Writing Quality: Free Online-Proofreader across the Boundaries , 2018 .

[23]  Dante Elías,et al.  A Novel Stuttering Disfluency Classification System Based on Respiratory Biosignals , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[24]  Burr Settles,et al.  From Theories to Queries: Active Learning in Practice , 2011 .

[25]  The Indians have many terms for it: stuttering among the Bannock-Shoshoni. , 1983, Journal of speech and hearing research.

[26]  K. M. Ravikumar,et al.  Automatic Detection of Syllable Repetition in Read Speech for Objective Assessment of Stuttered Disfluencies , 2008 .

[27]  R. Bellamy,et al.  Explainable Active Learning (XAL): Toward AI Explanations as Interfaces for Machine Teachers , 2021 .

[28]  Afshin Rostamizadeh,et al.  The Practical Challenges of Active Learning: Lessons Learned from Live Experimentation , 2019, ArXiv.

[29]  Yang Gao,et al.  Active learning with confidence-based answers for crowdsourcing labeling tasks , 2018, Knowl. Based Syst..

[30]  Nicole Propst,et al.  Fluency And Stuttering , 2016 .

[31]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[32]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[33]  Andrew S. Gordon,et al.  Creative Help: A Story Writing Assistant , 2015, ICIDS.

[34]  Nedelina Ivanova,et al.  BiBiKit - A Bilingual Bimodal Reading and Writing Tool for Sign Language Users , 2012, LREC.

[35]  Maria Eugenia Ramirez-Loaiza,et al.  Active learning: an empirical study of common baselines , 2017, Data Mining and Knowledge Discovery.

[36]  Arun Das,et al.  Stuttering Speech Disfluency Prediction using Explainable Attribution Vectors of Facial Muscle Movements , 2020, ArXiv.

[37]  Hua Xu,et al.  A study of active learning methods for named entity recognition in clinical text , 2015, J. Biomed. Informatics.

[38]  Allison Parrish.  Poetic Sound Similarity Vectors Using Phonetic Features , 2017, AIIDE Workshops.

[39]  L. M. Hulit,et al.  The association of attitudes toward stuttering with selected variables , 1994 .

[40]  Tomi Kinnunen,et al.  SWAN - Scientific Writing AssistaNt. A Tool for Helping Scholars to Write Reader-Friendly Manuscripts , 2012, EACL.

[41]  Seth E. Tichenor,et al.  Variability of Stuttering: Behavior and Impact. , 2020, American journal of speech-language pathology.

[42]  Peter Dalsgaard,et al.  Mapping the Landscape of Creativity Support Tools in HCI , 2019, CHI.

[43]  Rong Jin,et al.  Active Learning by Querying Informative and Representative Examples , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  L. McAllister,et al.  An investigation of equity of rural speech-language pathology services for children: A geographic perspective , 2011, International Journal of Speech-Language Pathology.

[45]  P. Howell,et al.  Phonetic difficulty and stuttering in English , 2006, Clinical linguistics & phonetics.

[46]  Herman Kamper,et al.  Improving Unsupervised Acoustic Word Embeddings using Speaker and Gender Information , 2020, 2020 International SAUPEC/RobMech/PRASA Conference.

[47]  S. Hood,et al.  The impact of stuttering on employment opportunities and job performance. , 2004, Journal of fluency disorders.

[48]  David Lowell,et al.  Practical Obstacles to Deploying Active Learning , 2019, EMNLP/IJCNLP.

[49]  Daphne Koller,et al.  Support Vector Machine Active Learning with Applications to Text Classification , 2000, J. Mach. Learn. Res..

[50]  Chaozhe Zhu,et al.  Classification of Types of Stuttering Symptoms Based on Brain Activity , 2012, PloS one.

[51]  Jason S. Chang,et al.  FLOW: A First-Language-Oriented Writing Assistant System , 2012, ACL.

[52]  Covert Stuttering , 2019, The SAGE Encyclopedia of Human Communication Sciences and Disorders.

[53]  K. K. Paynter,et al.  Impact of stuttering on perception of occupational competence , 1990 .

[54]  H. Sebastian Seung,et al.  Selective Sampling Using the Query by Committee Algorithm , 1997, Machine Learning.

[55]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[56]  Michael P. Boyle,et al.  Disclosure of stuttering and quality of life in people who stutter. , 2018, Journal of fluency disorders.

[57]  Sazali Yaacob,et al.  Overview of Automatic Stuttering Recognition System , 2009 .

[58]  E. Boberg,et al.  An investigation of interclinic agreement in the identification of fluent and stuttered syllables , 1988 .

[59]  S. Nordstrom,et al.  Rethinking covert stuttering. , 2017, Journal of fluency disorders.

[60]  Mubbasir Kapadia,et al.  LISA: Lexically Intelligent Story Assistant , 2021, AIIDE.

[61]  Michael P. Boyle,et al.  Self-stigma and its associations with stress, physical health, and health care satisfaction in adults who stutter. , 2017, Journal of fluency disorders.

[62]  Tomás Hanakovic,et al.  Speech Recognition Helps Visually Impaired People Writing Mathematical Formulas , 2006, ICCHP.

[63]  A. Leung,et al.  Stuttering , 1990 .

[64]  H. Sebastian Seung,et al.  Query by committee , 1992, COLT '92.

[65]  M. Vanryckeghem,et al.  Effectiveness of Behavioral Stuttering Treatment: A Systemic Review and Meta-Analysis , 2006 .

[66]  G. Kikano,et al.  Stuttering: an overview. , 2008, American family physician.