Sibylle, An Assistive Communication System Adapting to the Context and Its User

In this article, we describe the latest version of Sibylle, an AAC system that permits persons who have severe physical disabilities to enter text with any computer application, as well as to compose messages to be read out through speech synthesis. The system consists of a virtual keyboard comprising a set of keypads that allow for the entering of characters or full words by a single-switch selection process. It also includes a sophisticated word prediction component which dynamically calculates the most appropriate words for a given context. This component is auto-adaptive, that is, it learns with every text the user enters. It thus adapts its predictions to the user's language and the current topic of communication as well. So far, the system works for French, German and English. Earlier versions of Sibylle have been used since 2001 in a rehabilitation center (Kerpape, France).

[1]  I. Scott MacKenzie,et al.  Metrics for text entry research: an evaluation of MSD and KSPC, and a new unified error metric , 2003, CHI '03.

[2]  Ronald Rosenfeld,et al.  A maximum entropy approach to adaptive statistical language modelling , 1996, Comput. Speech Lang..

[3]  Anthony J. Robinson,et al.  Language model adaptation using mixtures and an exponentially decaying cache , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[4]  James F. Allen.  Natural language understanding (2. ed.) , 1994 .

[5]  Stuart M. Shieber,et al.  Abbreviated text input , 2003, IUI '03.

[6]  Kathleen F. McCoy,et al.  Corpus studies in word prediction , 2007, Assets '07.

[7]  Jianhua Li,et al.  Semantic knowledge in word completion , 2005, Assets '05.

[8]  Simon P. Levine,et al.  Modeling the speed of text entry with a word prediction interface , 1994 .

[9]  Kenneth Ward Church,et al.  Word Association Norms, Mutual Information, and Lexicography , 1989, ACL.

[10]  Alan F. Newell,et al.  Adaptive and predictive techniques in a communication prosthesis , 1987 .

[11]  Kamel Smaïli,et al.  Dynamic Topic Identification: Towards Combination of Methods , 2001 .

[12]  Kathleen F. McCoy,et al.  The Effects of Word Prediction on Communication Rate for AAC , 2007, HLT-NAACL.

[13]  Philippe Boissière,et al.  From a specialised writing interface created for the disabled to a predictive interface for all: the VITIPI System , 2001, HCI.

[14]  Marco Baroni,et al.  THE LANGUAGE COMPONENT OF THE FASTY TEXT PREDICTION SYSTEM , 2005, Appl. Artif. Intell..

[15]  Graeme Hirst,et al.  Testing the Efficacy of Part-of-Speech Information in Word Completion , 2003 .

[16]  Jerome R. Bellegarda,et al.  A latent semantic analysis framework for large-Span language modeling , 1997, EUROSPEECH.

[17]  C Morris,et al.  Syntax PAL: a system to improve the written syntax of language-impaired users. , 1992, Assistive technology : the official journal of RESNA.

[18]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[19]  Andreas Stolcke,et al.  Entropy-based Pruning of Backoff Language Models , 2000, ArXiv.

[20]  Joshua Goodman,et al.  A bit of progress in language modeling , 2001, Comput. Speech Lang..

[21]  Denis Anson,et al.  The Effects of Word Completion and Word Prediction on Typing Rates Using On-Screen Keyboards , 2006, Assistive technology : the official journal of RESNA.

[22]  Marco Baroni,et al.  Wordform- and Class-based Prediction of the Components of German Nominal Compounds in an AAC System , 2002, COLING.

[23]  Renato De Mori,et al.  A Cache-Based Natural Language Model for Speech Recognition , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Joshua Goodman,et al.  A bit of progress in language modeling , 2001, Comput. Speech Lang..

[25]  Alan F. Newell,et al.  Prediction and conversational momentum in an augmentative communication system , 1992, CACM.

[26]  Daniel Jurafsky,et al.  Towards better integration of semantic predictors in statistical language modeling , 1998, ICSLP.

[27]  Tonio Wandmacher,et al.  How semantic is Latent Semantic Analysis? , 2005, JEPTALNRECITAL.

[28]  Tonio Wandmacher,et al.  SIBYLLE: a system for alternative communication adapting to the context and its user , 2007, Assets '07.

[29]  Tonio Wandmacher,et al.  Methods to Integrate a Language Model with Semantic Information for a Word Prediction Component , 2007, EMNLP.

[30]  Bob Rehder,et al.  How Well Can Passage Meaning be Derived without Using Word Order? A Comparison of Latent Semantic Analysis and Humans , 1997 .

[31]  Andreas Stolcke,et al.  SRILM - an extensible language modeling toolkit , 2002, INTERSPEECH.

[32]  Richard A. Harshman,et al.  Indexing by Latent Semantic Analysis , 1990, J. Am. Soc. Inf. Sci..

[33]  Kathleen F. McCoy,et al.  Topic modeling in fringe word prediction for AAC , 2006, IUI '06.

[34]  Stephen Clark,et al.  Informing Flexible Abbreviation Expansion for Users with Motor Disabilities , 2002, ICCHP.

[35]  Tina Magnuson,et al.  Profet, A New Generation of Word Prediction: An Evaluation Study , 1997 .

[36]  Anna Maria Di Sciullo,et al.  Natural Language Understanding , 2009, SoMeT.

[37]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[38]  Tonio Wandmacher,et al.  Training Language Models without Appropriate Language Resources: Experiments with an AAC System for Disabled People , 2006, LREC.

[39]  James F. Allen.  Natural language understanding , 1987, Bejnamin/Cummings series in computer science.

[40]  Claude E. Shannon,et al.  Prediction and Entropy of Printed English , 1951 .

[41]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[42]  Igor Schadle,et al.  Towards a large corpus of spoken dialogue in French that will be freely available: the "Parole Publique" project and its first realisations , 2002, LREC.

[43]  James F. Allen.  Natural language understanding (2nd ed.) , 1995 .

[44]  Aaron D. Wyner,et al.  Prediction and Entropy of Printed English , 1993 .

[45]  David Niemeijer,et al.  Evaluating effort reduction through different word prediction systems , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[46]  Peter Wiemer-Hastings,et al.  Latent semantic analysis , 2004, Annu. Rev. Inf. Sci. Technol..

[47]  Alan F. Blackwell,et al.  Dasher—a data entry interface using continuous gestures and language models , 2000, UIST '00.

[48]  Nestor Garay-Vitoria,et al.  Text prediction systems: a survey , 2006, Universal Access in the Information Society.

[49]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .