The Unsupervised Acquisition of a Lexicon from Continuous Speech

We present an unsupervised learning algorithm that acquires a natural- language lexicon from raw speech. The algorithm is based on the optimal encoding of symbol sequences in an MDL framework, and uses a hierarchical representation of language that overcomes many of the problems that have stymied previous grammar-induction procedures. The forward mapping from symbol sequences to the speech stream is modeled using features based on articulatory gestures. We present results on the acquisition of lexicons and language models from raw speech, text, and phonetic transcripts, and demonstrate that our algorithm compares very favorably to other reported results with respect to segmentation performance and statistical efficiency.

[1]  Carl de Marcken,et al.  Lexical Heads, Phrase Structure and the Induction of Grammar , 1995, VLC@ACL.

[2]  S. J. Keyser,et al.  The View from Building 20: Essays in Linguistics in Honor of Sylvain Bromberger , 1993 .

[3]  R. Burchfield.  Frequency Analysis of English Usage: Lexicon and Grammar. By W. Nelson Francis and Henry Kučera with the assistance of Andrew W. Mackie. Boston: Houghton Mifflin. 1982. x + 561 , 1985 .

[4]  Carl de Marcken.  The Acquisition of a Lexicon from Paired Phoneme Sequences and Semantic Representations , 1994, ICGI.

[5]  J. Baker.  Trainable grammars for speech recognition , 1979 .

[6]  John D. Lafferty,et al.  Inducing Features of Random Fields , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Steven Bird,et al.  One-Level Phonology: Autosegmental Representations and Rules as Finite Automata , 1994, Comput. Linguistics.

[8]  C. Snow,et al.  Input and interaction in language acquisition: The changing role of negative evidence in theories of language development , 1994 .

[9]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[10]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[11]  Morris Halle,et al.  On distinctive features and their articulatory implementation , 1983 .

[12]  Jacques Mehler,et al.  The role of attention in speech perception by young infants , 1990 .

[13]  Frédéric Bimbot,et al.  Language modeling by variable length sequences: theoretical formulation and evaluation of multigrams , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[14]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[15]  J. Gerard Wolff,et al.  Language acquisition, data compression and generalization , 1982 .

[16]  A. Cutler.  Segmentation problems, rhythmic solutions * , 1994 .

[17]  Stanley F. Chen,et al.  Bayesian Grammar Induction for Language Modeling , 1995, ACL.

[18]  Noam Chomsky,et al.  The Logical Structure of Linguistic Theory , 1975 .

[19]  Fernando Pereira,et al.  Inside-Outside Reestimation From Partially Bracketed Corpora , 1992, HLT.

[20]  J. Wolff,et al.  Language Acquisition and the Discovery of Phrase Structure , 1980, Language and speech.

[21]  Michael Kenstowicz,et al.  Phonology In Generative Grammar , 1994 .

[22]  Martin Kay,et al.  Regular Models of Phonological Rule Systems , 1994, CL.

[23]  W. Nelson Francis,et al.  FREQUENCY ANALYSIS OF ENGLISH USAGE: LEXICON AND GRAMMAR , 1983 .

[24]  Jeffrey Mark Siskind,et al.  Naive physics, event perception, lexical semantics, and language acquisition , 1992 .

[25]  Jeffrey Mark Siskind.  Lexical Acquisition as Constraint Satisfaction , 1993 .

[26]  C Snow,et al.  Child language data exchange system , 1984, Journal of Child Language.

[27]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[28]  Glenn Carroll,et al.  Learn-ing probaballstic dependency grammars from labelled text , 1992 .

[29]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[30]  Eric Sven Ristad,et al.  New Techniques for Context Modeling , 1995, ACL.

[31]  Jeffrey Mark Siskind,et al.  Lexical Acquisition in the Presence of Noise and Homonymy , 1994, AAAI.

[32]  J. Gerard Wolfp,et al.  Language Acquisition and the Discovery of Phrase Structure , 1980 .

[33]  Morris Halle,et al.  Distributed morphology and the pieces of inflection , 1993 .

[34]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[35]  R. Jansen,et al.  LANGUAGE ACQUISITION , 1977, The Medical journal of Australia.