Locality sensitive hashing for fast computation of correlational manifold learning based feature space transformations

Manifold learning based techniques have been found to be useful for feature space transformations and semi-supervised learning in speech processing. However, the immense computational requirements in building neighborhood graphs have hindered the application of these techniques to large speech corpora. This paper presents an approach for fast computation of neighborhood graphs in the context of manifold learning. The approach, known as locality sensitive hashing (LSH), has been applied to a discriminative manifold learning based feature space transformation technique that utilizes a cosine-correlation based distance measure. Performance is evaluated first in terms computational savings at a given level of ASR performance. The results demonstrate that LSH provides a factor of 9 reduction in the computational complexity with minimal impact on speech recognition performance. A study is also performed comparing the efficiency of the LSH algorithm presented here and other LSH approaches in identifying nearest neighbors.

[1]  Mjf Gales Adapting semi-tied full-convariance matrix HMMs , 1997 .

[2]  Laurent Amsaleg,et al.  Locality sensitive hashing: A comparison of hash function types and querying mechanisms , 2010, Pattern Recognit. Lett..

[3]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[4]  Yun Fu,et al.  Conformal Embedding Analysis with Local Graph Modeling on the Unit Hypersphere , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[6]  Biing-Hwang Juang,et al.  A family of distortion measures based upon projection operation for robust speech recognition , 1989, IEEE Trans. Acoust. Speech Signal Process..

[7]  David G. Stork,et al.  Pattern Classification , 1973 .

[8]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[9]  Nicole Immorlica,et al.  Locality-sensitive hashing scheme based on p-stable distributions , 2004, SCG '04.

[10]  V. Zolotarev One-dimensional stable distributions , 1986 .

[11]  Piotr Indyk,et al.  Approximate Nearest Neighbor: Towards Removing the Curse of Dimensionality , 2012, Theory Comput..

[12]  Hermann Ney,et al.  Experiments with linear feature extraction in speech recognition , 1995, EUROSPEECH.

[13]  Yun Tang,et al.  A study of using locality preserving projections for feature extraction in speech recognition , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[14]  Patrick Pantel,et al.  Randomized Algorithms and NLP: Using Locality Sensitive Hash Functions for High Speed Noun Clustering , 2005, ACL.

[15]  Piotr Indyk,et al.  Approximate nearest neighbors: towards removing the curse of dimensionality , 1998, STOC '98.

[16]  Mark A. Clements,et al.  Speech recognition in noise using a projection-based likelihood measure for mixture density HMM's , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[17]  Biing-Hwang Juang,et al.  A family of distortion measures base upon projection operation for robust speech recognition , 1988, ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing.

[18]  Aren Jansen,et al.  Efficient spoken term discovery using randomized algorithms , 2011, 2011 IEEE Workshop on Automatic Speech Recognition & Understanding.

[19]  Aren Jansen,et al.  Intrinsic Fourier Analysis on the Manifold of Speech Sounds , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[20]  Richard C. Rose,et al.  Noise aware manifold learning for robust speech recognition , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Richard C. Rose,et al.  Application of a locality preserving discriminant analysis approach to ASR , 2012, 2012 11th International Conference on Information Science, Signal Processing and their Applications (ISSPA).

[23]  Shihong Lao,et al.  Discriminant analysis in correlation similarity measure space , 2007, ICML '07.

[24]  Moses Charikar,et al.  Similarity estimation techniques from rounding algorithms , 2002, STOC '02.

[25]  Trevor Darrell,et al.  Nearest-Neighbor Methods in Learning and Vision: Theory and Practice (Neural Information Processing) , 2006 .

[26]  Richard C. Rose,et al.  A Correlational Discriminant Approach to Feature Extraction for Robust Speech Recognition , 2012, INTERSPEECH.