Development of a Real Time Sparse Non-Negative Matrix Factorization Module for Cochlear Implants by Using xPC Target

Cochlear implants (CIS) require efficient speech processing to maximize information transmission to the brain, especially in noise. A novel CI processing strategy was proposed in our previous studies, in which sparsity-constrained non-negative matrix factorization (NMF) was applied to the envelope matrix in order to improve the CI performance in noisy environments. It showed that the algorithm needs to be adaptive, rather than fixed, in order to adjust to acoustical conditions and individual characteristics. Here, we explore the benefit of a system that allows the user to adjust the signal processing in real time according to their individual listening needs and their individual hearing capabilities. In this system, which is based on MATLAB®, SIMULINK® and the xPC Target™ environment, the input/outupt (I/O) boards are interfaced between the SIMULINK blocks and the CI stimulation system, such that the output can be controlled successfully in the manner of a hardware-in-the-loop (HIL) simulation, hence offering a convenient way to implement a real time signal processing module that does not require any low level language. The sparsity constrained parameter of the algorithm was adapted online subjectively during an experiment with normal-hearing subjects and noise vocoded speech simulation. Results show that subjects chose different parameter values according to their own intelligibility preferences, indicating that adaptive real time algorithms are beneficial to fully explore subjective preferences. We conclude that the adaptive real time systems are beneficial for the experimental design, and such systems allow one to conduct psychophysical experiments with high ecological validity.

[1]  Hongmei Hu,et al.  Non-negative matrix factorization on the envelope matrix in cochlear implant , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[2]  B. Moore,et al.  Benefit of high-rate envelope cues in vocoder processing: effect of number of channels and spectral region. , 2008, The Journal of the Acoustical Society of America.

[3]  Yi Hu,et al.  Objective measures for predicting speech intelligibility in noisy conditions based on new band-importance functions. , 2009, The Journal of the Acoustical Society of America.

[4]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[5]  Arne Leijon,et al.  A new linear MMSE filter for single channel speech enhancement based on Nonnegative Matrix Factorization , 2011, 2011 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA).

[6]  John R. Hershey,et al.  Efficient model-based speech separation and denoising using non-negative subspace analysis , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  Guoping Li.  Speech perception in a sparse domain , 2008 .

[8]  James F Patrick,et al.  The Development of the Nucleus® Freedom™ Cochlear Implant System , 2006, Trends in amplification.

[9]  Jesper Jensen,et al.  An Algorithm for Intelligibility Prediction of Time–Frequency Weighted Noisy Speech , 2011, IEEE Transactions on Audio, Speech, and Language Processing.

[10]  Juan Wang,et al.  Improved Image Fusion Method Based on NSCT and Accelerated NMF , 2012, Sensors.

[11]  J Bamford,et al.  The BKB (Bamford-Kowal-Bench) sentence lists for partially-hearing children. , 1979, British journal of audiology.

[12]  Michael W. Spratling.  Learning Image Components for Object Recognition , 2006, J. Mach. Learn. Res..

[13]  Philipos C Loizou,et al.  Speech processing in vocoder-centric cochlear implants. , 2006, Advances in oto-rhino-laryngology.

[14]  Andrzej Cichocki,et al.  New Algorithms for Non-Negative Matrix Factorization in Applications to Blind Source Separation , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[15]  Richard M. Schwartz,et al.  Enhancement of speech corrupted by acoustic noise , 1979, ICASSP.

[16]  Fei Chen,et al.  Analysis of a simplified normalized covariance measure based on binary weighting functions for predicting the intelligibility of noise-suppressed speech. , 2010, The Journal of the Acoustical Society of America.

[17]  Jan Larsen,et al.  Single-channel source separation using non-negative matrix factorization , 2009 .

[18]  Raymond L. Goldsworthy,et al.  Analysis of speech-based Speech Transmission Index methods with implications for nonlinear operations. , 2004, The Journal of the Acoustical Society of America.

[19]  Hongmei Hu,et al.  Simulation of hearing loss using compressive gammachirp auditory filters , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[20]  Francesco Piazza,et al.  Nonlinear Speech Enhancement: An Overview , 2005, WNSP.

[21]  Patrik O. Hoyer,et al.  Non-negative sparse coding , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.

[22]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[23]  Ehud Weinstein,et al.  Iterative and sequential Kalman filter-based speech enhancement algorithms , 1998, IEEE Trans. Speech Audio Process..

[24]  Rainer Martin,et al.  Noise power spectral density estimation based on optimal smoothing and minimum statistics , 2001, IEEE Trans. Speech Audio Process..

[25]  R V Shannon,et al.  Speech Recognition with Primarily Temporal Cues , 1995, Science.

[26]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[27]  T Houtgast,et al.  A physical method for measuring speech-transmission quality. , 1980, The Journal of the Acoustical Society of America.

[28]  N. Mohammadiha,et al.  Nonnegative matrix factorization using projected gradient algorithms with sparseness constraints , 2009, 2009 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT).

[29]  Tuomas Virtanen,et al.  Monaural Sound Source Separation by Nonnegative Matrix Factorization With Temporal Continuity and Sparseness Criteria , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[30]  Andrzej Cichocki,et al.  Fast Nonnegative Matrix Factorization Algorithms Using Projected Gradient Approaches for Large-Scale Problems , 2008, Comput. Intell. Neurosci..

[31]  Stefan J. Mauger,et al.  Clinical Evaluation of Signal-to-Noise Ratio–Based Noise Reduction in Nucleus® Cochlear Implant Recipients , 2011, Ear and hearing.

[32]  P. Smaragdis,et al.  Non-negative matrix factorization for polyphonic music transcription , 2003, 2003 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (IEEE Cat. No.03TH8684).

[33]  Philipos C. Loizou,et al.  Speech Enhancement: Theory and Practice , 2007 .

[34]  Ephraim.  Speech enhancement using a minimum mean square error short-time spectral amplitude estimator , 1984 .

[35]  J. Boudy,et al.  Non-linear spectral subtraction (NSS) and hidden Markov models for robust speech recognition in car noise environments , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[36]  Richard C. Hendriks,et al.  Noise Correlation Matrix Estimation for Multi-Microphone Speech Enhancement , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[37]  Philipos C. Loizou,et al.  On the design and evaluation of the PDA-based research platform for electric and acoustic stimulation , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[38]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[39]  Liang Chen,et al.  Enhanced sparse speech processing strategy for cochlear implants , 2011, 2011 19th European Signal Processing Conference.

[40]  Bhiksha Raj,et al.  Non-negative Hidden Markov Modeling of Audio with Application to Source Separation , 2010, LVA/ICA.

[41]  Bhiksha Raj,et al.  Probabilistic Latent Variable Models as Nonnegative Factorizations , 2008, Comput. Intell. Neurosci..

[42]  Lars Kai Hansen,et al.  Approximate L0 constrained non-negative matrix and tensor factorization , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[43]  Jalil Taghia,et al.  Sparsity level in a non-negative matrix factorization based speech strategy in cochlear implants , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[44]  Y. Ephraim,et al.  A Brief Survey of Speech Enhancement , 2003 .

[45]  Andrzej Cichocki,et al.  A Multiplicative Algorithm for Convolutive Non-Negative Matrix Factorization Based on Squared Euclidean Distance , 2009, IEEE Transactions on Signal Processing.

[46]  Martin Cooke,et al.  A glimpsing model of speech perception in noise. , 2006, The Journal of the Acoustical Society of America.

[47]  Blake S. Wilson,et al.  The Surprising Performance of Present-Day Cochlear Implants , 2007, IEEE Transactions on Biomedical Engineering.

[48]  M E Lutman,et al.  Speech identification under simulated hearing-aid frequency response characteristics in relation to sensitivity, frequency resolution, and temporal resolution. , 1986, The Journal of the Acoustical Society of America.

[49]  Y. Ephraim,et al.  A Brief Survey of Speech Enhancement 1 , 2018, Microelectronics.

[50]  Wenwu Wang,et al.  Squared Euclidean Distance Based Convolutive Non-Negative Matrix Factorization with Multiplicative Learning Rules For Audio Pattern Separation , 2007, 2007 IEEE International Symposium on Signal Processing and Information Technology.

[51]  A. Benjamin Premkumar,et al.  Particle Filtering Approaches for Multiple Acoustic Source Detection and 2-D Direction of Arrival Estimation Using a Single Acoustic Vector Sensor , 2012, IEEE Transactions on Signal Processing.

[52]  Philipos C Loizou,et al.  The intelligibility of speech with "holes" in the spectrum. , 2002, The Journal of the Acoustical Society of America.

[53]  Nancy Bertin,et al.  Nonnegative Matrix Factorization with the Itakura-Saito Divergence: With Application to Music Analysis , 2009, Neural Computation.

[54]  Stefan Bleeck,et al.  Relationship between speech recognition in noise and sparseness , 2012, International journal of audiology.

[55]  Vince D. Calhoun,et al.  Group learning using contrast NMF : Application to functional and structural MRI of schizophrenia , 2008, 2008 IEEE International Symposium on Circuits and Systems.