A Stereo Music Preprocessing Scheme for Cochlear Implant Users

Objective: Listening to music is still one of the more challenging aspects of using a cochlear implant (CI) for most users. Simple musical structures, a clear rhythm/beat, and lyrics that are easy to follow are among the top factors contributing to music appreciation for CI users. Modifying the audio mix of complex music potentially improves music enjoyment in CI users. Methods: A stereo music preprocessing scheme is described in which vocals, drums, and bass are emphasized based on the representation of the harmonic and the percussive components in the input spectrogram, combined with the spatial allocation of instruments in typical stereo recordings. The scheme is assessed with postlingually deafened CI subjects (N = 7) using pop/rock music excerpts with different complexity levels. Results: The scheme is capable of modifying relative instrument level settings, with the aim of improving music appreciation in CI users, and allows individual preference adjustments. The assessment with CI subjects confirms the preference for more emphasis on vocals, drums, and bass as offered by the preprocessing scheme, especially for songs with higher complexity. Conclusion: The stereo music preprocessing scheme has the potential to improve music enjoyment in CI users by modifying the audio mix in widespread (stereo) music recordings. Significance: Since music enjoyment in CI users is generally poor, this scheme can assist the music listening experience of CI users as a training or rehabilitation tool.

[1]  Jay T. Rubinstein,et al.  Clinical Assessment of Music Perception in Cochlear Implant Listeners , 2008, Otology & neurotology : official publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology.

[2]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[3]  Tuomas Virtanen,et al.  Sound Source Separation in Monaural Music Signals , 2006 .

[4]  Philipos C. Loizou,et al.  Mimicking the human ear , 1998, IEEE Signal Process. Mag..

[5]  Emmanuel Vincent,et al.  A General Flexible Framework for the Handling of Prior Information in Audio Source Separation , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[6]  Tae-Hoon Kim,et al.  A Real Time Singing Voice Removal System Using DSP and Multichannel Audio Interface , 2012 .

[7]  Hugh J. McDermott.  Music Perception with Cochlear Implants: A Review , 2004, Trends in amplification.

[8]  Tuomas Virtanen,et al.  Separation of drums from polyphonic music using non-negative matrix factorization and support vector machine , 2005, 2005 13th European Signal Processing Conference.

[9]  Marc Moonen,et al.  Speech Understanding Performance of Cochlear Implant Subjects Using Time–Frequency Masking-Based Noise Reduction , 2012, IEEE Transactions on Biomedical Engineering.

[10]  Xavier Serra,et al.  Musical Sound Modeling with Sinusoids plus Noise , 1997 .

[11]  Mark D. Plumbley,et al.  Polyphonic transcription by non-negative sparse coding of power spectra , 2004, ISMIR.

[12]  Astrid van Wieringen,et al.  LIST and LINT: Sentences and numbers for quantifying speech understanding in severely impaired listeners for Flanders and the Netherlands , 2008, International journal of audiology.

[13]  Antoine Liutkus,et al.  Kernel spectrogram models for source separation , 2014, 2014 4th Joint Workshop on Hands-free Speech Communication and Microphone Arrays (HSCMA).

[14]  Derry Fitzgerald,et al.  Harmonic/Percussive Separation Using Median Filtering , 2010 .

[15]  Hirokazu Kameoka,et al.  Separation of a monaural audio signal into harmonic/percussive components by complementary diffusion on spectrogram , 2008, 2008 16th European Signal Processing Conference.

[16]  Hirokazu Kameoka,et al.  Comparative evaluations of various harmonic/percussive sound separation algorithms based on anisotropic continuity of spectrogram , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[17]  P. Loizou.  Introduction to cochlear implants. , 1999, IEEE engineering in medicine and biology magazine : the quarterly magazine of the Engineering in Medicine & Biology Society.

[18]  J. Knutson,et al.  Musical backgrounds, listening habits, and aesthetic enjoyment of adult cochlear implant recipients. , 2000, Journal of the American Academy of Audiology.

[19]  Jong Ho Won,et al.  Development and Validation of the University of Washington Clinical Assessment of Music Perception Test , 2009, Ear and hearing.

[20]  Julius O. Smith,et al.  PARSHL: An Analysis/Synthesis Program for Non-Harmonic Sounds Based on a Sinusoidal Representation , 1987, ICMC.

[21]  Shigeki Sagayama,et al.  Melody line estimation in homophonic music audio signals based on temporal-variability of melodic source , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[22]  J. Knutson,et al.  The effects of familiarity and complexity on appraisal of complex songs by cochlear implant recipients and normal hearing adults. , 2003, Journal of music therapy.

[23]  Lydia Timm.  Music perception of cochlear implant users , 2012 .

[24]  Tuomas Virtanen,et al.  Sound Source Separation Using Sparse Coding with Temporal Continuity Objective , 2003, ICMC.

[25]  M. Moonen,et al.  Music mixing preferences of cochlear implant recipients: A pilot study , 2014, International journal of audiology.

[26]  Sherif A. Omran,et al.  Pitch Ranking, Melody Contour and Instrument Recognition Tests Using Two Semitone Frequency Maps for Nucleus Cochlear Implants , 2010, EURASIP J. Audio Speech Music. Process..

[27]  Xavier Serra,et al.  A system for sound analysis/transformation/synthesis based on a deterministic plus stochastic decomposition , 1989 .

[28]  George York.  Teaching Real Time Dsp Applications (Voice Removal) With The C6711 Dsk And Matlab , 2004 .

[29]  Marc Moonen,et al.  A harmonic/percussive sound separation based music pre-processing scheme for cochlear implant users , 2013, 21st European Signal Processing Conference (EUSIPCO 2013).

[30]  Fan-Gang Zeng,et al.  Music Perception with Temporal Cues in Acoustic and Electric Hearing , 2004, Ear and hearing.

[31]  Christian Uhle,et al.  EXTRACTION OF DRUM TRACKS FROM POLYPHONIC MUSIC USING INDEPENDENT SUBSPACE ANALYSIS , 2003 .