Inter-rater reliability of accessing the intelligibility of band-limited transformed speech using nonlinear frequency compression

Speech intelligibility verification through a listening test session by human raters was conducted to assess the intelligibility of band-limited speech. The band-limited speech was generated using nonlinear frequency compression via spectral envelope transformation to assist individuals with high-frequency hearing loss. However, addressing the issue on the reliability of the assessment method is important because of the human factors that cause variability in the intelligibility test results. This study investigated the use of Cohen's Kappa coefficient to measure the inter-rater reliability in the intelligibility test. The inter-rater reliability was found to be high when Cohen's Kappa coefficient was over 0.6. The result indicates that the human raters showed a high level of agreement in assessing the intelligibility of the band-limited speech.

[1]  C. Sheard,et al.  Reliability and agreement of ratings of ataxic dysarthric speech samples with varying intelligibility. , 1991, Journal of speech and hearing research.

[2]  Aini Hussain,et al.  Improved intelligibility of band limited speech using spectral envelope transformation , 2011 .

[3]  Nathaniel I. Durlach,et al.  Pitch invariant frequency lowering with nonuniform spectral compression , 1981, ICASSP.

[4]  B. Everitt,et al.  Statistical methods for rates and proportions , 1973 .

[5]  Aini Hussain,et al.  Improving Low Pass Filtered Speech Intelligibility Using Nonlinear Frequency Compression with Cepstrum and Spectral Envelope Transformation , 2011 .

[6]  A. Butcher,et al.  Development of a modified diagnostic classification system for voice disorders with inter-rater reliability study , 2007, Logopedics, phoniatrics, vocology.

[7]  Andrea Simpson,et al.  Frequency-Lowering Devices for Managing High-Frequency Hearing Loss: A Review , 2009, Trends in amplification.

[8]  A. Bowen,et al.  Apraxia of speech: how reliable are speech and language therapists' diagnoses? , 2007, Clinical rehabilitation.

[9]  Ying-Yee Kong,et al.  On the development of a frequency-lowering system that enhances place-of-articulation perception , 2012, Speech Commun..

[10]  J.C. Rutledge,et al.  Frequency lowering processing for listeners with significant hearing loss , 1999, ICECS'99. Proceedings of ICECS '99. 6th IEEE International Conference on Electronics, Circuits and Systems (Cat. No.99EX357).

[11]  Arie Ben-David,et al.  Comparison of classification accuracy using Cohen's Weighted Kappa , 2008, Expert Syst. Appl..

[12]  V J Samar,et al.  Criterion validity of speech intelligibility rating-scale procedures for the hearing-impaired population. , 1988, Journal of speech and hearing research.

[13]  Xianbo Xiao,et al.  Evaluation of frequency-lowering algorithms for intelligibility of Chinese speech in hearing-aid users , 2009 .

[14]  C M Reed,et al.  Intelligibility of frequency-lowered speech produced by a channel vocoder. , 1993, Journal of rehabilitation research and development.

[15]  Hugh J. McDermott.  A Technical Comparison of Digital Frequency-Lowering Algorithms Available in Two Current Hearing Aids , 2011, PloS one.

[16]  J. Fleiss,et al.  Statistical methods for rates and proportions , 1973 .