Software for automatic analysis of image and sound data simultaneously acquired from high-speed videoendocopy

High-speed digital videoendoscopy system is emerging as a new clinical tool for voice assessment. The system can acquire images of the vibrating vocal folds with simultaneous recording of voice data from the patient. The laryngeal image-based analysis has been proven valuable for objective and quantitative assessment of voice kinematics in health and disease, and meanwhile, acoustic analysis of voice data could assist in the study of phonatory characteristics and reveal useful information related to laryngeal pathophysiology. Contrast to the hardware acquisition systems, the development of effective software for handling such massive visual/sound data has lagged behind. In this paper, a software system is designed to process the laryngeal image sequences and perform image-based analyses as well as acoustic analyses. Our software contains following modules: (1) Import and view Module - to read AVI video data and sound data (wave file), edit/compile and save selected data, make image montages using DirectShow technology and display the acoustic waveform using DirectSound technology; (2) Image Process Module – to perform frame-by-frame image segmentation to delineate the glottis, to extract the GAW and bilateral vocal fold displacements; (3) Image Analysis Module – to adopt Nyquist plot displays that involves the Hilbert transform based analysis of GAW, and to provide instantaneous frequency and amplitude distributions; (4) Acoustic Analysis Module – to perform Fast Fourier Transform (FFT) and Spectrogram analyses of the imported sound data, to display the plot of the sound data and provide instantaneous frequency and amplitude distributions and Nyqiust plot and (5) Dual GAW and sound wave display module. Upon rigorous testing of this software using clinical data samples we demonstrate the applications of the software to the study of dynamic characteristics of the glottis, which may correlate with voice quality and health condition.