Swallow segmentation with artificial neural networks and multi-sensor fusion.

Swallow segmentation is a critical precursory step to the analysis of swallowing signal characteristics. In an effort to automatically segment swallows, we investigated artificial neural networks (ANN) with information from cervical dual-axis accelerometry, submental MMG, and nasal airflow. Our objectives were (1) to investigate the relationship between segmentation performance and the number of signal sources and (2) to identify the signals or signal combinations most useful for swallow segmentation. Signals were acquired from 17 healthy adults in both discrete and continuous swallowing tasks using five stimuli. Training and test feature vectors were constructed with variances from single or multiple signals, estimated within 200 ms moving windows with 50% overlap. Corresponding binary target labels (swallow or non-swallow) were derived by manual segmentation. A separate 3-layer ANN was trained for each participant-signal combination, and all possible signal combinations were investigated. As more signal sources were included, segmentation performance improved in terms of sensitivity, specificity, accuracy, and adjusted accuracy. The combination of all four signal sources achieved the highest mean accuracy and adjusted accuracy of 88.5% and 89.6%, respectively. A-P accelerometry proved to be the most discriminatory source, while the inclusion of MMG or nasal airflow resulted in the least performance improvement. These findings suggest that an ANN, multi-sensor fusion approach to segmentation is worthy of further investigation in swallowing studies.

[1]  N. P. Reddy,et al.  Measurements of acceleration during videofluorographic evaluation of dysphagic patients. , 2000, Medical engineering & physics.

[2]  A. Foundas,et al.  Swallowing Physiology of Sequential Straw Drinking , 2001, Dysphagia.

[3]  Paul Finn,et al.  Reliability and Validity of Cervical Auscultation: A Controlled Comparison Using Video fluoroscopy , 2004, Dysphagia.

[4]  M. Tarata.  Mechanomyography versus Electromyography, in monitoring the muscular fatigue , 2003, Biomedical engineering online.

[5]  Ren C. Luo,et al.  Multisensor integration and fusion in intelligent systems , 1989, IEEE Trans. Syst. Man Cybern..

[6]  T. Chau,et al.  Investigating the stationarity of paediatric aspiration signals , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[7]  Benoit M. Dawant,et al.  Neural-network-based segmentation of multi-modal medical images: a comparative and prospective study , 1993, IEEE Trans. Medical Imaging.

[8]  A. Jean.  Brain stem control of swallowing: neuronal network and cellular mechanisms. , 2001, Physiological reviews.

[9]  Bonnie Martin-Harris,et al.  Breathing and swallowing dynamics across the adult lifespan. , 2005, Archives of otolaryngology--head & neck surgery.

[10]  Jake K. Aggarwal,et al.  Experiments in combining intensity and range edge maps , 1983, Comput. Vis. Graph. Image Process..

[11]  Bernadette Dorizzi,et al.  ECG signal analysis through hidden Markov models , 2006, IEEE Transactions on Biomedical Engineering.

[12]  R Shaker,et al.  Coordination between respiration and swallowing: respiratory phase relationships and temporal integration. , 1994, Journal of applied physiology.

[13]  J. J. Chen,et al.  Classification ensembles for unbalanced class sizes in predictive toxicology , 2005, SAR and QSAR in environmental research.

[14]  Ioannis P. Vlahavas,et al.  Improving the Accuracy of Classifiers for the Prediction of Translation Initiation Sites in Genomic Sequences , 2005, Panhellenic Conference on Informatics.

[15]  Rangaraj M. Rangayyan,et al.  A Three-Channel Microcomputer System for Segmentation and Characterization of the Phonocardiogram , 1987, IEEE Transactions on Biomedical Engineering.

[16]  Abtin Tabaee,et al.  Patient‐Controlled Comparison of Flexible Endoscopic Evaluation of Swallowing With Sensory Testing (FEESST) and Videofluoroscopy , 2006, The Laryngoscope.

[17]  Arthur J. Miller,et al.  Book Reviews: The Neuroscientific Principles of Swallowing and Dysphagia , 1999 .

[18]  D. Cerenko,et al.  Timing of major events of pharyngeal swallowing. , 1988, Archives of otolaryngology--head & neck surgery.

[19]  S L Hamlet,et al.  Interpreting the Sounds of Swallowing: Fluid Flow through the Cricopharyngeus , 1990, The Annals of otology, rhinology, and laryngology.

[20]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[21]  Régine André-Obrecht,et al.  A new statistical approach for the automatic segmentation of continuous speech signals , 1988, IEEE Trans. Acoust. Speech Signal Process..

[22]  J. Jesberger,et al.  Assessment of Dysphagia with the Use of Pulse Oximetry , 1999, Dysphagia.

[23]  Mubarak Shah,et al.  Multi-sensor fusion: a perspective , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[24]  Michael A. Crary,et al.  Surface Electromyographic Characteristics of Swallowing in Dysphagia Secondary to Brainstem Stroke , 1997, Dysphagia.

[25]  Jan Cornelis,et al.  Segmentation of medical images , 1993, Image Vis. Comput..

[26]  Yoshiyasu Takefuji,et al.  Optimization neural networks for the segmentation of magnetic resonance images , 1992, IEEE Trans. Medical Imaging.

[27]  Eva van Leer,et al.  Bolus location associated with videofluoroscopic and respirodeglutometric events. , 2005, Journal of speech, language, and hearing research : JSLHR.

[28]  Tom Chau,et al.  A radial basis classifier for the automatic detection of aspiration in children with dysphagia , 2006, Journal of NeuroEngineering and Rehabilitation.

[29]  Richard O. Duda,et al.  Use of Range and Reflectance Data to Find Planar Surface Regions , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  T Chau,et al.  Time and time–frequency characterization of dual-axis swallowing accelerometry signals , 2008, Physiological measurement.

[31]  Tom Chau,et al.  Coupled microphone-accelerometer sensor pair for dynamic noise reduction in MMG signal recording , 2003 .

[32]  W. Selley,et al.  Respiratory patterns associated with swallowing: Part 2. Neurologically impaired dysphagic patients. , 1989, Age and ageing.

[33]  Ren C. Luo,et al.  Multisensor fusion and integration: approaches, applications, and future research directions , 2002 .

[34]  Tom Chau,et al.  A self-contained, mechanomyography-driven externally powered prosthesis. , 2005, Archives of physical medicine and rehabilitation.

[35]  Michele Nappi,et al.  Different methods to segment biomedical images , 1997, Pattern Recognit. Lett..

[36]  A. Perlman,et al.  Respiratory and Acoustic Signals Associated with Bolus Passage during Swallowing , 2000, Dysphagia.

[37]  J. Logemann,et al.  Evaluation and treatment of swallowing disorders , 1983 .

[38]  Robert J. Wood,et al.  Towards a 3g crawling robot through the integration of microrobot technologies , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[39]  Yi Lu,et al.  Robust neural learning from unbalanced data samples , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[40]  Amitava Das,et al.  Hybrid fuzzy logic committee neural networks for recognition of swallow acceleration signals , 2001, Comput. Methods Programs Biomed..

[41]  Jason Williams,et al.  Emotion Recognition Using Bio-sensors: First Steps towards an Automatic System , 2004, ADS.

[42]  David G. Stork,et al.  Pattern Classification , 1973 .

[43]  C. Orizio.  Muscle sound: bases for the introduction of a mechanomyographic signal in muscle studies. , 1993, Critical reviews in biomedical engineering.