Non-invasive monitoring of chewing and swallowing for objective quantification of ingestive behavior.

A methodology of studying of ingestive behavior by non-invasive monitoring of swallowing (deglutition) and chewing (mastication) has been developed. The target application for the developed methodology is to study the behavioral patterns of food consumption and producing volumetric and weight estimates of energy intake. Monitoring is non-invasive based on detecting swallowing by a sound sensor located over laryngopharynx or by a bone-conduction microphone and detecting chewing through a below-the-ear strain sensor. Proposed sensors may be implemented in a wearable monitoring device, thus enabling monitoring of ingestive behavior in free-living individuals. In this paper, the goals in the development of this methodology are two-fold. First, a system comprising sensors, related hardware and software for multi-modal data capture is designed for data collection in a controlled environment. Second, a protocol is developed for manual scoring of chewing and swallowing for use as a gold standard. The multi-modal data capture was tested by measuring chewing and swallowing in 21 volunteers during periods of food intake and quiet sitting (no food intake). Video footage and sensor signals were manually scored by trained raters. Inter-rater reliability study for three raters conducted on the sample set of five subjects resulted in high average intra-class correlation coefficients of 0.996 for bites, 0.988 for chews and 0.98 for swallows. The collected sensor signals and the resulting manual scores will be used in future research as a gold standard for further assessment of sensor design, development of automatic pattern recognition routines and study of the relationship between swallowing/chewing and ingestive behavior.

[1]  J. Baerdemaeker,et al.  Principal component analysis of chewing sounds to detect differences in apple crispness , 2000 .

[2]  Michael R. Neuman,et al.  Limited receptive area neural classifier for recognition of swallowing sounds using short-time Fourier transform , 2007, 2007 International Joint Conference on Neural Networks.

[3]  Jacob Cohen,et al.  The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability , 1973 .

[4]  S. Hamlet,et al.  Stethoscope acoustics and cervical auscultation of swallowing , 2004, Dysphagia.

[5]  Zata M Vickers,et al.  RELATIONSHIPS BETWEEN SENSORY CRISPNESS AND OTHER SENSORY AND INSTRUMENTAL PARAMETERS , 1980 .

[6]  J. Heinz,et al.  Cervical Auscultation Of Suckle Feeding In Newborn Infants , 1990, Developmental medicine and child neurology.

[7]  M. Wong,et al.  Epidemiological assessment of diet: a comparison of a 7-day diary with a food frequency questionnaire using urinary markers of nitrogen, potassium and sodium. , 2001, International journal of epidemiology.

[8]  Julian F. V. Vincent,et al.  The quantification of crispness , 1998 .

[9]  A. Szczesniak,et al.  Texture : is it still an overlooked food attribute ? texture has gained a new importance and desirability for today's consumer , 1990 .

[10]  Herbert L. Meiselman,et al.  Methodology and theory in human eating research , 1992, Appetite.

[11]  V. Stanford,et al.  Validity of self-reported energy intake in lean and obese young women, using two nutrient databases, compared with total energy expenditure assessed by doubly labeled water , 2001, European Journal of Clinical Nutrition.

[12]  Sheena Reilly,et al.  NON-INVASIVE MONITORING OF REFLEXIVE SWALLOWING , 1997 .

[13]  Cumhur Ertekin,et al.  Clinical and electrophysiological evaluation of dysphagia in myasthenia gravis , 1998, Journal of neurology, neurosurgery, and psychiatry.

[14]  Paul Lukowicz,et al.  Analysis of Chewing Sounds for Dietary Monitoring , 2005, UbiComp.

[15]  Cumhur Ertekin,et al.  An electronic device measuring the frequency of spontaneous swallowing: Digital Phagometer , 2004, Dysphagia.

[16]  B. Rolls,et al.  Increasing the portion size of a packaged snack increases energy intake in men and women , 2004, Appetite.

[17]  D A Schoeller,et al.  Measurement of energy expenditure in free-living humans by using doubly labeled water. , 1988, The Journal of nutrition.

[18]  W. E. Lee,et al.  Analysis of food crushing sounds during mastication : total sound level studies , 1990 .

[19]  C. Champagne,et al.  Energy intake and energy expenditure: a controlled study comparing dietitians and non-dietitians. , 2002, Journal of the American Dietetic Association.

[20]  A. Prentice,et al.  Fast foods, energy density and obesity: a possible mechanistic link , 2003, Obesity reviews : an official journal of the International Association for the Study of Obesity.

[21]  Gerhard Tröster,et al.  Probabilistic parsing of dietary activity events , 2007, BSN.

[22]  Karim Allaf,et al.  Characterization of brittle food products : Application of the acoustical emission method , 1996 .

[23]  Paul Lukowicz,et al.  SoundButton: design of a low power wearable audio classification system , 2003, Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings..

[24]  Alex Mihailidis,et al.  The use of computer vision in an intelligent environment to support aging-in-place, safety, and independence in the home , 2004, IEEE Transactions on Information Technology in Biomedicine.

[25]  F Bellisle,et al.  Chewing and swallowing as indices of the stimulation to eat during meals in humans: effects revealed by the edogram method and video recordings , 2000, Neuroscience & Biobehavioral Reviews.

[26]  Albrecht Schmidt,et al.  Mediacups: experience with design and use of computer-augmented everyday artefacts , 2001, Comput. Networks.

[27]  B. Johnston,et al.  Does age or sex affect pharyngeal swallowing? , 1998, Clinical otolaryngology and allied sciences.

[28]  P. R. Murgatroyd,et al.  Metabolism or appetite: questions of energy balance with particular reference to obesity , 1989 .

[29]  Zata M Vickers,et al.  Instrumental Acoustical Measures of Crispness in Foods. , 1985 .

[30]  Christopher R Carpenter,et al.  Kappa statistic , 2005, Canadian Medical Association Journal.

[31]  J. Heinz,et al.  CORRELATION OF CERVICAL AUSCULTATION WITH PHYSIOLOGICAL RECORDING DURING SUCKLE‐FEEDING. IN NEWBORN INFANTS , 1995, Developmental medicine and child neurology.

[32]  P. Lillford,et al.  THE PERCEPTION OF FOOD TEXTURE ‐ THE PHILOSOPHY OF THE BREAKDOWN PATH , 1988 .

[33]  Paul Lukowicz,et al.  Gesture spotting with body-worn inertial sensors to detect user activities , 2008, Pattern Recognit..

[34]  Scott R. Youmans Increasing the Objectivity of the Clinical Dysphagia Evaluation , 2003 .

[35]  M. Clark,et al.  Interindividual Variation in Posture Allocation: Possible Role in Human Obesity , 2005, Science.

[36]  W Mertz,et al.  What are people really eating? The relation between energy intake derived from estimated diet records and intake determined to maintain body weight. , 1991, The American journal of clinical nutrition.

[37]  H S Brenman,et al.  Sound as a diagnostic aid in the detection of occlusal discrepancies. , 1966, The Penn dental journal.

[38]  Context-Aware Computing,et al.  Inferring Activities from Interactions with Objects , 2004 .

[39]  I. Aydogdu,et al.  Oropharyngeal swallowing in craniocervical dystonia , 2002, Journal of neurology, neurosurgery, and psychiatry.

[40]  E Stellar,et al.  Chews and swallows and the microstructure of eating. , 1985, The American journal of clinical nutrition.

[41]  C. C. Johnson,et al.  Intra- and inter-rater agreement in the assessment of occupational exposure to metals. , 1998, International journal of epidemiology.

[42]  M. Livingstone,et al.  Markers of the validity of reported energy intake. , 2003, The Journal of nutrition.

[43]  Arthur D. Fisk,et al.  Aware technologies for aging in place: understanding user needs and attitudes , 2004, IEEE Pervasive Computing.

[44]  Stan Z. Li,et al.  Content-based audio classification and retrieval using the nearest feature line method , 2000, IEEE Trans. Speech Audio Process..

[45]  Krishan K Kapur,et al.  FREQUENCY SPECTROGRAPHIC ANALYSIS OF BONE CONDUCTED CHEWING SOUNDS IN PERSONS WITH NATURAL AND ARTIFICIAL DENTITIONS. , 1971, Journal of texture studies.

[46]  Ishwar K. Sethi,et al.  Classification of general audio data for content-based retrieval , 2001, Pattern Recognit. Lett..

[47]  Michael R. Neuman,et al.  Infant polysomnography: Reliability , 1997 .

[48]  J. Fleiss Statistical methods for rates and proportions , 1974 .

[49]  Gerhard Tröster,et al.  Methods for Detection and Classification of Normal Swallowing from Muscle Activation and Sound , 2006, 2006 Pervasive Health Conference and Workshops.

[50]  F Bellisle,et al.  Why should we study human food intake behaviour? , 2003, Nutrition, metabolism, and cardiovascular diseases : NMCD.

[51]  Kay Römer,et al.  Smart identification frameworks for ubiquitous computing applications , 2003, Proceedings of the First IEEE International Conference on Pervasive Computing and Communications, 2003. (PerCom 2003)..

[52]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[53]  B. Rolls,et al.  Portion size of food affects energy intake in normal-weight and overweight men and women. , 2002, The American journal of clinical nutrition.

[54]  J F Prinz,et al.  Computer aided gnathosonic analysis: distinguishing between single and multiple tooth impact sounds. , 2000, Journal of oral rehabilitation.

[55]  S. Widmalm,et al.  Time frequency distributions of TMJ sounds. , 2008, Journal of oral rehabilitation.

[56]  Vesa T. Peltonen,et al.  Computational auditory scene recognition , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[57]  Gerhard Tröster,et al.  Detection of eating and drinking arm gestures using inertial body-worn sensors , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[58]  J. D. Castro,et al.  Methodology, Correlational Analysis, and Interpretation of Diet Diary Records of the Food and Fluid Intake of Free-living Humans , 1994, Appetite.

[59]  C. Lear,et al.  THE FREQUENCY OF DEGLUTITION IN MAN. , 1965, Archives of oral biology.

[60]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[61]  Tom Fawcett,et al.  Analysis and Visualization of Classifier Performance: Comparison under Imprecise Class and Cost Distributions , 1997, KDD.

[62]  Zata M Vickers,et al.  Sensory, Acoustical, and Force-Deformation Measurements of Potato Chip Crispness , 1987 .

[63]  Eliot Stellar,et al.  Responses of lean and obese subjects to preloads, deprivation, and palatability , 1989, Appetite.

[64]  O. Makeyev,et al.  Limited receptive area neural classifier for recognition of swallowing sounds using continuous wavelet transform , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[65]  J. Siegel,et al.  Sleep , 2007, Neuromolecular medicine.

[66]  Gerhard Tröster,et al.  Recognition of dietary activity events using on-body sensors , 2008, Artif. Intell. Medicine.

[67]  Neil Mann,et al.  Origins and evolution of the Western diet: health implications for the 21st century. , 2005, The American journal of clinical nutrition.

[68]  C. Dacremont,et al.  CONTRIBUTION OF AIR-AND BONE-CONDUCTION TO THE CREATION OF SOUNDS PERCEIVED DURING SENSORY EVALUATION OF FOODS , 1991 .

[69]  Birger Drake,et al.  Food Crushing Sounds. An Introductory Studya,b , 1963 .

[70]  Marjorie P. Penfield,et al.  SPEECH ANALYSIS TECHNIQUES: A POTENTIAL MODEL FOR THE STUDY OF MASTICATION SOUNDS , 1992 .

[71]  David M. Watt,et al.  Gnathosonics—A study of sounds produced by the masticatory mechanism , 1966 .

[72]  Zata M Vickers,et al.  Relationships of Chewing Sounds to Judgments of Crispness, Crunchiness and Hardness , 1982 .

[73]  S A Jebb,et al.  Assessment of human energy balance. , 1997, The Journal of endocrinology.

[74]  Albrecht Schmidt,et al.  Multi-Sensor Context-Awareness in Mobile Devices and Smart Artifacts , 2002, Mob. Networks Appl..

[75]  P. Lucas,et al.  Food Properties that Influence Neuromuscular Activity During Human Mastication , 1998, Journal of dental research.

[76]  K W Tyson Monitoring the state of the occlusion--gnathosonics can be reliable. , 1998, Journal of oral rehabilitation.

[77]  J. Robbins,et al.  Comparison of effortful and noneffortful swallows in healthy middle-aged and older adults. , 2001, Archives of physical medicine and rehabilitation.

[78]  Gerhard Tröster,et al.  Automatic Identification of Temporal Sequences in Chewing Sounds , 2007, 2007 IEEE International Conference on Bioinformatics and Biomedicine (BIBM 2007).

[79]  M. Livingstone,et al.  Accuracy of weighed dietary records in studies of diet and health. , 1990, BMJ.

[80]  Gerhard Tröster,et al.  On-Body Sensing Solutions for Automatic Dietary Monitoring , 2009, IEEE Pervasive Computing.

[81]  A. Crompton,et al.  Coordination of mastication and swallowing , 1992, Dysphagia.

[82]  Michael R Neuman,et al.  Agreement among Raters in Assessment of Physiologic Waveforms Recorded by a Cardiorespiratory Monitor for Home Use , 1998, Pediatric Research.

[83]  A. Perlman,et al.  Deglutition and Its Disorders: Anatomy, Physiology, Clinical Diagnosis and Management , 1996 .