The need for external validation in machine olfaction: emphasis on health-related applications

AbstractOver the last two decades, electronic nose research has produced thousands of research works. Many of them were describing the ability of the e-nose technology to solve diverse applications in domains ranging from food technology to safety, security, or health. It is, in fact, in the biomedical field where e-nose technology is finding a research niche in the last years. Although few success stories exist, most described applications never found the road to industrial or clinical exploitation. Most described methodologies were not reliable and were plagued by numerous problems that prevented practical application beyond the lab. This work emphasizes the need of external validation in machine olfaction. I describe some statistical and methodological pitfalls of the e-nose practice and I give some best practice recommendations for researchers in the field. FigureState-of-the-art electronic noses feature digitally embedded multivariate predictive system: either pattern recognition systems or quantitative predictors

[1]  Dana Ron,et al.  Algorithmic Stability and Sanity-Check Bounds for Leave-one-Out Cross-Validation , 1997, COLT.

[2]  Henri Knobloch,et al.  Methodological variation in headspace analysis of liquid samples using electronic nose , 2009 .

[3]  C. Serra,et al.  Quality assurance of pharmaceuticals: a compendium of guidelines and related materials , 2007 .

[4]  E. Petricoin,et al.  Use of proteomic patterns in serum to identify ovarian Cancer , 2002 .

[5]  S. Goodman Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy , 1999, Annals of Internal Medicine.

[6]  Niki Fens,et al.  Exhaled breath profiling enables discrimination of chronic obstructive pulmonary disease and asthma. , 2009, American journal of respiratory and critical care medicine.

[7]  H. T. Nagle,et al.  Transient response analysis of an electronic nose using multi-exponential models , 1999 .

[8]  Wolfgang Koch,et al.  Breath profiles by electronic nose correlate with systemic markers but not ozone response. , 2011, Respiratory medicine.

[9]  Véronique Bellon-Maurel,et al.  Robustness of models developed by multivariate calibration. Part I: The assessment of robustness , 2004 .

[10]  James A. Covington,et al.  The Detection of Patients at Risk of Gastrointestinal Toxicity during Pelvic Radiotherapy by Electronic Nose and FAIMS: A Pilot Study , 2012, Sensors.

[11]  Lennart Eriksson,et al.  Model validation by permutation tests: Applications to variable selection , 1996 .

[12]  Takamichi Nakamoto,et al.  Study on the odor classification in dynamical concentration robust against humidity and temperature changes , 2008 .

[13]  J. Roger,et al.  Robustness of models developed by multivariate calibration. Part II: The influence of pre-processing methods , 2005 .

[14]  Deborah H Yates,et al.  A breath test for malignant mesothelioma using an electronic nose , 2011, European Respiratory Journal.

[15]  Ricardo Gutierrez-Osuna,et al.  Pattern analysis for machine olfaction: a review , 2002 .

[16]  Pere Caminal,et al.  Drift Compensation of Gas Sensor Array Data by Common Principal Component Analysis , 2010 .

[17]  J. Samitier,et al.  A novel time-domain method to analyse multicomponent exponential transients , 1995 .

[18]  E. Llobet,et al.  Analysis of the conductance transient in thick-film tin oxide gas sensors , 1996 .

[19]  José Luis Valera,et al.  Use of the electronic nose for diagnosing respiratory diseases. , 2012 .

[20]  Rasmus Bro,et al.  Feasibility of serodiagnosis of ovarian cancer by mass spectrometry. , 2009, Analytical chemistry.

[21]  Steven Goodman Toward Evidence-Based Medical Statistics. 2: The Bayes Factor , 1999, Annals of Internal Medicine.

[22]  Mia Hubert,et al.  Robustness and Outlier Detection in Chemometrics , 2006 .

[23]  Giovanni Squillero,et al.  Increasing pattern recognition accuracy for chemical sensing by evolutionary based drift compensation , 2011, Pattern Recognit. Lett..

[24]  R. Gutierrez-Osuna Drift Reduction For Metal-Oxide Sensor Arrays Using Canonical Correlation Regression And Partial Least Squares , 2000 .

[25]  P. Filzmoser,et al.  Repeated double cross validation , 2009 .

[26]  E. Martinelli,et al.  Lung cancer identification by the analysis of breath by means of an array of non-selective gas sensors. , 2003, Biosensors & bioelectronics.

[27]  O. Ruiz,et al.  A new method to analyse signal transients in chemical sensors , 1994 .

[28]  Krishna C Persaud Medical Applications of Odor-Sensing Devices , 2005, The international journal of lower extremity wounds.

[29]  Zsofia Lazar,et al.  Electronic Nose Breathprints Are Independent of Acute Changes in Airway Caliber in Asthma , 2010, Sensors.

[30]  Giorgio Pennazza,et al.  Application of a quartz microbalance based gas sensor array for the study of halitosis , 2008, Journal of breath research.

[31]  Dominique Bonvin,et al.  Framework for explicit drift correction in multivariate calibration models , 2010 .

[32]  Paolo Montuschi,et al.  The Electronic Nose in Respiratory Medicine , 2012, Respiration.

[33]  Antonio Pardo Martínez,et al.  Gas identification with tin oxide sensor array and self-organizing maps: adaptive correction of sensor drifts , 1998 .

[34]  Muhammad F. Walji,et al.  Human-centered design of a distributed knowledge management system , 2005, J. Biomed. Informatics.

[35]  T. Greulich,et al.  Detection of obstructive sleep apnoea by an electronic nose , 2012, European Respiratory Journal.

[36]  E. K. Kemsley,et al.  THE USE AND MISUSE OF CHEMOMETRICS FOR TREATING CLASSIFICATION PROBLEMS , 1997 .

[37]  Anne-Claude Romain In situ measurement of olfactive pollution with inorganic semiconductors : Limitations due to humidity and temperature influence , 1997 .

[38]  Paul Geladi,et al.  Principles of Proper Validation: use and abuse of re‐sampling for validation , 2010 .

[39]  K. R. Kashwan,et al.  Robust electronic-nose system with temperature and humidity drift compensation for tea and spice flavour discrimination , 2005, 2005 Asian Conference on Sensors and the International Conference on New Techniques in Pharmaceutical and Biomedical Research.

[40]  J. Brooks Why most published research findings are false: Ioannidis JP, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece , 2008 .

[41]  Y. Heyden,et al.  Guidance for robustness/ruggedness tests in method validation. , 2001, Journal of pharmaceutical and biomedical analysis.

[42]  Peter J Sterk,et al.  An electronic nose in the discrimination of patients with asthma and controls. , 2007, The Journal of allergy and clinical immunology.

[43]  R. Beccherelli,et al.  Large-Scale Chemical Sensor Array Testing Biological Olfaction Concepts , 2012, IEEE Sensors Journal.

[44]  Raffaele Di Fuccio,et al.  An adaptive classification model based on the Artificial Immune System for chemical sensor drift mitigation , 2013 .

[45]  J. Habbema,et al.  Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. , 2001, Journal of clinical epidemiology.

[46]  Douglas B. Kell,et al.  Statistical strategies for avoiding false discoveries in metabolomics and related experiments , 2007, Metabolomics.

[47]  John N. Lygouras,et al.  Artificial Odor Discrimination System Using Electronic Nose and Neural Networks for the Identification of Urinary Tract Infection , 2008, IEEE Transactions on Information Technology in Biomedicine.

[48]  B. K. Lavine,et al.  Validation of Classifiers , 2009 .

[49]  L. T. Tanoue Detection of Lung Cancer by Sensor Array Analyses of Exhaled Breath , 2007 .

[50]  Fabrizio Davide,et al.  Different strategies for the identification of gas sensing systems , 1996 .

[51]  Alexandre Perera,et al.  Drift compensation of gas sensor array data by Orthogonal Signal Correction , 2010 .

[52]  Lucila Ohno-Machado,et al.  The use of receiver operating characteristic curves in biomedical informatics , 2005, J. Biomed. Informatics.

[53]  D. Ransohoff Lessons from controversy: ovarian cancer screening and serum proteomics. , 2005, Journal of the National Cancer Institute.

[54]  Gemma C. Garriga,et al.  Permutation Tests for Studying Classifier Performance , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[55]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[56]  F H Krouwels,et al.  Exhaled air molecular profiling in relation to inflammatory subtype and activity in COPD , 2011, European Respiratory Journal.

[57]  Age K. Smilde,et al.  UvA-DARE ( Digital Academic Repository ) Assessment of PLSDA cross validation , 2008 .

[58]  Anne-Claude Romain,et al.  Detection of diverse mould species growing on building materials by gas sensor arrays and pattern recognition , 2006 .

[59]  J. Haugen,et al.  Recalibration of a gas-sensor array system related to sensor replacement , 2004 .

[60]  A. Gutierrez-Galvez,et al.  Signal and Data Processing for Machine Olfaction and Chemical Sensing: A Review , 2012, IEEE Sensors Journal.

[61]  A. Romain,et al.  Fuzzy K-NN applied to moulds detection , 2005 .

[62]  Douglas B. Kell,et al.  Novel biomarkers for pre-eclampsia detected using metabolomics and machine learning , 2005, Metabolomics.

[63]  Anne-Claude Romain,et al.  Evaluation of an electronic nose for the early detection of organic overload of anaerobic digesters , 2012, Bioprocess and Biosystems Engineering.

[64]  J. Cornfield Sequential Trials, Sequential Analysis and the Likelihood Principle , 1966 .

[65]  T. Greulich,et al.  Discrimination between COPD patients with and without alpha 1‐antitrypsin deficiency using an electronic nose , 2011, Respirology.

[66]  S. Goodman,et al.  Toward Evidence-Based Medical Statistics. 2: The Bayes Factor , 1999, Annals of Internal Medicine.

[67]  M. Gardner,et al.  Confidence intervals rather than P values: estimation rather than hypothesis testing. , 1986, British medical journal.

[68]  Age K. Smilde,et al.  Assessing the performance of statistical validation tools for megavariate metabolomics data , 2006, Metabolomics.

[69]  M. Sjöström,et al.  Drift correction for gas sensors using multivariate methods , 2000 .