Mutual Information Input Selector and Probabilistic Machine Learning Utilisation for Air Pollution Proxies

An air pollutant proxy is a mathematical model that estimates an unobserved air pollutant using other measured variables. The proxy is advantageous to fill missing data in a research campaign or to substitute a real measurement for minimising the cost as well as the operators involved (i.e., virtual sensor). In this paper, we present a generic concept of pollutant proxy development based on an optimised data-driven approach. We propose a mutual information concept to determine the interdependence of different variables and thus select the most correlated inputs. The most relevant variables are selected to be the best proxy inputs, where several metrics and data loss are also involved for guidance. The input selection method determines the used data for training pollutant proxies based on a probabilistic machine learning method. In particular, we use a Bayesian neural network that naturally prevents overfitting and provides confidence intervals around its output prediction. In this way, the prediction uncertainty could be assessed and evaluated. In order to demonstrate the effectiveness of our approach, we test it on an extensive air pollution database to estimate ozone concentration.

[1]  Osman Taylan,et al.  Modelling and analysis of ozone concentration by artificial intelligent techniques for estimating air quality , 2017 .

[2]  Michael I. Jordan,et al.  Variational Bayesian Inference with Stochastic Search , 2012, ICML.

[3]  Zoubin Ghahramani,et al.  Probabilistic machine learning and artificial intelligence , 2015, Nature.

[4]  Xiaobo Zhang,et al.  Developing an early-warning system for air quality prediction and assessment of cities in China , 2017, Expert Syst. Appl..

[5]  Yuzhen Ye,et al.  Functional association prediction by community profiling. , 2017, Methods.

[6]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[7]  Robert M. Pap,et al.  Handbook of neural computing applications , 1990 .

[8]  G. Hidy.  Ozone process insights from field experiments – part I: overview , 2000 .

[9]  L. Spinelle,et al.  Field calibration of a cluster of low-cost commercially available sensors for air quality monitoring. Part B: NO, CO and CO2 , 2017 .

[10]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[11]  L. Morawska,et al.  The rise of low-cost sensing for managing air pollution in cities. , 2015, Environment international.

[12]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[13]  L. Spinelle,et al.  Sensors and Actuators B: Chemical Field calibration of a cluster of low-cost available sensors for air quality monitoring. Part A: Ozone and nitrogen dioxide (cid:2) , 2022 .

[14]  Wilhelm Kuttler,et al.  Long-term analysis of NO, NO2 and O3 concentrations in North Rhine-Westphalia, Germany , 2012 .

[15]  Carlo Ratti,et al.  End-user perspective of low-cost sensors for outdoor air pollution monitoring. , 2017, The Science of the total environment.

[16]  Chee-Yee Chong,et al.  Sensor networks: evolution, opportunities, and challenges , 2003, Proc. IEEE.

[17]  Zhan Zhao,et al.  High Accuracy and Miniature 2-D Wind Sensor for Boundary Layer Meteorological Observation † , 2019, Sensors.

[18]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[19]  A. S. Abdelmaksoud,et al.  Seasonal and diurnal variations of BTEX and their potential for ozone formation in the urban background atmosphere of the coastal city Jeddah, Saudi Arabia , 2014, Air Quality, Atmosphere & Health.

[20]  P. Stevens,et al.  Development of an instrument for direct ozone production rate measurements: measurement reliability and current limitations , 2017 .

[21]  Pericles A. Mitkas,et al.  Sparse episode identification in environmental datasets: The case of air quality assessment , 2011, Expert Syst. Appl..

[22]  M. Sofiev,et al.  A Construction and Evaluation of Eulerian Dynamic Core for the Air Quality and Emergency Modelling System SILAM , 2008 .

[23]  L. Modig,et al.  The spatial variation of O3, NO, NO2 and NOx and the relation between them in two Swedish cities , 2017, Environmental Monitoring and Assessment.

[24]  Hayit Greenspan,et al.  Task-Driven Dictionary Learning Based on Mutual Information for Medical Image Classification , 2017, IEEE Transactions on Biomedical Engineering.

[25]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[26]  Álvaro Herrero,et al.  Neural Models for Imputation of Missing Ozone Data in Air-Quality Datasets , 2018, Complex..

[27]  Jicai Ning,et al.  Artificial neural network model for ozone concentration estimation and Monte Carlo analysis , 2018, Atmospheric Environment.

[28]  Washington Leite Junger,et al.  Imputation of missing data in time series for air pollutants , 2015 .

[29]  Anondo Mukherjee,et al.  Assessing the Utility of Low-Cost Particulate Matter Sensors over a 12-Week Period in the Cuyama Valley of California , 2017, Sensors.

[30]  Ryan P. Adams,et al.  Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks , 2015, ICML.

[31]  David M. Blei,et al.  Variational Inference: A Review for Statisticians , 2016, ArXiv.

[32]  Harri Niska,et al.  Methods for imputation of missing values in air quality data sets , 2004 .

[33]  M. I. Mead,et al.  Use of networks of low cost air quality sensors to quantify air quality in urban settings , 2018, Atmospheric Environment.

[34]  M. A. Zaidan,et al.  Predicting atmospheric particle formation days by Bayesian classification of the time series features , 2018 .

[35]  Jean-Pierre Chevallet,et al.  A Comparison of Deep Learning Based Query Expansion with Pseudo-Relevance Feedback and Mutual Information , 2016, ECIR.

[36]  A. S. Abdelmaksoud,et al.  Particulate Matter and Number Concentrations of Particles Larger than 0.25 µm in the Urban Atmosphere of Jeddah, Saudi Arabia , 2014 .

[37]  Fei Wu,et al.  Weekly patterns of México City's surface concentrations of CO, NO x , PM 10 and O 3 during 1986–2007 , 2008 .

[38]  Manuel Aleixandre,et al.  Review of Small Commercial Sensors for Indicative Monitoring of Ambient Gas , 2012 .

[39]  A. S. Abdelmaksoud,et al.  Temporal variations of O3 and NOx in the urban background atmosphere of the coastal city Jeddah, Saudi Arabia , 2014 .

[40]  M. A. Zaidan,et al.  Bayesian framework for aerospace gas turbine engine prognostics , 2013, 2013 IEEE Aerospace Conference.

[41]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[42]  Adam S. Foster,et al.  Mixture of Clustered Bayesian Neural Networks for Modeling Friction Processes at the Nanoscale. , 2017, Journal of chemical theory and computation.

[43]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[44]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[45]  Oliver Ebenhöh,et al.  Measuring correlations in metabolomic networks with mutual information. , 2008, Genome informatics. International Conference on Genome Informatics.

[46]  M. Zunckel,et al.  A novel air pollution index based on the relative risk of daily mortality associated with short-term exposure to common air pollutants , 2007 .

[47]  M. Ketzel,et al.  Statistical modelling of aerosol particle number size distributions in urban and rural environments – A multi-site study , 2015 .

[48]  Sergio Machado Corrêa,et al.  Prediction of ozone concentration in tropospheric levels using artificial neural networks and support vector machine at Rio de Janeiro, Brazil , 2014 .

[49]  Benjamin M. Smith,et al.  Association Between Long-term Exposure to Ambient Air Pollution and Change in Quantitatively Assessed Emphysema and Lung Function. , 2019, JAMA.

[50]  Manuel Aleixandre,et al.  Performance evaluation of amperometric sensors for the monitoring of O3 and NO2 in ambient air at ppb level , 2015 .

[51]  Jun Wang,et al.  Satellite remote sensing of particulate matter and air quality assessment over global cities , 2006 .

[52]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[53]  S. Low Choy,et al.  Using the Generalised Additive Model to model the particle number count of ultrafine particles , 2011 .

[54]  Jukka Corander,et al.  Evaluation of a statistical forecast model for size-fractionated urban particle number concentrations using data from five European cities , 2013 .

[55]  D. Schwela,et al.  Strengths and Weaknesses of the WHO Urban Air Pollutant Database , 2020 .

[56]  T. Butler,et al.  CO2-equivalent emissions from European passenger vehicles in the years 1995–2015 based on real-world use: Assessing the climate benefit of the European “diesel boom” , 2019, Atmospheric Environment.

[57]  Charles M. Bishop,et al.  Ensemble learning in Bayesian neural networks , 1998 .

[58]  R. Muller,et al.  Air Pollution in China: Mapping of Concentrations and Sources , 2015, PloS one.

[59]  Massimiliano Cannata,et al.  Boosting a Weather Monitoring System in Low Income Economies Using Open and Non-Conventional Systems: Data Quality Analysis , 2019, Sensors.

[60]  Sancho Salcedo-Sanz,et al.  Prediction of hourly O3 concentrations using support vector regression algorithms , 2010 .

[61]  David M. Broday,et al.  Wireless Distributed Environmental Sensor Networks for Air Pollution Measurement—The Promise and the Current Reality , 2017, Sensors.

[62]  Jukka Corander,et al.  Forecasting size-fractionated particle number concentrations in the urban atmosphere , 2012 .

[63]  Claudio Carnevale,et al.  An integrated assessment tool to define effective air quality policies at regional scale , 2012, Environ. Model. Softw..

[64]  R. Harrison,et al.  Ozone balances in urban Saudi Arabia , 2018, npj Climate and Atmospheric Science.

[65]  Ana Isabel Miranda,et al.  Air Pollution Modeling and Its Application XIX , 2008 .

[66]  Peter J. Fleming,et al.  Gas turbine engine prognostics using Bayesian hierarchical models: A variational approach , 2016 .

[67]  M. A. Zaidan,et al.  A Predictive Model for Steady State Ozone Concentration at an Urban-Coastal Site , 2019, International journal of environmental research and public health.

[68]  M. Xue,et al.  Impacts of Mixing Processes in Nocturnal Atmospheric Boundary Layer on Urban Ozone Concentrations , 2013, Boundary-Layer Meteorology.

[69]  Li-Yen Shue,et al.  Data mining to aid policy making in air pollution management , 2004, Expert Syst. Appl..

[70]  Geb Thomas,et al.  Evaluation of low-cost electro-chemical sensors for environmental monitoring of ozone, nitrogen dioxide, and carbon monoxide , 2018, Journal of occupational and environmental hygiene.

[71]  Peter J. Fleming,et al.  Bayesian Hierarchical Models for aerospace gas turbine engine prognostics , 2015, Expert Syst. Appl..

[72]  Qijun Jiang,et al.  Field calibration of electrochemical NO 2 sensors in a citizen science context , 2018 .

[73]  Douglas R Lawson,et al.  Differences between Weekday and Weekend Air Pollutant Levels in Atlanta; Baltimore; Chicago; Dallas–Fort Worth; Denver; Houston; New York; Phoenix; Washington, DC; and Surrounding Areas , 2008, Journal of the Air & Waste Management Association.

[74]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.