Environmental Sensor Placement with Convolutional Gaussian Neural Processes

Environmental sensors are crucial for monitoring weather conditions and the impacts of climate change. However, it is challenging to maximise measurement informativeness and place sensors efficiently, particularly in remote regions like Antarctica. Probabilistic machine learning models can evaluate placement informativeness by predicting the uncertainty reduction provided by a new sensor. Gaussian process (GP) models are widely used for this purpose, but they struggle with capturing complex non-stationary behaviour and scaling to large datasets. This paper proposes using a convolutional Gaussian neural process (ConvGNP) to address these issues. A ConvGNP uses neural networks to parameterise a joint Gaussian distribution at arbitrary target locations, enabling flexibility and scalability. Using simulated surface air temperature anomaly over Antarctica as ground truth, the ConvGNP learns spatial and seasonal non-stationarities, outperforming a non-stationary GP baseline. In a simulated sensor placement experiment, the ConvGNP better predicts the performance boost obtained from new observations than GP baselines, leading to more informative sensor placements. We contrast our approach with physics-based sensor placement methods and propose future work towards an operational sensor placement recommendation system. This system could help to realise environmental digital twins that actively direct measurement sampling to improve the digital representation of reality.

[1]  Richard E. Turner,et al.  Autoregressive Conditional Neural Processes , 2023, ICLR.

[2]  Jayesh K. Gupta,et al.  ClimaX: A foundation model for weather and climate , 2023, ArXiv.

[3]  Sarah F. McGough,et al.  Learning from data with structured missingness , 2023, Nature Machine Intelligence.

[4]  Suman V. Ravuri,et al.  Machine learning emulation of a local-scale UK climate model , 2022, ArXiv.

[5]  Lingxi Xie,et al.  Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast , 2022, ArXiv.

[6]  K. Murphy,et al.  Uncertainty Disentanglement with Non-stationary Heteroscedastic Gaussian Processes for Active Learning , 2022, ArXiv.

[7]  G. Carmichael,et al.  The future of Earth system prediction: Advances in model-data fusion , 2022, Science advances.

[8]  Richard E. Turner,et al.  Practical Conditional Neural Processes Via Tractable Dependent Predictions , 2022, ICLR.

[9]  Richard E. Turner,et al.  Convolutional conditional neural processes for local climate downscaling , 2021, Geoscientific Model Development.

[10]  Kevin W. Manning,et al.  Assessing observation network design predictions for monitoring Antarctic surface temperature , 2021, Quarterly Journal of the Royal Meteorological Society.

[11]  Gordon S. Blair,et al.  Digital twins of the natural environment , 2021, Patterns.

[12]  Richard E. Turner,et al.  Efficient Gaussian Neural Processes for Regression , 2021, ArXiv.

[13]  Raia Hadsell,et al.  Skilful precipitation nowcasting using deep generative models of radar , 2021, Nature.

[14]  Carl E. Rasmussen,et al.  The Promises and Pitfalls of Deep Kernel Learning , 2021, UAI.

[15]  Chris Russell,et al.  Seasonal Arctic sea ice forecasting with probabilistic deep learning , 2021, Nature Communications.

[16]  Richard E. Turner,et al.  The Gaussian Neural Process , 2021, ArXiv.

[17]  P. Heimbach,et al.  Leveraging Uncertainty Quantification to Design Ocean Climate Observing Systems , 2020, Journal of Advances in Modeling Earth Systems.

[18]  Elliot J. Crowley,et al.  Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels , 2020, NeurIPS.

[19]  翼 峰松,et al.  5分で分かる!? 有名論文ナナメ読み:Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K-R. and Samek, W. : On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation , 2020 .

[20]  Richard E. Turner,et al.  Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processes , 2020, NeurIPS.

[21]  J. Thepaut,et al.  The ERA5 global reanalysis , 2020, Quarterly Journal of the Royal Meteorological Society.

[22]  P. Heimbach,et al.  Quantifying Dynamical Proxy Potential Through Shared Adjustment Physics in the North Atlantic , 2020, Journal of Geophysical Research: Oceans.

[23]  Jordan G. Powers,et al.  Optimal Network Design Applied to Monitoring and Forecasting Surface Temperature in Antarctica , 2020, Monthly Weather Review.

[24]  Ralph C. Smith,et al.  Sequential optimal positioning of mobile sensors using mutual information , 2019, Stat. Anal. Data Min..

[25]  Iain Murray,et al.  Neural Spline Flows , 2019, NeurIPS.

[26]  Stephen Tyree,et al.  Exact Gaussian Processes on a Million Data Points , 2019, NeurIPS.

[27]  Gunnar Rätsch,et al.  Meta-Learning Mean Functions for Gaussian Processes , 2019, 1901.08098.

[28]  Aaron J. Fisher,et al.  All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously , 2018, J. Mach. Learn. Res..

[29]  Karina Gibert,et al.  Environmental Data Science , 2018, Environ. Model. Softw..

[30]  Yee Whye Teh,et al.  Conditional Neural Processes , 2018, ICML.

[31]  Sharanya J. Majumdar,et al.  A review of targeted observations , 2016 .

[32]  Robert Atlas,et al.  Future Observing System Simulation Experiments , 2016 .

[33]  Christopher W. Fairall,et al.  Advancing Polar Prediction Capabilities on Daily to Seasonal Time Scales , 2016 .

[34]  Vincent Dumoulin,et al.  Deconvolution and Checkerboard Artifacts , 2016 .

[35]  Andrew Gordon Wilson,et al.  Deep Kernel Learning , 2015, AISTATS.

[36]  Vincent Guidard,et al.  Observation impact over the southern polar area during the Concordiasi field campaign , 2016 .

[37]  L. Vogt Statistics For Spatial Data , 2016 .

[38]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.

[39]  Alexander Binder,et al.  On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation , 2015, PloS one.

[40]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[41]  Neil D. Lawrence,et al.  Gaussian Processes for Big Data , 2013, UAI.

[42]  Fabio Tozeto Ramos,et al.  Bayesian optimisation for Intelligent Environmental Monitoring , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[43]  Matthew A. Lazzara,et al.  Antarctic Automatic Weather Station Program: 30 Years of Polar Observation , 2012 .

[44]  T. Bracegirdle,et al.  The reliability of Antarctic tropospheric pressure and temperature in the latest global reanalyses , 2012 .

[45]  Koji Yamashita,et al.  The Influence of Assimilating Dropsonde Data on Typhoon Track and Midlatitude Forecasts , 2011 .

[46]  Hugh F. Durrant-Whyte,et al.  Modeling and decision making in spatio-temporal processes for environmental surveillance , 2010, 2010 IEEE International Conference on Robotics and Automation.

[47]  Michalis K. Titsias,et al.  Variational Learning of Inducing Variables in Sparse Gaussian Processes , 2009, AISTATS.

[48]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[49]  Daniel Rueckert,et al.  Medical Image Computing and Computer-Assisted Intervention − MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part II , 2017, Lecture Notes in Computer Science.

[50]  KrauseAndreas,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008 .

[51]  Andreas Krause,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008, J. Mach. Learn. Res..

[52]  Ryan D. Torn,et al.  Ensemble-Based Sensitivity Analysis , 2008 .

[53]  A. Raftery,et al.  Probabilistic forecasts, calibration and sharpness , 2007 .

[54]  Andreas Krause,et al.  Efficient Planning of Informative Paths for Multiple Robots , 2006, IJCAI.

[55]  C. Guestrin,et al.  Near-optimal sensor placements: maximizing information while minimizing communication cost , 2006, 2006 5th International Conference on Information Processing in Sensor Networks.

[56]  D. Bromwich,et al.  Strong Trends in the Skill of the ERA-40 and NCEP–NCAR Reanalyses in the High and Midlatitudes of the Southern Hemisphere, 1958–2001* , 2004 .

[57]  Rolf H. Langland,et al.  Estimation of observation impact using the NRL atmospheric variational data assimilation adjoint system , 2004 .

[58]  Maxim Sviridenko,et al.  A note on maximizing a submodular set function subject to a knapsack constraint , 2004, Oper. Res. Lett..

[59]  Klaus Obermayer,et al.  Gaussian process regression: active data selection and test point rejection , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[60]  Klaus Obermayer,et al.  Gaussian Process Regression: Active Data Selection and Test Point , 2000 .

[61]  David A. Cohn,et al.  Neural Network Exploration Using Optimal Experiment Design , 1993, NIPS.

[62]  David J. C. MacKay,et al.  Information-Based Objective Functions for Active Data Selection , 1992, Neural Computation.

[63]  D. Lindley On a Measure of the Information Provided by an Experiment , 1956 .