Physics-Guided Machine Learning for Scientific Discovery: An Application in Simulating Lake Temperature Profiles

Physics-based models are often used to study engineering and environmental systems. The ability to model these systems is the key to achieving our future environmental sustainability and improving the quality of human life. This article focuses on simulating lake water temperature, which is critical for understanding the impact of changing climate on aquatic ecosystems and assisting in aquatic resource management decisions. General Lake Model (GLM) is a state-of-the-art physics-based model used for addressing such problems. However, like other physics-based models used for studying scientific and engineering systems, it has several well-known limitations due to simplified representations of the physical processes being modeled or challenges in selecting appropriate parameters. While state-of-the-art machine learning models can sometimes outperform physics-based models given ample amount of training data, they can produce results that are physically inconsistent. This article proposes a physics-guided recurrent neural network model (PGRNN) that combines RNNs and physics-based models to leverage their complementary strengths and improves the modeling of physical processes. Specifically, we show that a PGRNN can improve prediction accuracy over that of physics-based models (by over 20% even with very little training data), while generating outputs consistent with physical laws. An important aspect of our PGRNN approach lies in its ability to incorporate the knowledge encoded in physics-based models. This allows training the PGRNN model using very few true observed data while also ensuring high prediction accuracy. Although we present and evaluate this methodology in the context of modeling the dynamics of temperature in lakes, it is applicable more widely to a range of scientific and engineering disciplines where physics-based (also known as mechanistic) models are used.

[1]  B. Hicks Some evaluations of drag and bulk transfer coefficients over water bodies of different sizes , 1972 .

[2]  S. Tabata A Simple but Accurate Formula for the Saturation Vapor Pressure Over Liquid Water , 1973 .

[3]  J. Magnuson,et al.  Temperature as an Ecological Resource , 1979 .

[4]  Sheng Chen,et al.  Neural Networks for Nonlinear Dynamic System Modelling and Identification , 1992 .

[5]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[6]  Urban Forssell,et al.  Combining Semi-Physical and Neural Network Modeling: An Example ofIts Usefulness , 1997 .

[7]  Hod Lipson,et al.  Automated reverse engineering of nonlinear dynamical systems , 2007, Proceedings of the National Academy of Sciences.

[8]  V. Hayward,et al.  Big data: The next Google , 2008, Nature.

[9]  H. Paerl,et al.  Blooms Like It Hot , 2008, Science.

[10]  F. Rahel,et al.  Assessing the Effects of Climate Change on Aquatic Invasive Species , 2008, Conservation biology : the journal of the Society for Conservation Biology.

[11]  George Sugihara,et al.  Detecting Causality in Complex Ecosystems , 2012, Science.

[12]  Abhinav Gupta,et al.  Constrained Semi-Supervised Learning Using Attributes and Comparative Attributes , 2012, ECCV.

[13]  James J. Roberts,et al.  Fragmentation and thermal risks from climate change interact to affect persistence of native trout in the Colorado River basin , 2013, Global change biology.

[14]  Andrew J. Majda,et al.  Physics constrained nonlinear regression models for time series , 2012 .

[15]  D. Lazer,et al.  The Parable of Google Flu: Traps in Big Data Analysis , 2014, Science.

[16]  J. McDonnell,et al.  Debates—The future of hydrological sciences: A (common) path forward? A call to action aimed at understanding velocities, celerities and residence time distributions of the headwater hydrograph , 2014 .

[17]  Hoshin Vijai Gupta,et al.  Debates—the future of hydrological sciences: A (common) path forward? Using models and data to learn: A systems theoretic perspective on the future of hydrological science , 2014 .

[18]  Upmanu Lall Debates—The future of hydrological sciences: A (common) path forward? One water. One world. Many climes. Many souls , 2014 .

[19]  Albert J. Valocchi,et al.  Data-driven methods to improve baseflow prediction of a regional groundwater model , 2015, Comput. Geosci..

[20]  Misha Denil,et al.  From Group to Individual Labels Using Deep Features , 2015, KDD.

[21]  P. McCartney Big data science. , 2015, MCN. The American journal of maternal child nursing.

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[24]  George Em Karniadakis,et al.  Inferring solutions of dierential equations using noisy multi-delity data , 2016 .

[25]  Steven L. Brunton,et al.  Data-driven discovery of partial differential equations , 2016, Science Advances.

[26]  Theodore D. Harris,et al.  Predicting cyanobacterial abundance, microcystin, and geosmin in a eutrophic drinking-water reservoir using a 14-year dataset , 2017 .

[27]  Stefano Ermon,et al.  Label-Free Supervision of Neural Networks with Physics and Domain Knowledge , 2016, AAAI.

[28]  J. Read,et al.  Water quality data for national‐scale aquatic research: The Water Quality Portal , 2017 .

[29]  Kevin B. Flores,et al.  Hybrid modeling and prediction of dynamical systems , 2017, PLoS Comput. Biol..

[30]  Paris Perdikaris,et al.  Inferring solutions of differential equations using noisy multi-fidelity data , 2016, J. Comput. Phys..

[31]  Sean McGregor,et al.  FlareNet : A Deep Learning Framework for Solar Phenomena Prediction , 2017 .

[32]  Anuj Karpatne,et al.  Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling , 2017, ArXiv.

[33]  Ann Gordon-Ross,et al.  Accelerating High-energy Physics Exploration with Deep Learning , 2017, PEARC.

[34]  Abhinav Vishnu,et al.  Deep learning for computational chemistry , 2017, J. Comput. Chem..

[35]  Nagiza F. Samatova,et al.  Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data , 2016, IEEE Transactions on Knowledge and Data Engineering.

[36]  Ahmed H. Elsheikh,et al.  DR-RNN: A deep residual recurrent neural network for model reduction , 2017, ArXiv.

[37]  James J. Roberts,et al.  Nonnative Trout Invasions Combined with Climate Change Threaten Persistence of Isolated Cutthroat Trout Populations in the Southern Rocky Mountains , 2017 .

[38]  Nikhil Muralidhar,et al.  Incorporating Prior Domain Knowledge into Deep Neural Networks , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[39]  David P. Hamilton,et al.  A multi-lake comparative analysis of the General Lake Model (GLM): Stress-testing across a global observatory network , 2018, Environ. Model. Softw..

[40]  George E. Karniadakis,et al.  Hidden Fluid Mechanics: A Navier-Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data , 2018, ArXiv.

[41]  Stefano Ermon,et al.  Learning with Weak Supervision from Physics and Data-Driven Constraints , 2018, AI Mag..

[42]  J. Wiens,et al.  A Domain Guided CNN Architecture for Predicting Age from Structural Brain Images , 2018, MLHC.

[43]  David W Toth,et al.  The TensorMol-0.1 model chemistry: a neural network augmented with long-range physics , 2017, Chemical science.

[44]  G. Karniadakis,et al.  Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems , 2018, 1801.01236.

[45]  Omer San,et al.  Machine learning closures for model order reduction of thermal fluids , 2018, Applied Mathematical Modelling.

[46]  Daniel W. Davies,et al.  Machine learning for molecular and materials science , 2018, Nature.

[47]  Karthik Duraisamy,et al.  Long-time predictive modeling of nonlinear dynamical systems using neural networks , 2018, Complex..

[48]  Petros Koumoutsakos,et al.  Data-assisted reduced-order modeling of extreme events in complex dynamical systems , 2018, PloS one.

[49]  Paris Perdikaris,et al.  Learning Parameters and Constitutive Relationships with Physics Informed Deep Neural Networks , 2018, 1808.03398.

[50]  Maziar Raissi,et al.  Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations , 2018, J. Mach. Learn. Res..

[51]  Loong Fah Cheong,et al.  Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  Pierre Gentine,et al.  Achieving Conservation of Energy in Neural Network Emulators for Climate Modeling , 2019, ArXiv.

[53]  Anuj Karpatne,et al.  Physics Guided RNNs for Modeling Dynamical Systems: A Case Study in Simulating Lake Temperature Profiles , 2018, SDM.

[54]  Anuj Karpatne,et al.  Process‐Guided Deep Learning Predictions of Lake Water Temperature , 2019, Water Resources Research.

[55]  David P. Hamilton,et al.  A General Lake Model (GLM 3.0) for linking with high-frequency sensor data from the Global Lake Ecological Observatory Network (GLEON) , 2019, Geoscientific Model Development.

[56]  Vipin Kumar,et al.  Integrating Physics-Based Modeling with Machine Learning: A Survey , 2020, ArXiv.

[57]  X. Jia,et al.  Predicting lake surface water phosphorus dynamics using process-guided machine learning , 2020, Ecological Modelling.

[58]  Xiaowei Jia,et al.  Predicting Water Temperature Dynamics of Unmonitored Lakes With Meta‐Transfer Learning , 2020, Water Resources Research.

[59]  Yang Liu,et al.  Physics-Based Generative Adversarial Models for Image Restoration and Beyond , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.