The Application of Machine Learning Techniques to Improve El Niño Prediction Skill

We review prediction efforts of El Ni\~no events in the tropical Pacific with particular focus on using modern machine learning (ML) methods based on artificial neural networks. With current classical prediction methods using both statistical and dynamical models, the skill decreases substantially for lead times larger than about 6 months. Initial ML results have shown enhanced skill for lead times larger than 12 months. The search for optimal attributes in these methods is described, in particular those derived from complex network approaches, and a critical outlook on further developments is given.

[1]  Michael J. McPhaden,et al.  Observations of Warm Water Volume Changes in the Equatorial Pacific and Their Relationship to El Niño and La Niña , 2000 .

[2]  Tao Lian,et al.  Strong influence of westerly wind bursts on El Niño diversity , 2015 .

[3]  William J. Blackwell,et al.  Neural Network Applications in High-Resolution Atmospheric Remote Sensing , 2005 .

[4]  Dimitrios I. Fotiadis,et al.  Machine learning applications in cancer prognosis and prediction , 2014, Computational and structural biotechnology journal.

[5]  Christopher K. Wikle,et al.  Bayesian Recurrent Neural Network Models for Forecasting and Quantifying Uncertainty in Spatial-Temporal Data , 2017, Entropy.

[6]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[7]  G. Bianconi,et al.  Machine learning meets complex networks via coalescent embedding in the hyperbolic space , 2016, Nature Communications.

[8]  Elizabeth C. Kent,et al.  Global analyses of sea surface temperature, sea ice, and night marine air temperature since the late nineteenth century , 2003 .

[9]  Eli Tziperman,et al.  Westerly Wind Bursts: ENSO's tail rather than the dog? , 2004 .

[10]  Yang Wang,et al.  ClimateLearn: A machine-learning approach for climate prediction using network measures , 2016 .

[11]  Shlomo Havlin,et al.  Very early warning of next El Niño , 2014, Proceedings of the National Academy of Sciences.

[12]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[13]  P. Webster.  The annual cycle and the predictability of the tropical coupled ocean-atmosphere system , 1995 .

[14]  Ernesto Tarantino,et al.  A Genetic Programming System for Time Series Prediction and Its Application to El Niño Forecast , 2005 .

[15]  Michael J. McPhaden,et al.  Tropical Pacific Ocean heat content variations and ENSO persistence barriers , 2003 .

[16]  A. Timmermann,et al.  El Niño–Southern Oscillation complexity , 2018, Nature.

[17]  Cristina Masoller,et al.  Networks in Climate , 2019 .

[18]  Evolutionary computation for climate and ocean forecasting: “El Niño forecasting” , 2002 .

[19]  Donald D. Lucas,et al.  Machine Learning Predictions of a Multiresolution Climate Model Ensemble , 2018 .

[20]  Mojib Latif,et al.  A review of ENSO prediction studies , 1994 .

[21]  S. Redner,et al.  Introduction To Percolation Theory , 2018 .

[22]  Youmin Tang.  Hybrid coupled models of the tropical Pacific: I interannual variability , 2002 .

[23]  Potsdam,et al.  Complex networks in climate dynamics. Comparing linear and nonlinear network construction methods , 2009, 0907.4359.

[24]  T. Evgeniou,et al.  To combine or not to combine: selecting among forecasts and their combinations , 2005 .

[25]  A. Timmermann,et al.  (Un)predictability of strong El Ni\~no events , 2017, 1710.03216.

[26]  Jan Hendrik Witte,et al.  Deep Learning for Finance: Deep Portfolios , 2016 .

[27]  Emilio Hernández-García,et al.  Erratum: Percolation-based precursors of transitions in extended systems , 2016, Scientific Reports.

[28]  K. Hanawa,et al.  Breakdown of ENSO predictors in the 2000s: Decadal changes of recharge/discharge‐SST phase relation and atmospheric intraseasonal forcing , 2012 .

[29]  A. Barnston,et al.  Skill of Real-Time Seasonal ENSO Model Predictions During 2002–11: Is Our Capability Increasing? , 2012 .

[30]  Nicholas G. Polson,et al.  Deep learning for finance: deep portfolios: J. B. HEATON, N. G. POLSON AND J. H. WITTE , 2017 .

[31]  M. Cane,et al.  A Model El Niñ–Southern Oscillation , 1987 .

[32]  P. O'Gorman,et al.  Using Machine Learning to Parameterize Moist Convection: Potential for Modeling of Climate, Climate Change, and Extreme Events , 2018, Journal of Advances in Modeling Earth Systems.

[33]  William W. Hsieh,et al.  Forecasting regional sea surface temperatures in the tropical Pacific by neural network models, with wind stress and sea level pressure as predictors , 1998 .

[34]  Andrew T. Wittenberg,et al.  How Predictable is El Niño , 2003 .

[35]  Dake Chen,et al.  El Niño prediction and predictability , 2008, J. Comput. Phys..

[36]  William W. Hsieh.  Machine Learning Methods in the Environmental Sciences: Contents , 2009 .

[37]  William W. Hsieh,et al.  Forecasting the equatorial Pacific sea surface temperatures by neural network models , 1997 .

[38]  Sylvie Thiria,et al.  Use of neural networks for predictions using time series: Illustration with the El Niño Southern oscillation phenomenon , 2000, Neurocomputing.

[39]  Mojib Latif,et al.  Dynamics of Interdecadal Variability in Coupled Ocean–Atmosphere Models , 1998 .

[40]  Stephan M. Winkler,et al.  Genetic Algorithms and Genetic Programming - Modern Concepts and Practical Applications , 2009 .

[41]  Dejian Yang,et al.  Progress in ENSO prediction and predictability study , 2018, National Science Review.

[42]  Henk A. Dijkstra,et al.  Using network theory and machine learning to predict El Niño , 2018, Earth System Dynamics.

[43]  M. Iredell,et al.  The NCEP Climate Forecast System Version 2 , 2014 .

[44]  R. Preisendorfer,et al.  Principal Component Analysis in Meteorology and Oceanography , 1988 .

[45]  A. Timmermann,et al.  A Nonlinear Theory for El Niño Bursting , 2003 .

[46]  Peter Bauer,et al.  Challenges and design choices for global weather and climate models based on machine learning , 2018, Geoscientific Model Development.

[47]  Sotiris B. Kotsiantis,et al.  Bagging and boosting variants for handling classifications problems: a survey , 2013, The Knowledge Engineering Review.

[48]  M. Roulston,et al.  The response of an ENSO Model to climate noise, weather noise and intraseasonal forcing , 2000 .

[49]  David S. Stoffer,et al.  Time series analysis and its applications , 2000 .

[50]  Robert H. Shumway,et al.  Time series analysis and its applications : with R examples , 2017 .

[51]  F. Tangang,et al.  Forecasting ENSO Events: A Neural Network–Extended EOF Approach. , 1998 .

[52]  Sebastian Scher,et al.  Toward Data‐Driven Weather and Climate Forecasting: Approximating a Simple General Circulation Model With Deep Learning , 2018, Geophysical Research Letters.

[53]  Sue Ellen Haupt,et al.  Artificial Intelligence Methods in the Environmental Sciences , 2008 .

[54]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[55]  William W. Hsieh,et al.  Neural network forecasts of the tropical Pacific sea surface temperatures , 2006, Neural Networks.

[56]  Thomas M. Smith,et al.  Extended Reconstructed Sea Surface Temperature, Version 5 (ERSSTv5): Upgrades, Validations, and Intercomparisons , 2017 .

[57]  William W. Hsieh,et al.  Machine Learning Methods in the Environmental Sciences: Neural Networks and Kernels , 2009 .

[58]  B. McCarl,et al.  The economic consequences of ENSO events for agriculture , 1999 .

[59]  Yuval.  Neural Network Training for Prediction of Climatological Time Series, Regularized by Minimization of the Generalized Cross-Validation Function , 2000 .

[60]  A. Barnston,et al.  Observing and Predicting the 2015/16 El Niño , 2017 .

[61]  Paul J. Roebber,et al.  What Do Networks Have to Do with Climate , 2006 .

[62]  Michael Ghil,et al.  El Ni�o on the Devil's Staircase: Annual Subharmonic Steps to Chaos , 1994, Science.

[63]  W. Duan,et al.  Exploring the initial errors that cause a significant “spring predictability barrier” for El Niño events , 2009 .

[64]  Bin Wang,et al.  Season‐dependent dynamics of nonlinear optimal error growth and El Niño‐Southern Oscillation predictability in a theoretical model , 2007 .

[65]  W. Hsieh,et al.  Hybrid coupled models of the tropical Pacific – II ENSO prediction , 2002 .

[66]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[67]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[68]  Mahad Baawain,et al.  El Niño southern-oscillation prediction using southern oscillation index and Niño3 as onset indicators: Application of artificial neural networks , 2005 .

[69]  W. Duan,et al.  Does Model Parameter Error Cause a Significant “Spring Predictability Barrier” for El Niño Events in the Zebiak–Cane Model? , 2012 .

[70]  Shlomo Havlin,et al.  Improved El Niño forecasting by cooperativity detection , 2013, Proceedings of the National Academy of Sciences.

[71]  S. Havlin,et al.  Emergence of El Niño as an autonomous component in the climate network. , 2010, Physical review letters.

[72]  Aranildo R. Lima,et al.  Nonlinear regression in environmental sciences using extreme learning machines: A comparative evaluation , 2015, Environ. Model. Softw..

[73]  J. Thepaut,et al.  The ERA‐Interim reanalysis: configuration and performance of the data assimilation system , 2011 .

[74]  Andrew Stuart,et al.  Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High‐Resolution Simulations , 2017, 1709.00037.

[75]  Joachim Denzler,et al.  Deep learning and process understanding for data-driven Earth system science , 2019, Nature.

[76]  A. Timmermann,et al.  The Curious Case of the EL Niño That Never Happened: A Perspective from 40 Years of Progress in Climate Research and Forecasting , 2015 .

[77]  Dake Chen,et al.  Effects of westerly wind bursts on El Niño: A new perspective , 2014 .

[78]  Fei-Fei Jin,et al.  An Equatorial Ocean Recharge Paradigm for ENSO. Part II: A Stripped-Down Coupled Model , 1997 .

[79]  Shlomo Havlin,et al.  Percolation framework to describe El Niño conditions. , 2016, Chaos.

[80]  Eli Tziperman,et al.  El Ni�o Chaos: Overlapping of Resonances Between the Seasonal Cycle and the Pacific Ocean-Atmosphere Oscillator , 1994, Science.

[81]  Lidia Auret,et al.  Machine learning applications in minerals processing: A review , 2019, Minerals Engineering.