Ultra-fast analog ensemble using kd-tree

Analog ensemble (AnEn) is a popular probabilistic weather forecasting method based on similarity search. In that, forecasters are tasked to search for the top-m nearest neighbors (e.g., in terms of Euclidean distance) to a length-k query, from a set of historical data points in k-dimensional space. This is a straightforward yet time-consuming procedure, and few methods seem to be significantly better than a brute-force computation of all distances. To that end, I recommend using a kd-tree to perform AnEn, which appears to be one of (if not) the fastest approaches.

[1]  A. H. Murphy,et al.  What Is a Good Forecast? An Essay on the Nature of Goodness in Weather Forecasting , 1993 .

[2]  Dazhi Yang,et al.  Ultra-fast preselection in lasso-type spatio-temporal solar forecasting problems , 2018, Solar Energy.

[3]  L. D. Monache,et al.  An analog ensemble for short-term probabilistic solar power forecast , 2015 .

[4]  Rob J Hyndman,et al.  Probabilistic energy forecasting: Global Energy Forecasting Competition 2014 and beyond , 2016 .

[5]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[6]  Stefano Alessandrini,et al.  A novel application of an analog ensemble for short-term wind power forecasting , 2015 .

[7]  Emmanuel Papadakis,et al.  Taking R to its limits: 70+ tips , 2018, PeerJ Prepr..

[8]  Jan Kleissl,et al.  Operational solar forecasting for the real-time market , 2019, International Journal of Forecasting.

[9]  Dazhi Yang,et al.  A guideline to solar forecasting research practice: Reproducible, operational, probabilistic or physically-based, ensemble, and skill (ROPES) , 2019, Journal of Renewable and Sustainable Energy.

[10]  Luca Delle Monache,et al.  Wind resource estimates with an analog ensemble approach , 2015 .

[11]  David P. Larson,et al.  A comprehensive dataset for the accelerated development and benchmarking of solar forecasting methods , 2019, Journal of Renewable and Sustainable Energy.

[12]  Jon Louis Bentley,et al.  An Algorithm for Finding Best Matches in Logarithmic Expected Time , 1977, TOMS.

[13]  Stefano Alessandrini,et al.  An ultra-fast way of searching weather analogs for renewable energy forecasting , 2019, Solar Energy.

[14]  Luca Delle Monache,et al.  Post-processing techniques and principal component analysis for regional wind power and solar irradiance forecasting , 2016 .

[15]  Carlos F.M. Coimbra,et al.  History and trends in solar irradiance and PV power forecasting: A preliminary assessment and review using text mining , 2018, Solar Energy.

[16]  Joakim Widén,et al.  Review on probabilistic forecasting of photovoltaic power production and electricity consumption , 2018 .

[17]  Luca Delle Monache,et al.  Short-term photovoltaic power forecasting using Artificial Neural Networks and an Analog Ensemble , 2017 .