Predictive Analytics for Extreme Events in Big Data

This paper presents an efficient computational methodology for longitudinal and cross-sectional analysis of extreme event statistics in large data sets. The analyzed data are available across multiple time periods and multiple individuals in a population. Some of the periods and individuals might have no extreme events and some might have much data. The extreme events are modeled with a Pareto or exponential tail distribution. The proposed approach to longitudinal and cross-sectional analysis of the tail models is based on non-parametric Bayesian formulation. The maximum a posteriori probability problem leads to two convex problems for the tail parameters. Solving one problem yields the trends for the tail decay rate across the population and time periods. Solving another gives the trends of the tail quintile level. The approach is illustrated by providing analysis of 10-and 100-year extreme event risks for extreme climate events and for peak power loads in electrical utility data.