For many applications, a probability model can be more easily expressed as a cumulative distribution functions (CDF) as compared to the use of probability density or mass functions (PDF/PMFs). One advantage of CDF models is the simplicity of representing multivariate heavy-tailed distributions. Examples of fields that can benefit from the use of graphical models for CDFs include climatology and epidemiology, where data follow heavy-tailed distributions and exhibit spatial correlations so that dependencies between model variables must be accounted for. However, in most cases the problem of learning from data consists of optimizing the log-likelihood function with respect to model parameters where we are required to optimize a log-PDF/PMF and not a log-CDF. Given a CDF defined on a graph, we present a message-passing algorithm called the gradient-derivative-product (GDP) algorithm that allows us to learn the model in terms of the log-likelihood function whereby messages correspond to local gradients of the likelihood with respect to model parameters. We demonstrate the GDP algorithm on real-world rainfall and H1N1 mortality data and we show that the heavy-tailed multivariate distributions that arise in these problems can both be naturally parameterized and tractably estimated from data using our algorithm.
[1]
G. J. G. Upton,et al.
An Introduction to Statistical Modelling
,
1983
.
[2]
T. Speed,et al.
Gaussian Markov Distributions over Finite Graphs
,
1986
.
[3]
X. Jin.
Factor graphs and the Sum-Product Algorithm
,
2002
.
[4]
T. Richardson.
Markov Properties for Acyclic Directed Mixed Graphs
,
2003
.
[5]
Thomas S. Richardson,et al.
Iterative Conditional Fitting for Gaussian Ancestral Graph Models
,
2004,
UAI.
[6]
Brendan J. Frey,et al.
Cumulative Distribution Networks and the Derivative-sum-product Algorithm: Models and Inference for Cumulative Distribution Functions on Graphs
,
2008,
J. Mach. Learn. Res..
[7]
Brendan J. Frey,et al.
Structured ranking learning using cumulative distribution networks
,
2008,
NIPS.
[8]
Jim C. Huang.
Cumulative distribution networks: inference, estimation and applications of graphical models for cumulative distribution functions
,
2009
.
[9]
F. Mallor,et al.
An introduction to statistical modelling of extreme values. Application to calculate extreme wind speeds
,
2009
.
[10]
Zoubin Ghahramani,et al.
Factorial Mixture of Gaussians and the Marginal Independence Model
,
2009,
AISTATS.