A New Class of Time Dependent Latent Factor Models with Applications

In many applications, observed data are influenced by some combination of latent causes. For example, suppose sensors are placed inside a building to record responses such as temperature, humidity, power consumption and noise levels. These random, observed responses are typically affected by many unobserved, latent factors (or features) within the building such as the number of individuals, the turning on and off of electrical devices, power surges, etc. These latent factors are usually present for a contiguous period of time before disappearing; further, multiple factors could be present at a time. This paper develops new probabilistic methodology and inference methods for random object generation influenced by latent features exhibiting temporal persistence. Every datum is associated with subsets of a potentially infinite number of hidden, persistent features that account for temporal dynamics in an observation. The ensuing class of dynamic models constructed by adapting the Indian Buffet Process --- a probability measure on the space of random, unbounded binary matrices --- finds use in a variety of applications arising in operations, signal processing, biomedicine, marketing, image analysis, etc. Illustrations using synthetic and real data are provided.

[1]  Zoubin Ghahramani,et al.  Modeling Dyadic Data with Binary Latent Factors , 2006, NIPS.

[2]  Zoubin Ghahramani,et al.  Infinite Sparse Factor Analysis and Infinite Independent Components Analysis , 2007, ICA.

[3]  S. J. Press,et al.  Bayesian Inference in Factor Analysis , 1989 .

[4]  M. Veloso,et al.  Latent Variable Models , 2019, Statistical and Econometric Methods for Transportation Data Analysis.

[5]  Thomas L. Griffiths,et al.  Infinite latent feature models and the Indian buffet process , 2005, NIPS.

[6]  K. Vijayaraghavan,et al.  Application of Factor Analysis to Identify Dietary Patterns and Use of Factor Scores to Study Their Relationship with Nutritional Status of Adult Rural Populations , 2011, Journal of health, population, and nutrition.

[7]  Fernando Pérez-Cruz,et al.  Infinite Factorial Dynamical Model , 2015, NIPS.

[8]  Arnaud Doucet,et al.  Generalized Polya Urn for Time-varying Dirichlet Process Mixtures , 2007, UAI.

[9]  Guillermo Sapiro,et al.  Non-Parametric Bayesian Dictionary Learning for Sparse Image Representations , 2009, NIPS.

[10]  Zoubin Ghahramani,et al.  Accelerated sampling for the Indian Buffet Process , 2009, ICML '09.

[11]  Michael I. Jordan,et al.  Hierarchical Beta Processes and the Indian Buffet Process , 2007, AISTATS.

[12]  Michalis K. Titsias,et al.  The Infinite Gamma-Poisson Feature Model , 2007, NIPS.

[13]  W. Eric L. Grimson,et al.  Construction of Dependent Dirichlet Processes based on Poisson Processes , 2010, NIPS.

[14]  John W. Paisley,et al.  Stochastic Variational Inference for the HDP-HMM , 2016, AISTATS.

[15]  D. Lindley,et al.  Bayes Estimates for the Linear Model , 1972 .

[16]  Nicholas J. Foti,et al.  A Survey of Non-Exchangeable Priors for Bayesian Nonparametric Models , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[18]  Zoubin Ghahramani,et al.  A Unifying Review of Linear Gaussian Models , 1999, Neural Computation.

[19]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[20]  Finale Doshi-Velez,et al.  The Indian Buffet Process: Scalable Inference and Extensions , 2009 .

[21]  N. Hjort Nonparametric Bayes Estimators Based on Beta Processes in Models for Life History Data , 1990 .

[22]  Michael I. Jordan,et al.  Sharing Features among Dynamical Systems with Beta Processes , 2009, NIPS.

[23]  David B. Dunson,et al.  The Kernel Beta Process , 2011, NIPS.

[24]  Michael I. Jordan,et al.  Combinatorial Clustering and the Beta Negative Binomial Process , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Thomas L. Griffiths,et al.  Particle Filtering for Nonparametric Bayesian Matrix Factorization , 2006, NIPS.

[26]  Fernando Pérez-Cruz,et al.  Infinite Factorial Unbounded-State Hidden Markov Model , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Andrew Gelman,et al.  General methods for monitoring convergence of iterative simulations , 1998 .

[28]  C. Antoniak Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[29]  Yee Whye Teh,et al.  Variational Inference for the Indian Buffet Process , 2009, AISTATS.

[30]  Kenneth M. Hanson,et al.  Introduction to Bayesian image analysis , 1993 .

[31]  Radford M. Neal Slice Sampling , 2003, The Annals of Statistics.

[32]  P. Deb Finite Mixture Models , 2008 .

[33]  Matthew T. Harrison,et al.  A simple example of Dirichlet process mixture inconsistency for the number of components , 2013, NIPS.

[34]  Matthew J. Johnson,et al.  Bayesian nonparametric hidden semi-Markov models , 2012, J. Mach. Learn. Res..

[35]  Seungjin Choi,et al.  Independent Component Analysis , 2009, Handbook of Natural Computing.

[36]  J. Griffin The Ornstein–Uhlenbeck Dirichlet process and other time-varying processes for Bayesian nonparametric inference , 2011 .

[37]  Lawrence M. Leemis,et al.  Chapter 6 Arrival Processes, Random Lifetimes and Random Objects , 2006, Simulation.

[38]  Michael I. Jordan,et al.  A Sticky HDP-HMM With Application to Speaker Diarization , 2009, 0905.2592.

[39]  Zoubin Ghahramani,et al.  Dependent Indian Buffet Processes , 2010, AISTATS.

[40]  Gerhard Winkler,et al.  Image Analysis, Random Fields and Markov Chain Monte Carlo Methods: A Mathematical Introduction , 2002 .

[41]  Michael I. Jordan,et al.  JOINT MODELING OF MULTIPLE TIME SERIES VIA THE BETA PROCESS WITH APPLICATION TO MOTION CAPTURE SEGMENTATION , 2013, 1308.4747.

[42]  John Geweke,et al.  Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments , 1991 .

[43]  Yee Whye Teh,et al.  Stick-breaking Construction for the Indian Buffet Process , 2007, AISTATS.

[44]  Daniel N. Rockmore,et al.  A unifying representation for a class of dependent random measures , 2012, AISTATS.

[45]  Thomas L. Griffiths,et al.  A Non-Parametric Bayesian Method for Inferring Hidden Causes , 2006, UAI.

[46]  Savannah Wei Shi,et al.  Information Acquisition During Online Decision Making: A Model-Based Exploration Using Eye-Tracking Data , 2013, Manag. Sci..

[47]  Yee Whye Teh,et al.  Spatial Normalized Gamma Processes , 2009, NIPS.

[48]  Daniel P. W. Ellis,et al.  A Discriminative Model for Polyphonic Piano Transcription , 2007, EURASIP J. Adv. Signal Process..

[49]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[50]  Klaus Obermayer,et al.  A new summarization method for affymetrix probe level data , 2006, Bioinform..

[51]  Peter I. Frazier,et al.  Distance Dependent Infinite Latent Feature Models , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  L. Baum,et al.  Statistical Inference for Probabilistic Functions of Finite State Markov Chains , 1966 .

[53]  Matti Pietikäinen,et al.  Machine Learning for Vision-Based Motion Analysis , 2011 .

[54]  T. Ferguson A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[55]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[56]  Michael I. Jordan,et al.  An HDP-HMM for systems with state persistence , 2008, ICML '08.

[57]  Michael,et al.  On a Class of Bayesian Nonparametric Estimates : I . Density Estimates , 2008 .

[58]  Yee Whye Teh,et al.  The Infinite Factorial Hidden Markov Model , 2008, NIPS.

[59]  David B. Dunson,et al.  Dependent Hierarchical Beta Process for Image Interpolation and Denoising , 2011, AISTATS.

[60]  R. MacCallum,et al.  THE APPLICATION OF EXPLORATORY FACTOR ANALYSIS IN APPLIED PSYCHOLOGY: A CRITICAL REVIEW AND ANALYSIS , 1986 .

[61]  R. Darlington,et al.  Factor Analysis , 2008 .

[62]  Fernando Pérez-Cruz,et al.  Bayesian nonparametric comorbidity analysis of psychiatric disorders , 2014, J. Mach. Learn. Res..