Automatic Bayesian Density Analysis

Making sense of a dataset in an automatic and unsupervised fashion is a challenging problem in statistics and AI. Classical approaches for {exploratory data analysis} are usually not flexible enough to deal with the uncertainty inherent to real-world data: they are often restricted to fixed latent interaction models and homogeneous likelihoods; they are sensitive to missing, corrupt and anomalous data; moreover, their expressiveness generally comes at the price of intractable inference. As a result, supervision from statisticians is usually needed to find the right model for the data. However, since domain experts are not necessarily also experts in statistics, we propose Automatic Bayesian Density Analysis (ABDA) to make exploratory data analysis accessible at large. Specifically, ABDA allows for automatic and efficient missing value estimation, statistical data type and likelihood discovery, anomaly detection and dependency structure mining, on top of providing accurate density estimation. Extensive empirical evidence shows that ABDA is a suitable tool for automatic exploratory analysis of mixed continuous and discrete tabular data.

[1]  Paul D. McNicholas,et al.  Model-based clustering of microarray expression data via latent Gaussian mixture models , 2010, Bioinform..

[2]  John Hinde,et al.  Special issue on mixture models , 2017 .

[3]  Mark Jenkinson,et al.  The minimal preprocessing pipelines for the Human Connectome Project , 2013, NeuroImage.

[4]  Bernhard Schölkopf,et al.  The Randomized Dependence Coefficient , 2013, NIPS.

[5]  Sebastian Tschiatschek,et al.  On Theoretical Properties of Sum-Product Networks , 2015, AISTATS.

[6]  Pedro M. Domingos,et al.  Learning the Structure of Sum-Product Networks , 2013, ICML.

[7]  Floriana Esposito,et al.  Visualizing and understanding Sum-Product Networks , 2016, Machine Learning.

[8]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[9]  Padhraic Smyth,et al.  Stacked Density Estimation , 1997, NIPS.

[10]  Sergio Escalera,et al.  A brief Review of the ChaLearn AutoML Challenge: Any-time Any-dataset Learning without Human Intervention , 2016, AutoML@ICML.

[11]  Adnan Darwiche,et al.  A differential approach to inference in Bayesian networks , 2000, JACM.

[12]  Rajesh P. N. Rao,et al.  Deep Spatial Affordance Hierarchy : Spatial Knowledge Representation for Planning in Large-scale Environments , 2017 .

[13]  Adnan Darwiche,et al.  On the Robustness of Most Probable Explanations , 2006, UAI.

[14]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[15]  Ramakrishnan Srikant,et al.  Fast algorithms for mining association rules , 1998, VLDB 1998.

[16]  Zoubin Ghahramani,et al.  Variational Inference for Bayesian Mixtures of Factor Analysers , 1999, NIPS.

[17]  Peter Filzmoser,et al.  Robust fitting of mixtures using the trimmed likelihood estimator , 2007, Comput. Stat. Data Anal..

[18]  Pedro M. Domingos,et al.  Discriminative Learning of Sum-Product Networks , 2012, NIPS.

[19]  Joshua B. Tenenbaum,et al.  CrossCat: A Fully Bayesian Nonparametric Method for Analyzing Heterogeneous, High Dimensional Data , 2015, J. Mach. Learn. Res..

[20]  Seiichi Uchida,et al.  A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data , 2016, PloS one.

[21]  Daniel Lowd,et al.  Learning Sum-Product Networks with Direct and Indirect Variable Interactions , 2014, ICML.

[22]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[23]  Andreas Dengel,et al.  Histogram-based Outlier Score (HBOS): A fast Unsupervised Anomaly Detection Algorithm , 2012 .

[24]  Adnan Darwiche,et al.  Modeling and Reasoning with Bayesian Networks , 2009 .

[25]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[26]  Joshua B. Tenenbaum,et al.  Structure Discovery in Nonparametric Regression through Compositional Kernel Search , 2013, ICML.

[27]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[28]  Zoubin Ghahramani,et al.  Automatic Discovery of the Statistical Types of Variables in a Dataset , 2017, ICML.

[29]  P. Deb Finite Mixture Models , 2008 .

[30]  Adnan Darwiche,et al.  On Relaxing Determinism in Arithmetic Circuits , 2017, ICML.

[31]  Kristian Kersting,et al.  Poisson Sum-Product Networks: A Deep Architecture for Tractable Multivariate Poisson Distributions , 2017, AAAI.

[32]  Joshua B. Tenenbaum,et al.  Automatic Construction and Natural-Language Description of Nonparametric Regression Models , 2014, AAAI.

[33]  Dan Ventura,et al.  Greedy Structure Search for Sum-Product Networks , 2015, IJCAI.

[34]  Zoubin Ghahramani,et al.  General Latent Feature Models for Heterogeneous Datasets , 2017, J. Mach. Learn. Res..

[35]  Pedro M. Domingos,et al.  Sum-product networks: A new deep architecture , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[36]  Douglas A. Reynolds,et al.  Speaker Verification Using Adapted Gaussian Mixture Models , 2000, Digit. Signal Process..

[37]  Franz Pernkopf,et al.  On the Latent Variable Interpretation in Sum-Product Networks , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Pascal Poupart,et al.  A Unified Approach for Learning the Parameters of Sum-Product Networks , 2016, NIPS.

[39]  Kristian Kersting,et al.  Sum-Product Autoencoding: Encoding and Decoding Representations Using Sum-Product Networks , 2018, AAAI.

[40]  Dan Ventura,et al.  Learning the Architecture of Sum-Product Networks Using Clustering on Variables , 2012, NIPS.

[41]  Franz Pernkopf,et al.  Safe Semi-Supervised Learning of Sum-Product Networks , 2017, UAI.

[42]  Bradley P. Carlin,et al.  Markov Chain Monte Carlo conver-gence diagnostics: a comparative review , 1996 .

[43]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[44]  Max Welling,et al.  Semi-supervised Learning with Deep Generative Models , 2014, NIPS.

[45]  Yee Whye Teh,et al.  Scaling up the Automatic Statistician: Scalable Structure Discovery using Gaussian Processes , 2017, AISTATS.

[46]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[47]  Thomas L. Griffiths,et al.  Hierarchical Topic Models and the Nested Chinese Restaurant Process , 2003, NIPS.

[48]  Franz Pernkopf,et al.  Greedy Part-Wise Learning of Sum-Product Networks , 2013, ECML/PKDD.

[49]  Aaron Klein,et al.  Efficient and Robust Automated Machine Learning , 2015, NIPS.

[50]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD '00.

[51]  Yee Whye Teh,et al.  Sharing Clusters among Related Groups: Hierarchical Dirichlet Processes , 2004, NIPS.

[52]  Floriana Esposito,et al.  Simplifying, Regularizing and Strengthening Sum-Product Network Structure Learning , 2015, ECML/PKDD.

[53]  Kristian Kersting,et al.  Mixed Sum-Product Networks: A Deep Architecture for Hybrid Domains , 2018, AAAI.