Machine Learning with Interdependent and Non-identically Distributed Data (Dagstuhl Seminar 15152)

One of the most common assumptions in many machine learning and data analysis tasks is that the given data points are realizations of independent and identically distributed (IID) random variables. However, this assumption is often violated, e.g., when training and test data come from different distributions (dataset bias or domain shift) or the data points are highly interdependent (e.g., when the data exhibits temporal or spatial correlations). Both scenarios are typical situations in visual recognition and computational biology. For instance, computer vision and image analysis models can be learned from object-centric internet resources, but are often rather applied to real-world scenes. In computational biology and personalized medicine, training data may be recorded at a particular hospital, but the model is applied to make predictions on data from different hospitals, where patients exhibit a different population structure. In the seminar report, we discuss, present, and explore new machine learning methods that can deal with non-i.i.d. data as well as new application scenarios.

[1]  Mario Fritz,et al.  See the Difference: Direct Pre-Image Reconstruction and Pose Estimation by Differentiating HOG , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Anne Leucht,et al.  Dependent wild bootstrap for degenerate U- and V-statistics , 2013, J. Multivar. Anal..

[3]  Alessandro Lazaric,et al.  Transfer in Reinforcement Learning: A Framework and a Survey , 2012, Reinforcement Learning.

[4]  C. I. Bliss,et al.  THE METHOD OF PROBITS. , 1934, Science.

[5]  C. F. Sirmans,et al.  Spatial Modeling With Spatially Varying Coefficient Processes , 2003, Journal of the American Statistical Association.

[6]  Mario Fritz,et al.  Multi-class Video Co-segmentation with a Generative Multi-video Model , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Le Song,et al.  A Kernel Statistical Test of Independence , 2007, NIPS.

[8]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[9]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[10]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[11]  Alessandro Lazaric,et al.  Sequential Transfer in Multi-armed Bandit with Finite Set of Models , 2013, NIPS.

[12]  Konrad Rieck,et al.  DREBIN: Effective and Explainable Detection of Android Malware in Your Pocket , 2014, NDSS.

[13]  Mario Fritz,et al.  Object Tracking and Pose Estimation Using Light-Field Object Models , 2002, VMV.

[14]  Mario Fritz,et al.  Sequential Bayesian Model Update under Structured Scene Prior for Semantic Road Scenes Labeling , 2013, 2013 IEEE International Conference on Computer Vision.

[15]  Christophe Andrieu,et al.  Kernel Adaptive Metropolis-Hastings , 2014, ICML.

[16]  Arthur Gretton,et al.  A Wild Bootstrap for Degenerate Kernel Tests , 2014, NIPS.

[17]  Peter Kraft,et al.  Replication in genome-wide association studies. , 2009, Statistical science : a review journal of the Institute of Mathematical Statistics.

[18]  Yann LeCun,et al.  Tangent Prop - A Formalism for Specifying Selected Invariances in an Adaptive Network , 1991, NIPS.

[19]  Joachim Denzler,et al.  Learning with few examples for binary and multiclass classification using regularization of randomized trees , 2011, Pattern Recognit. Lett..

[20]  Trevor Darrell,et al.  LSDA: Large Scale Detection through Adaptation , 2014, NIPS.

[21]  David Haussler,et al.  Exploiting Generative Models in Discriminative Classifiers , 1998, NIPS.

[22]  Christoph H. Lampert,et al.  A PAC-Bayesian bound for Lifelong Learning , 2013, ICML.

[23]  Bernt Schiele,et al.  RALF: A reinforced active learning formulation for object class recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Joachim Denzler,et al.  Selecting Influential Examples: Active Learning with Expected Model Output Changes , 2014, ECCV.

[25]  Ruth Urner,et al.  Active Nearest Neighbors in Changing Environments , 2015, ICML.

[26]  Peter Stone,et al.  Transfer Learning for Reinforcement Learning Domains: A Survey , 2009, J. Mach. Learn. Res..

[27]  F. Barthe.  Learning from Dependent Observations , 2006 .

[28]  Ying Liu,et al.  FaST linear mixed models for genome-wide association studies , 2011, Nature Methods.

[29]  Erik Rodner,et al.  Transform-based Domain Adaptation for Big Data , 2013 .

[30]  Bernhard Schölkopf,et al.  Kernel Measures of Conditional Dependence , 2007, NIPS.

[31]  Mario Fritz,et al.  Appearance-based gaze estimation in the wild , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[33]  H. Shimodaira,et al.  Improving predictive inference under covariate shift by weighting the log-likelihood function , 2000 .

[34]  Bernhard Schölkopf,et al.  Training Invariant Support Vector Machines , 2002, Machine Learning.

[35]  Bernt Schiele,et al.  Integrating representative and discriminant models for object category detection , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[36]  Bernhard Schölkopf,et al.  Statistical analysis of coupled time series with Kernel Cross-Spectral Density operators , 2013, NIPS.

[37]  Sebastian Thrun,et al.  Lifelong robot learning , 1993, Robotics Auton. Syst..

[38]  Bernhard Schölkopf,et al.  Kernel-based Conditional Independence Test and Application in Causal Discovery , 2011, UAI.

[39]  Michael J. Black,et al.  OpenDR: An Approximate Differentiable Renderer , 2014, ECCV.

[40]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[41]  Joachim Denzler,et al.  Kernel Null Space Methods for Novelty Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Arthur Gretton,et al.  A Kernel Test for Three-Variable Interactions , 2013, NIPS.

[43]  Joachim Denzler,et al.  Nonparametric Part Transfer for Fine-Grained Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Joachim Denzler,et al.  Part Detector Discovery in Deep Convolutional Neural Networks , 2014, ACCV.

[45]  J. Cunningham,et al.  Gaussian Probabilities and Expectation Propagation , 2011, 1111.6832.

[46]  Neil D. Lawrence,et al.  Joint Modelling of Confounding Factors and Prominent Genetic Regulators Provides Increased Accuracy in Genetical Genomics Studies , 2012, PLoS Comput. Biol..

[47]  Karsten M. Borgwardt,et al.  ccSVM: correcting Support Vector Machines for confounding factors in biological data classification , 2011, Bioinform..

[48]  Joachim Denzler,et al.  Exemplar-Specific Patch Features for Fine-Grained Recognition , 2014, GCPR.

[49]  Bjarni J. Vilhjálmsson,et al.  The nature of confounding in genome-wide association studies , 2012, Nature Reviews Genetics.

[50]  Trevor Darrell,et al.  Adapting Visual Category Models to New Domains , 2010, ECCV.

[51]  Mario Fritz,et al.  Recognizing Materials from Virtual Examples , 2012, ECCV.

[52]  Arthur Gretton,et al.  A Kernel Independence Test for Random Processes , 2014, ICML.

[53]  Pietro Perona,et al.  Combining generative models and Fisher kernels for object recognition , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[54]  R. Tibshirani,et al.  Sparse Principal Component Analysis , 2006 .

[55]  Mario Fritz,et al.  Image-Based Synthesis and Re-synthesis of Viewpoints Guided by 3D Models , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Sébastien Bubeck,et al.  Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems , 2012, Found. Trends Mach. Learn..

[57]  N. Breslow,et al.  Approximate inference in generalized linear mixed models , 1993 .