Generative versus discriminative training of RBMs for classification of fMRI images

Neuroimaging datasets often have a very large number of voxels and a very small number of training cases, which means that overfitting of models for this data can become a very serious problem. Working with a set of fMRI images from a study on stroke recovery, we consider a classification task for which logistic regression performs poorly, even when L1- or L2- regularized. We show that much better discrimination can be achieved by fitting a generative model to each separate condition and then seeing which model is most likely to have generated the data. We compare discriminative training of exactly the same set of models, and we also consider convex blends of generative and discriminative training.

[1]  J. Mazziotta,et al.  Automated image registration , 1993 .

[2]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[3]  Scott T. Grafton,et al.  Automated image registration: I. General methods and intrasubject, intramodality validation. , 1998, Journal of computer assisted tomography.

[4]  Michael I. Jordan,et al.  On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes , 2001, NIPS.

[5]  C. Genovese,et al.  Cerebellar hemispheric activation ipsilateral to the paretic hand correlates with functional recovery after stroke. , 2002, Brain : a journal of neurology.

[6]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[7]  Geoffrey E. Hinton,et al.  Exponential Family Harmoniums with an Application to Information Retrieval , 2004, NIPS.

[8]  Christopher Joseph Pal,et al.  Multi-Conditional Learning: Generative/Discriminative Training for Clustering and Classification , 2006, AAAI.

[9]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[10]  Thomas Hofmann,et al.  Greedy Layer-Wise Training of Deep Networks , 2007 .

[11]  Geoffrey E. Hinton,et al.  To recognize shapes, first learn to generate images. , 2007, Progress in brain research.

[12]  L. K. Hansen,et al.  Multivariate strategies in functional magnetic resonance imaging , 2007, Brain and Language.

[13]  Alice J. O'Toole,et al.  Theoretical, Statistical, and Practical Perspectives on Pattern-based Classification Approaches to the Analysis of Functional Neuroimaging Data , 2007, Journal of Cognitive Neuroscience.

[14]  A. P. Dawid,et al.  Generative or Discriminative? Getting the Best of Both Worlds , 2007 .

[15]  Yoshua Bengio,et al.  Classification using discriminative restricted Boltzmann machines , 2008, ICML '08.

[16]  Yaroslav O. Halchenko,et al.  Brain Reading Using Full Brain Support Vector Machines for Object Recognition: There Is No Face Identification Area , 2008, Neural Computation.

[17]  Geoffrey E. Hinton,et al.  A Comparison of Classification Methods for Longitudinal fMRI Studies , 2009, NeuroImage.