Theoretical Analysis of the Optimal Free Responses of Graph-Based SFA for the Design of Training Graphs

Slow feature analysis (SFA) is an unsupervised learning algorithm that extracts slowly varying features from a time series. Graph-based SFA (GSFA) is a supervised extension that can solve regression problems if followed by a post-processing regression algorithm. A training graph specifies arbitrary connections between the training samples. The connections in current graphs, however, only depend on the rank of the involved labels. Exploiting the exact label values makes further improvements in estimation accuracy possible. In this article, we propose the exact label learning (ELL) method to create a graph that codes the desired label explicitly, so that GSFA is able to extract a normalized version of it directly. The ELL method is used for three tasks: (1) We estimate gender from artificial images of human faces (regression) and show the advantage of coding additional labels, particularly skin color. (2) We analyze two existing graphs for regression. (3) We extract compact discriminative features to classify traffic sign images. When the number of output features is limited, a higher classification rate is obtained compared to a graph equivalent to nonlinear Fisher discriminant analysis. The method is versatile, directly supports multiple labels, and provides higher accuracy compared to current graphs for the problems considered.

[1]  Pietro Berkes.  Handwritten Digit Recognition with Nonlinear Fisher Discriminant Analysis , 2005, ICANN.

[2]  Xuelong Li,et al.  Patch Alignment for Dimensionality Reduction , 2009, IEEE Transactions on Knowledge and Data Engineering.

[3]  Laurenz Wiskott,et al.  How to solve classification and regression problems on high-dimensional data with a supervised extension of slow feature analysis , 2013, J. Mach. Learn. Res..

[4]  Henning Sprekeler,et al.  On the Relation of Slow Feature Analysis and Laplacian Eigenmaps , 2011, Neural Computation.

[5]  Laurenz Wiskott,et al.  Gender and Age Estimation from Synthetic Face Images with Hierarchical Slow Feature Analysis , 2010 .

[6]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[7]  Terrence J. Sejnowski,et al.  Slow Feature Analysis: Unsupervised Learning of Invariances , 2002, Neural Computation.

[8]  Wolfgang Maass,et al.  Replacing supervised classification learning by Slow Feature Analysis in spiking neural networks , 2009, NIPS.

[9]  Guodong Guo,et al.  A framework for joint estimation of age, gender and ethnicity on a large database , 2014, Image Vis. Comput..

[10]  Laurenz Wiskott,et al.  Slowness and Sparseness Lead to Place, Head-Direction, and Spatial-View Cells , 2007, PLoS Comput. Biol..

[11]  Sinan Kalkan,et al.  Deep Hierarchies in the Primate Visual Cortex: What Can We Learn for Computer Vision? , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Jürgen Schmidhuber,et al.  Multi-column deep neural network for traffic sign classification , 2012, Neural Networks.

[13]  Laurenz Wiskott,et al.  Learning invariance manifolds , 1998, Neurocomputing.

[14]  P. Berkes.  Pattern Recognition with Slow Feature Analysis , 2005 .

[15]  Laurenz Wiskott,et al.  Heuristic Evaluation of Expansions for Non-linear Hierarchical Slow Feature Analysis , 2011, 2011 10th International Conference on Machine Learning and Applications and Workshops.

[16]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[17]  Johannes Stallkamp,et al.  Detection of traffic signs in real-world images: The German traffic sign detection benchmark , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[18]  Laurenz Wiskott,et al.  Slow Feature Analysis: A Theoretical Analysis of Optimal Free Responses , 2003, Neural Computation.

[19]  Stan Z. Li,et al.  Age Estimation by Multi-scale Convolutional Network , 2014, ACCV.

[20]  Henning Sprekeler,et al.  Nonlinear Supervised Locality Preserving Projections for Visual Pattern Discrimination , 2014, 2014 22nd International Conference on Pattern Recognition.

[21]  Peter Földiák,et al.  Learning Invariance from Transformation Sequences , 1991, Neural Comput..

[22]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[23]  Laurenz Wiskott,et al.  Gender and Age Estimation from Synthetic Face Images , 2010, IPMU.

[24]  Graeme Mitchison,et al.  Removing Time Variation with the Anti-Hebbian Differential Synapse , 1991, Neural Computation.

[25]  Niko Wilbert,et al.  Invariant Object Recognition and Pose Estimation with Slow Feature Analysis , 2011, Neural Computation.