Making a Low-Dimensional Representation Suitable for Diverse Tasks

We introduce a new approach to the training of classifiers for performance on multiple tasks. The proposed hybrid training method leads to improved generalization via a better low-dimensional representation of the problem space. The quality of the representation is assessed by embedding it in a 2D space using multidimensional scaling, allowing a direct visualization of the results. The performance of the approach is demonstrated on a highly nonlinear image classification task.

[1]  Shimon Edelman,et al.  Receptive field spaces and class-based generalization from a single view in face recognition , 1995 .

[2]  G. Logan.  Toward an instance theory of automatization. , 1988 .

[3]  I. Borg.  Multidimensional similarity structure analysis , 1987 .

[4]  J. Brigham.  The Influence of Race on Face Recognition , 1986 .

[5]  R. Bellman,et al.  V. Adaptive Control Processes , 1964 .

[6]  John W. Sammon,et al.  A Nonlinear Mapping for Data Structure Analysis , 1969, IEEE Transactions on Computers.

[7]  D. L. Hintzman.  Twenty-five years of learning and memory: was the cognitive revolution a mistake? , 1993 .

[8]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[9]  D. Bairaktaris,et al.  Transfer of learning in backpropagation networks and in related neural network models , 1995 .

[10]  Nathan Intrator,et al.  Objective function formulation of the BCM theory of visual cortical plasticity: Statistical connections, stability conditions , 1992, Neural Networks.

[11]  R. Tibshirani,et al.  Combining Estimates in Regression and Classification , 1996 .

[12]  Nathan Intrator,et al.  Bootstrapping with Noise: An Effective Regularization Technique , 1996, Connect. Sci..

[13]  S. Edelman.  Representation of Similarity in 3D Object Discrimination , 1995 .

[14]  Tal Grossman,et al.  Use of Bad Training Data for Better Predictions , 1993, NIPS.

[15]  Shimon Edelman,et al.  Representation of Similarity in Three-Dimensional Object Discrimination , 1995, Neural Computation.

[16]  S. Edelman,et al.  Explorations of Shape Space , 1995 .

[17]  J. Kruskal.  Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis , 1964 .

[18]  Gérard Dreyfus,et al.  Pairwise Neural Network Classifiers with Probabilistic Outputs , 1994, NIPS.

[19]  Jonathan Baxter,et al.  Learning internal representations , 1995, COLT '95.

[20]  R. Shepard,et al.  Toward a universal law of generalization for psychological science. , 1987, Science.

[21]  Michael Gasser,et al.  Transfer in a Connectionist Model of the Acquisition of Morphology , 1995, ArXiv.

[22]  Lorien Y. Pratt,et al.  Transferring previously learned back-propagation neural networks to new learning tasks , 1993 .

[23]  Clifford A. Pickover,et al.  Computers, Pattern, Chaos and Beauty , 1990 .

[24]  Nathan Intrator,et al.  Combining Exploratory Projection Pursuit and Projection Pursuit Regression with Application to Neural Networks , 1993, Neural Computation.

[25]  Joachim M. Buhmann,et al.  Multidimensional Scaling and Data Clustering , 1994, NIPS.

[26]  R. Shepard.  Metric structures in ordinal data , 1966 .

[27]  A. Householder,et al.  Discussion of a set of points in terms of their mutual distances , 1938 .

[28]  W T Maddox,et al.  Comparing decision bound and exemplar models of categorization , 1993, Perception & psychophysics.

[29]  Halbert White,et al.  Bootstrapping Confidence Intervals for Clinical Input Variable Effects in a Network Trained to Identify the Presence of Acute Myocardial Infarction , 1995, Neural Computation.

[30]  Yann LeCun,et al.  Tangent Prop - A Formalism for Specifying Selected Invariances in an Adaptive Network , 1991, NIPS.

[31]  Rich Caruana,et al.  Learning Many Related Tasks at the Same Time with Backpropagation , 1994, NIPS.

[32]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[33]  M. Stone,et al.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[34]  R N Shepard,et al.  Multidimensional Scaling, Tree-Fitting, and Clustering , 1980, Science.

[35]  Sebastian Thrun,et al.  Learning One More Thing , 1994, IJCAI.

[36]  A F Kramer,et al.  Development and transfer of automatic processing. , 1990, Journal of experimental psychology. Human perception and performance.