Visual Categorization with Random Projection

Abstract Humans learn categories of complex objects quickly and from a few examples. Random projection has been suggested as a means to learn and categorize efficiently. We investigate how random projection affects categorization by humans and by very simple neural networks on the same stimuli and categorization tasks, and how this relates to the robustness of categories. We find that (1) drastic reduction in stimulus complexity via random projection does not degrade performance in categorization tasks by either humans or simple neural networks, (2) human accuracy and neural network accuracy are remarkably correlated, even at the level of individual stimuli, and (3) the performance of both is strongly indicated by a natural notion of category robustness.

[1]  J. Read,et al.  The place of human psychophysics in modern neuroscience , 2015, Neuroscience.

[2]  Nir Shavit,et al.  Johnson-Lindenstrauss Compression with Neuroscience-Based Constraints , 2014, ArXiv.

[3]  Nir Shavit,et al.  Sparse sign-consistent Johnson–Lindenstrauss matrices: Compression with neuroscience-based constraints , 2014, Proceedings of the National Academy of Sciences.

[4]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[5]  Joshua B. Tenenbaum,et al.  One-shot learning by inverting a compositional causal process , 2013, NIPS.

[6]  Fei Xu,et al.  Infants Are Rational Constructivist Learners , 2013 .

[7]  S. Brosnan,et al.  Visual Processing Speed in Capuchin Monkeys (Cebus apella) and Rhesus Macaques (Macaca mulatta) , 2013 .

[8]  Tara N. Sainath,et al.  FUNDAMENTAL TECHNOLOGIES IN MODERN SPEECH RECOGNITION Digital Object Identifier 10.1109/MSP.2012.2205597 , 2012 .

[9]  Piotr Indyk,et al.  Approximate Nearest Neighbor: Towards Removing the Curse of Dimensionality , 2012, Theory Comput..

[10]  Zhenghao Chen,et al.  On Random Weights and Unsupervised Feature Learning , 2011, ICML.

[11]  Charles Kemp,et al.  How to Grow a Mind: Statistics, Structure, and Abstraction , 2011, Science.

[12]  Sanjoy Dasgupta,et al.  Learning the structure of manifolds using random projections , 2007, NIPS.

[13]  Chinmay Hegde,et al.  Random Projections for Manifold Learning , 2007, NIPS.

[14]  Hiroshi Makino,et al.  Discrimination of artificial categories structured by family resemblances: a comparative study in people (Homo sapiens) and pigeons (Columba livia). , 2007, Journal of comparative psychology.

[15]  Chinmay Hegde,et al.  Efficient Machine Learning Using Random Projections , 2007 .

[16]  Santosh S. Vempala,et al.  Kernels as features: On kernels, margins, and low-dimensional mappings , 2006, Machine Learning.

[17]  Amy E Booth,et al.  Object Function and Categorization in Infancy: Two Mechanisms of Facilitation. , 2006 .

[18]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[19]  Pietro Perona,et al.  One-shot learning of object categories , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Santosh S. Vempala,et al.  An algorithmic theory of learning: Robust concepts and random projection , 1999, Machine Learning.

[21]  George Bebis,et al.  Face recognition experiments with random projection , 2005, SPIE Defense + Commercial Sensing.

[22]  Lisa M. Oakes,et al.  Early Category and Concept Development: Making Sense of the Blooming, Buzzing Confusion , 2008 .

[23]  Santosh S. Vempala,et al.  The Random Projection Method , 2005, DIMACS Series in Discrete Mathematics and Theoretical Computer Science.

[24]  Sanjoy Dasgupta,et al.  An elementary proof of a theorem of Johnson and Lindenstrauss , 2003, Random Struct. Algorithms.

[25]  Dan Roth,et al.  Margin Distribution and Learning , 2003, ICML.

[26]  Michael I. Jordan,et al.  The Handbook of Brain Theory and Neural Networks , 2002 .

[27]  Samy Bengio,et al.  The Handbook of Brain Theory and Neural Networks , 2002 .

[28]  Barry J. Richmond,et al.  Consistency of Encoding in Monkey Visual Cortex , 2001, The Journal of Neuroscience.

[29]  Heikki Mannila,et al.  Random projection in dimensionality reduction: applications to image and text data , 2001, KDD '01.

[30]  P. Fldik,et al.  The Speed of Sight , 2001, Journal of Cognitive Neuroscience.

[31]  Sanjoy Dasgupta,et al.  Experiments with Random Projection , 2000, UAI.

[32]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[33]  J. D. Smith,et al.  Thirty categorization results in search of a model. , 2000, Journal of experimental psychology. Learning, memory, and cognition.

[34]  Sanjoy Dasgupta,et al.  Learning mixtures of Gaussians , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[35]  Peter M. Vishton,et al.  Rule learning by seven-month-old infants. , 1999, Science.

[36]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[37]  Piotr Indyk,et al.  Approximate nearest neighbors: towards removing the curse of dimensionality , 1998, STOC '98.

[38]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[39]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[40]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[41]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[42]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .