Are unsupervised neural networks ignorant? Sizing the effect of environmental distributions on unsupervised learning

Learning environmental biases is a rational behavior: by using prior odds, Bayesian networks rapidly became a benchmark in machine learning. Moreover, a growing body of evidence now suggests that humans are using base rate information. Unsupervised connectionist networks are used in computer science for machine learning and in psychology to model human cognition, but it is unclear whether they are sensitive to prior odds. In this paper, we show that hard competitive learners are unable to use environmental biases while recurrent associative memories use frequency of exemplars and categories independently. Hence, it is concluded that recurrent associative memories are more useful than hard competitive networks to model human cognition and have a higher potential in machine learning.

[1]  Alan F. Murray,et al.  International Joint Conference on Neural Networks , 1993 .

[2]  Gerd Gigerenzer,et al.  How to Improve Bayesian Reasoning Without Instruction: Frequency Formats , 1995 .

[3]  J. Kruschke,et al.  ALCOVE: an exemplar-based connectionist model of category learning. , 1992, Psychological review.

[4]  Stephen Grossberg,et al.  Adaptive pattern classification and universal recoding: II. Feedback, expectation, olfaction, illusions , 1976, Biological Cybernetics.

[5]  Robert Proulx,et al.  Categorization in unsupervised neural networks: the Eidos model , 1996, IEEE Trans. Neural Networks.

[6]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[7]  Teuvo Kohonen,et al.  Correlation Matrix Memories , 1972, IEEE Transactions on Computers.

[8]  Mounir Boukadoum,et al.  A bidirectional heteroassociative memory for binary and grey-level patterns , 2006, IEEE Transactions on Neural Networks.

[9]  Stephen K. Reed,et al.  Perceptual vs conceptual categorization , 1973, Memory & cognition.

[10]  Stephen A. Ritz,et al.  Distinctive features, categorical perception, and probability learning: some applications of a neural model , 1977 .

[11]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[12]  John R. Anderson,et al.  The Adaptive Nature of Human Categorization. , 1991 .

[13]  Morton Wagman.  Problem-Solving Processes in Humans and Computers: Theory and Research in Psychology and Artificial Intelligence , 2001 .

[14]  Terrence J. Sejnowski,et al.  Unsupervised Learning , 2018, Encyclopedia of GIS.

[15]  Sun-Yuan Kung,et al.  Principal Component Neural Networks: Theory and Applications , 1996 .

[16]  L. Rips.  Similarity, typicality, and categorization , 1989 .

[17]  R. Nosofsky,et al.  An exemplar-based random walk model of speeded classification. , 1997, Psychological review.

[18]  Juha Karhunen,et al.  Principal component neural networks — Theory and applications , 1998, Pattern Analysis and Applications.

[19]  Gèunther Palm,et al.  Neural Assemblies: An Alternative Approach to Artificial Intelligence , 1982 .

[20]  Robert Proulx,et al.  A self-scaling procedure in unsupervised correlational neural networks , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[21]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[22]  S. Ross.  A First Course in Probability , 1977 .

[23]  R. Golden.  The :20Brain-state-in-a-box Neural model is a gradient descent algorithm , 1986 .

[24]  Sébastien Hélie,et al.  ADAPTIVE CATEGORIZATION AND NEURAL NETWORKS , 2005 .

[25]  Konrad Paul Kording,et al.  Bayesian integration in sensorimotor learning , 2004, Nature.

[26]  S. Grossberg,et al.  Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors , 1976, Biological Cybernetics.

[27]  James A. Freeman,et al.  Simulating neural networks - with Mathematica , 1993 .

[28]  Liva Nohre,et al.  Instance Frequency, Categorization, and the Modulating Effect of Experience , 1991 .

[29]  Peter Norvig,et al.  Artificial intelligence - a modern approach, 2nd Edition , 2003, Prentice Hall series in artificial intelligence.

[30]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[31]  M. Posner,et al.  On the genesis of abstract ideas. , 1968, Journal of experimental psychology.

[32]  R M Nosofsky,et al.  Similarity-scaling studies of dot-pattern classification and recognition. , 1992, Journal of experimental psychology. General.

[33]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[34]  Henri Cohen,et al.  Handbook of categorization in cognitive science , 2005 .

[35]  R. Nosofsky.  Similarity, frequency, and category representations. , 1988 .

[36]  Stephen Wolfram,et al.  The Mathematica Book , 1996 .

[37]  BART KOSKO,et al.  Bidirectional associative memories , 1988, IEEE Trans. Syst. Man Cybern..

[38]  L. Cosmides,et al.  Are humans good intuitive statisticians after all? Rethinking some conclusions from the literature on judgment under uncertainty , 1996, Cognition.

[39]  Robert Proulx,et al.  NDRAM: nonlinear dynamic recurrent associative memory for learning bipolar and nonbipolar correlated patterns , 2005, IEEE Transactions on Neural Networks.

[40]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[41]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[42]  David Zipser,et al.  Feature Discovery by Competive Learning , 1986, Cogn. Sci..

[43]  Steven J. Nowlan,et al.  Maximum Likelihood Competitive Learning , 1989, NIPS.

[45]  N. Chater,et al.  Rational models of cognition , 1998 .

[46]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[47]  J. Kruschke.  Base rates in category learning. , 1996, Journal of experimental psychology. Learning, memory, and cognition.

[48]  GrossbergS..  Adaptive pattern classification and universal recoding , 1976 .

[49]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[50]  R. Nosofsky.  Tests of an exemplar model for relating perceptual classification and recognition memory. , 1991, Journal of experimental psychology. Human perception and performance.

[51]  Richard,et al.  The “Brain-State-in-a-Box” Neural Model Is a Gradient Descent Algorithm , 2003 .

[52]  J. Kruschke,et al.  Rules and exemplars in category learning. , 1998, Journal of experimental psychology. General.

[53]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.