Entropy Driven Artificial Neuronal Networks and Sensorial Representation: A Proposal

Abstract A hierarchical Artificial Neuronal Network (ANN) is proposed as a model sensorium wherein feedback is allowed to modify the categorization abilities of the system. In this way, the original representation, being abstract and precategorical, is refined, yielding a more concrete representation. As thermodynamical entropy is a hierarchical invariant and an explicitly time dependent and compact measure of state dynamics, it is chosen as feedback measure. The main features of the network are shown to be plausible from the point of view of the physiology and anatomy of the visual system of cats and primates and one of these, double-layered maps performing combinatorial processing and evaluation, respectively, is illustrated by simulations in the orientation domain.

[1]  I. Richardson,et al.  A dynamical field theory for dissipative systems: The hierarchical structure of field thermodynamics , 1978 .

[2]  C. Desoer,et al.  Tellegen's theorem and thermodynamic inequalities. , 1971, Journal of theoretical biology.

[3]  Steven W. Zucker,et al.  Early orientation selection: Tangent fields and the dimensionality of their support , 1985, Comput. Vis. Graph. Image Process..

[4]  K. Martin,et al.  Cortical circuitry underlying inhibitory processes in cat area 17 , 1985 .

[5]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[6]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[7]  S. Amari,et al.  Competition and Cooperation in Neural Nets , 1982 .

[8]  T. Poggio,et al.  The synaptic veto mechanism: does it underlie direction and orientation selectivity in the visual cortex , 1985 .

[9]  Professor Dr. Guy A. Orban.  Neuronal Operations in the Visual Cortex , 1983, Studies of Brain Function.

[10]  Jerome A. Feldman,et al.  Connectionist Models and Their Properties , 1982, Cogn. Sci..

[11]  P. Mazur,et al.  Non-equilibrium thermodynamics, , 1963 .

[12]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[13]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  H. Haken.  Synergetics , 1988, IEEE Circuits and Devices Magazine.

[15]  J. Marroquín.  Surface Reconstruction Preserving Discontinuities , 1984 .

[16]  Christopher L. Scofield.  A Mean Field Theory of Layer IV of Visual Cortex and Its Application to Artificial Neural Networks , 1987, NIPS.

[17]  Terrence J. Sejnowski,et al.  NETtalk: a parallel network that learns to read aloud , 1988 .

[18]  Marc M. Van Hulle,et al.  A field approach to higher order ANNs for sensorial representation , 1989 .

[19]  D. O. Hebb,et al.  The organization of behavior , 1988 .

[20]  E Harth,et al.  The inversion of sensory processing by feedback pathways: a model of visual cognitive functions. , 1987, Science.

[21]  D. Diamond,et al.  Physiological plasticity in auditory cortex: Rapid induction by learning , 1987, Progress in Neurobiology.

[22]  M. Arbib,et al.  Conceptual models of neural organization. , 1974, Neurosciences Research Program bulletin.

[23]  T Poggio,et al.  Parallel integration of vision modules. , 1988, Science.

[24]  L C Katz,et al.  Local circuitry of identified projection neurons in cat visual cortex brain slices , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[26]  J J Hopfield,et al.  Neural computation by concentrating information in time. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[27]  J. Kaas,et al.  A representation of the visual field in the caudal third of the middle tempral gyrus of the owl monkey (Aotus trivirgatus). , 1971, Brain research.

[28]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[29]  T. Wiesel,et al.  Relationships between horizontal interactions and functional architecture in cat striate cortex as revealed by cross-correlation analysis , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[30]  S Grossberg,et al.  Cortical dynamics of three-dimensional form, color, and brightness perception: II. Binocular theory , 1988, Perception & psychophysics.

[31]  R Linsker,et al.  From basic network principles to neural architecture: emergence of spatial-opponent cells. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[32]  L. Peusner,et al.  Studies in network thermodynamics , 1986 .

[33]  P. Glansdorff,et al.  Thermodynamic theory of structure, stability and fluctuations , 1971 .

[34]  James A. Anderson,et al.  Cognitive and psychological computation with neural models , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[35]  C Koch,et al.  Analog "neuronal" networks in early vision. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[36]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[37]  Geoffrey E. Hinton,et al.  Distributed Representations , 1986, The Philosophy of Artificial Intelligence.

[38]  Pierre Peretto,et al.  Stochastic Dynamics of Neural Networks , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[39]  S. Geman.  SOME AVERAGING AND STABILITY RESULTS FOR RANDOM DIFFERENTIAL EQUATIONS , 1979 .

[40]  C. Blakemore,et al.  Spatial and temporal selectivity in the suprasylvian visual cortex of the cat , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[41]  E. Adelson,et al.  The analysis of moving visual patterns , 1985 .

[42]  S. Zeki.  Interhemispheric connections of prestriate cortex in monkey. , 1970, Brain research.

[43]  Steven W. Zucker,et al.  From orientation selection to optical flow , 1987, Comput. Vis. Graph. Image Process..

[44]  H. Kennedy,et al.  Topography of the afferent connectivity of area 17 in the macaque monkey: A double‐labelling study , 1986, The Journal of comparative neurology.

[45]  P Erdi,et al.  Hierarchial thermodynamic approach to the brain. , 1983, The International journal of neuroscience.

[46]  John H. R. Maunsell,et al.  Visual processing in monkey extrastriate cortex. , 1987, Annual review of neuroscience.

[47]  B Kosko,et al.  Adaptive bidirectional associative memories. , 1987, Applied optics.

[48]  I. Prigogine,et al.  From Being to Becoming: Time and Complexity in the Physical Sciences , 1982 .

[49]  John G. Daugman,et al.  Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression , 1988, IEEE Trans. Acoust. Speech Signal Process..

[50]  Norberto M. Grzywacz,et al.  A computational theory for the perception of coherent visual motion , 1988, Nature.

[51]  Shun-ichi Amari,et al.  Field theory of self-organizing neural nets , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[52]  Geoffrey E. Hinton,et al.  A general framework for parallel distributed processing , 1986 .

[53]  P. Érdi.  From Brain Theory to Future Generations Computer Systems , 1988 .

[54]  R. Erickson,et al.  The Across-Fiber Pattern Theory: An Organizing Principle for Molar Neural Function , 1982 .

[55]  Tomaso Poggio,et al.  Computational vision and regularization theory , 1985, Nature.

[56]  Local excitatory circuits in area 17 of the cat , 1985 .

[57]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[58]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[59]  Parvati Dev,et al.  Perception of Depth Surfaces in Random-Dot Stereograms: A Neural Model , 1975, Int. J. Man Mach. Stud..

[60]  D. Hubel,et al.  Receptive fields of single neurones in the cat's striate cortex , 1959, The Journal of physiology.