Learning and Coding in Neural Networks

How do neurons process, encode, and transmit information? This is a central question in neuroscience, referred to as the “neural coding” issue. There is a broad agreement that spikes are the basic currency for transmitting information between neurons, the reason being that they can propagate over large distances. How the brain actually uses them to encode information remains more controversial. In particular, the issue of the relevant timescales is intensely debated: do individual spike times matter, or does one need counting them over a long time window (say a few tens of milliseconds) to obtain a meaningful, reliable quantity? (see also Chapter 17). Typically electro-physiologists have assessed this question by recording in vivo in primary sensory or motor areas, and used information theory or decoding techniques (Quiroga and Panzeri, 2009) (see also Chapter 8) to estimate the amount of information about respectively the stimulus, or the motor command, contained in a given feature of the neural activity. Both techniques require multiple trials and make the implicit hypothesis that the recorded neurons only encode this stimulus (respectively motor command)—the remaining variability when controlling the stimulus (respectively motor command) is thus treated as noise. For higher-order neurons (i.e., farther away from the sensory input or motor output), this approach should be used with caution, because we do not know what the neurons encode. Thus the variability in the activity, instead of being mere noise, could come from uncontrolled variables. As Barlow wrote about neural responses in 1972, “their apparently erratic behavior was caused by our ignorance, not the neuron’s incompetence” (Barlow, 1972). In this chapter, we are using a completely different, theoretical, approach to tackle the neural coding issue: the set of candidate neural codes can be narrowed down by two main constraints. First, a neural code should be decodable by downstream neurons (Dayan and Abbott, 2001). This problem is often ignored when using the two above-mentioned approaches: information theory completely ignores the decoder, and decoding approaches often use nonbiologically plausible decoders such as support vector machines. However, this decoding problem is crucial: it is not because there is information in a certain activity feature that it can indeed be used by the brain. It may well be an epiphenomenon and the brain may extract the same information from another feature of the neuronal activity, or from other neurons. One has to show how downstream neurons can respond selectively to the candidate features. Q1 26