Neurons That Confuse Mirror-Symmetric Object Views

AbstractNeurons in inferotemporal cortex that respond similarly to many pairsof mirror-symmetric images { for example, 45  and 45  views of thesame face { have often been reported. The phenomenon seemed to be aninteresting oddity. However, the same phenomenon has also emerged insimple hierarchical models of the ventral stream. Here we state a theoremcharacterizing sucient conditions for this curious invariance to occurin a rather large class of hierarchical networks and demonstrate it withsimulations. Neurons in inferotemporal (IT) cortex { the highest visual area in the ventralvisual pathway { respond with varying degrees of selectivity to a speci c patternsuch as one particular view of an object. In some cases, single cells respondingin a similar way to many pairs of images, each representing an object view andits mirror-symmetric counterpart, have been reported (Logothetis et al., 1995),as shown in g. 1. Similar tuning has also been observed for some units in ahierarchical model of cortex, as shown in g. 2.This property seems puzzling and so far appeared to originate from an acci-dental tuning of neurons in IT to features in common between views and theirmirror-symmetric versions. Without discounting this possibility, we focus hereon establishing an alternative { and quite possibly, complementary { explana-tion. We argue that it may (also) reect a property of a large class of hierarchicalarchitectures. Recently, we have been developing a general theory (Smale et al.,2010) of hierarchical feedforward architectures such as the model of visual cor-tex shown in g. 3. Within this theoretical framework we have established1