A method for real-time visual stimulus selection in the study of cortical object perception

The properties utilized by visual object perception in the mid- and high-level ventral visual pathway are poorly understood. To better establish and explore possible models of these properties, we adopt a data-driven approach in which we repeatedly interrogate neural units using functional Magnetic Resonance Imaging (fMRI) to establish each unit's image selectivity. This approach to imaging necessitates a search through a broad space of stimulus properties using a limited number of samples. To more quickly identify the complex visual features underlying human cortical object perception, we implemented a new functional magnetic resonance imaging protocol in which visual stimuli are selected in real-time based on BOLD responses to recently shown images. Two variations of this protocol were developed, one relying on natural object stimuli and a second based on synthetic object stimuli, both embedded in feature spaces based on the complex visual properties of the objects. During fMRI scanning, we continuously controlled stimulus selection in the context of a real-time search through these image spaces in order to maximize neural responses across pre-determined 1cm(3) rain regions. Elsewhere we have reported the patterns of cortical selectivity revealed by this approach (Leeds et al., 2014). In contrast, here our objective is to present more detailed methods and explore the technical and biological factors influencing the behavior of our real-time stimulus search. We observe that: 1) Searches converged more reliably when exploring a more precisely parameterized space of synthetic objects; 2) real-time estimation of cortical responses to stimuli is reasonably consistent; 3) search behavior was acceptably robust to delays in stimulus displays and subject motion effects. Overall, our results indicate that real-time fMRI methods may provide a valuable platform for continuing study of localized neural selectivity, both for visual object representation and beyond.

[1]  Keiji Tanaka.  Columns for complex visual object features in the inferotemporal cortex: clustering of cells with similar but slightly different stimulus selectivities. , 2003, Cerebral cortex.

[2]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[3]  John A. Pyles,et al.  Exploration of complex visual feature spaces for object perception , 2014, Front. Comput. Neurosci..

[4]  A. H. Mohamed,et al.  Adaptive Kalman Filtering for INS/GPS , 1999 .

[5]  Ivanei E. Bramati,et al.  Real-Time fMRI Pattern Decoding and Neurofeedback Using FRIEND: An FSL-Integrated BCI Toolbox , 2013, PloS one.

[6]  I. C. Whitfield,et al.  Chapter 5 – THE NEURAL CODE , 1978 .

[7]  M. F. Cardoso,et al.  The simplex-simulated annealing approach to continuous non-linear optimization , 1996 .

[8]  D G Pelli,et al.  Pixel independence: measuring spatial interactions on a CRT display. , 1997, Spatial vision.

[9]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[10]  Eric T. Carlson,et al.  Medial Axis Shape Coding in Macaque Inferotemporal Cortex , 2012, Neuron.

[11]  Jascha D. Swisher,et al.  Multiscale Pattern Analysis of Orientation-Selective Activity in the Primary Visual Cortex , 2010, The Journal of Neuroscience.

[12]  Eero P. Simoncelli,et al.  A functional and perceptual signature of the second visual area in primates , 2013, Nature Neuroscience.

[13]  Keiji Tanaka,et al.  Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey , 2008, Neuron.

[14]  Michel Vidal-Naquet,et al.  Visual features of intermediate complexity and their use in classification , 2002, Nature Neuroscience.

[15]  P. Mahalanobis.  On the generalized distance in statistics , 1936 .

[16]  T. Poggio,et al.  A model of V4 shape selectivity and invariance. , 2007, Journal of neurophysiology.

[17]  Eric T. Carlson,et al.  A neural code for three-dimensional object shape in macaque inferotemporal cortex , 2008, Nature Neuroscience.

[18]  Tom Michael Mitchell,et al.  A Neurosemantic Theory of Concrete Noun Representation Based on the Underlying Brain Codes , 2010, PloS one.

[19]  Jack L. Gallant,et al.  A Continuous Semantic Space Describes the Representation of Thousands of Object and Action Categories across the Human Brain , 2012, Neuron.

[20]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[21]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[22]  Takeo Watanabe,et al.  Perceptual learning incepted by decoded fMRI neurofeedback without stimulus presentation , 2012 .

[23]  Yan Ma,et al.  Adaptive Kalman filtering for real-time mapping of the visual field , 2012, NeuroImage.

[24]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[25]  G. Seber.  Multivariate observations / G.A.F. Seber , 1983 .

[26]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[27]  Heinrich H. Bülthoff,et al.  Learned Non-Rigid Object Motion is a View-Invariant Cue to Recognizing Novel Objects , 2012, Front. Comput. Neurosci..

[28]  Daniel Leeds,et al.  Searching for the Visual Components of Object Perception , 2013 .

[29]  John A. Pyles,et al.  Comparing visual representations across human fMRI and computational vision. , 2013, Journal of vision.

[30]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[31]  Shimon Edelman,et al.  Renewing the respect for similarity , 2012, Front. Comput. Neurosci..

[32]  D. Simons,et al.  Detecting Changes in Novel, Complex Three-dimensional Objects , 2000 .