A stereoscopic system for viewing the temporal evolution of brain activity clusters in response to linguistic stimuli

In this paper, we present a novel application, 3D+Time Brain View, for the stereoscopic visualization of functional Magnetic Resonance Imaging (fMRI) data gathered from participants exposed to unfamiliar spoken languages. An analysis technique based on Independent Component Analysis (ICA) is used to identify statistically significant clusters of brain activity and their changes over time during different testing sessions. That is, our system illustrates the temporal evolution of participants' brain activity as they are introduced to a foreign language through displaying these clusters as they change over time. The raw fMRI data is presented as a stereoscopic pair in an immersive environment utilizing passive stereo rendering. The clusters are presented using a ray casting technique for volume rendering. Our system incorporates the temporal information and the results of the ICA into the stereoscopic 3D rendering, making it easier for domain experts to explore and analyze the data.

[1]  Vincent Schmithorst,et al.  A group independent component analysis of covert verb generation in children: A functional magnetic resonance imaging study , 2010, NeuroImage.

[2]  Matthew J. McAuliffe,et al.  Medical Image Processing, Analysis and Visualization in clinical research , 2001, Proceedings 14th IEEE Symposium on Computer-Based Medical Systems. CBMS 2001.

[3]  Helwig Hauser,et al.  Visualization and Visual Analysis of Multifaceted Scientific Data: A Survey , 2013, IEEE Transactions on Visualization and Computer Graphics.

[4]  Marc Levoy,et al.  Efficient ray tracing of volume data , 1990, TOGS.

[5]  James T. Kajiya,et al.  Ray tracing volume densities , 1984, SIGGRAPH.

[6]  Nelson L. Max,et al.  Optical Models for Direct Volume Rendering , 1995, IEEE Trans. Vis. Comput. Graph..

[7]  T. Sejnowski,et al.  Human Brain Mapping 6:368–372(1998) � Independent Component Analysis of fMRI Data: Examining the Assumptions , 2022 .

[8]  Kyle R. Almryde,et al.  The nature of the language input affects brain activation during learning from a natural language , 2015, Journal of Neurolinguistics.

[9]  Lori L. Holt,et al.  Expertise with Artificial Nonspeech Sounds Recruits Speech-Sensitive Cortical Regions , 2009, The Journal of Neuroscience.

[10]  Richard N. Aslin,et al.  The neural correlates of statistical learning in a word segmentation task: An fMRI study , 2013, Brain and Language.

[11]  Scott T. Grafton,et al.  Anatomical Substrates of Visual and Auditory Miniature Second-language Learning , 2006, Journal of Cognitive Neuroscience.

[12]  Stefano F. Cappa,et al.  Imaging semantics and syntax , 2012, NeuroImage.

[13]  Thomas Ertl,et al.  Simultaneous visualization of anatomical and functional 3D data by combining volume rendering and flow visualization , 2007, SPIE Medical Imaging.

[14]  Mark W. Greenlee,et al.  Distinct patterns of functional and structural neuroplasticity associated with learning Morse code , 2010, NeuroImage.

[15]  Christian F. Beckmann,et al.  Modelling with independent components , 2012, NeuroImage.

[16]  Ingrid Scholl,et al.  Comparing GPU-based multi-volume ray casting techniques , 2010, Computer Science - Research and Development.

[17]  Vincent Schmithorst,et al.  Cognitive modules utilized for narrative comprehension in children: a functional magnetic resonance imaging study , 2006, NeuroImage.

[18]  Shengyong Chen,et al.  Plane-Based Sampling for Ray Casting Algorithm in Sequential Medical Images , 2013, Comput. Math. Methods Medicine.

[19]  Cathy J. Price,et al.  A review and synthesis of the first 20 years of PET and fMRI studies of heard speech, spoken language and reading , 2012, NeuroImage.

[20]  Klaus Mueller,et al.  Overview of Volume Rendering , 2005, The Visualization Handbook.

[21]  Vincent J Schmithorst,et al.  Object identification and lexical/semantic access in children: A functional magnetic resonance imaging study of word‐picture matching , 2007, Human brain mapping.

[22]  C. Price.  The anatomy of language: a review of 100 fMRI studies published in 2009 , 2010, Annals of the New York Academy of Sciences.