The superior temporal sulcus and the N170 during face processing: Single trial analysis of concurrent EEG–fMRI

Face-selective neural signals have been reliably identified using both EEG and fMRI studies. These consist of the N170 component, a neural response peaking approximately 170ms after a face is presented, and face-selective activations in the fusiform face area (FFA), the occipital face area (OFA), and the superior temporal sulcus (STS). As most neuroimaging studies examine these face-selective processes separately, the relationship between the N170 neural response and activation in the fusiform gyrus is still debated. In this study, we concurrently measured EEG and fMRI responses to upright faces, inverted faces, and objects to examine this association. We introduce a method for single-trial estimation of N170 amplitudes and correlation of the trial-by-trial variation in N170 neural responses with fMRI BOLD responses. For upright faces, BOLD responses in the right STS were negatively correlated with N170 amplitudes, showing greater activation on trials in which N170 amplitudes were larger (more negative). For inverted faces, a medial region of the fusiform gyrus (mFG) was positively correlated with N170 amplitudes, showing greater activation on trials in which N170 amplitudes were smaller (less negative). This result points to the STS as a crucial region for generating the N170 associated with face perception, and suggests that the mFG is additionally recruited for processing inverted faces, particularly on trials in which N170 is small. Despite the different time resolution of fMRI and EEG signals, our single-trial estimation and EEG-fMRI correlation method can reveal associations between activation in face-selective brain regions and neural processes at 170ms associated with face perception.