Semi-supervised Discriminative CCA for Estimating Viewed Image Categories from fMRI Data

This paper presents a method that estimates viewed image categories from functional magnetic resonance imaging (fMRI) data via semi-supervised discriminative canonical correlation analysis (Semi-DCCA). We newly derive Semi-DCCA that enables direct comparison of fMRI data and visual features extracted from viewed images while taking into account the class information and additional visual features to avoid overfitting. The proposed method enables estimation of image categories from fMRI data measured when subjects view images by comparing fMRI data with visual features through Semi-DCCA. Experimental results show that Semi-DCCA can improve estimation performance of the viewed image categories.

[1]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[2]  E Donchin,et al.  Brain-computer interface technology: a review of the first international meeting. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[3]  Pengfei Shi,et al.  Discriminative Canonical Correlation Analysis with Missing Samples , 2009, 2009 WRI World Congress on Computer Science and Information Engineering.

[4]  Tomoyasu Horikawa,et al.  Generic decoding of seen and imagined objects using hierarchical visual features , 2015, Nature Communications.

[5]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Masa-aki Sato,et al.  Sparse estimation automatically selects voxels relevant for the decoding of fMRI activity patterns , 2008, NeuroImage.

[7]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Hirokazu Kameoka,et al.  SemiCCA: Efficient Semi-supervised Learning of Canonical Correlations , 2010, International Conference on Pattern Recognition.

[9]  Byoung-Kyong Min,et al.  Neuroimaging-based approaches in the brain-computer interface. , 2010, Trends in biotechnology.

[10]  Miki Haseyama,et al.  Estimation of Viewed Image Categories via CCA Using Human Brain Activity , 2018, 2018 IEEE 7th Global Conference on Consumer Electronics (GCCE).

[11]  Keum-Shik Hong,et al.  Hybrid EEG-NIRS based BCI for quadcopter control , 2015, 2015 54th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE).