Exploring EEG for Object Detection and Retrieval

This paper explores the potential for using Brain Computer Interfaces (BCI) as a relevance feedback mechanism in content-based image retrieval. Several experiments are performed using a rapid serial visual presentation (RSVP) of images at different rates (5Hz and 10Hz) on 8 users with different degrees of familiarization with BCI and the dataset. We compare the feedback from the BCI and mouse-based interfaces in a subset of TRECVid images, finding that, when users have limited time to annotate the images, both interfaces are comparable in performance. Comparing our best users in a retrieval task, we found that EEG-based relevance feedback can outperform mouse-based feedback.

[1]  Thomas S. Huang,et al.  Relevance feedback in image retrieval: A comprehensive review , 2003, Multimedia Systems.

[2]  Noel E. O'Connor,et al.  Insight Centre for Data Analytics (DCU) at TRECVid 2014: Instance Search and Semantic Indexing Tasks , 2014, TRECVID.

[3]  Alan F. Smeaton,et al.  Object Segmentation in Images using EEG Signals , 2014, ACM Multimedia.

[4]  N. Bigdely-Shamlo,et al.  Brain Activity-Based Image Classification From Rapid Serial Visual Presentation , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[5]  Rong Yan,et al.  Exploring the Synergy of Humans and Machines in Extreme Video Retrieval , 2006, CIVR.

[6]  Jun Yang,et al.  A general framework for classifier adaptation and its applications in multimedia , 2009 .

[7]  Xiang Ji,et al.  Representing and Retrieving Video Shots in Human-Centric Brain Imaging Space , 2013, IEEE Transactions on Image Processing.

[8]  Marina Schmid,et al.  An Introduction To The Event Related Potential Technique , 2016 .

[9]  Haim H. Permuter,et al.  Mutual relevance feedback for multimodal query formulation in video retrieval , 2005, MIR '05.

[10]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[11]  Rong Yan,et al.  Negative pseudo-relevance feedback in content-based video retrieval , 2003, MULTIMEDIA '03.

[12]  Edward Y. Chang,et al.  Support vector machine active learning for image retrieval , 2001, MULTIMEDIA '01.

[13]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[14]  Jun Yang,et al.  A framework for classifier adaptation and its applications in concept detection , 2008, MIR '08.

[15]  Alan F. Smeaton,et al.  Optimising the number of channels in EEG-augmented image search , 2011, BCS HCI.

[16]  Shih-Fu Chang,et al.  Brain state decoding for rapid image retrieval , 2009, ACM Multimedia.

[17]  Misha Pavel,et al.  A framework for rapid visual image search using single-trial brain evoked responses , 2011, Neurocomputing.