Estimating Imagined Images from Brain Activities via Visual Question Answering

Investigating human mental contents has been a topic for many years, but its ambiguous property has made the analysis difficult. We propose a neural decoding method via a machine learning model that predicts the imagined content based on measuring brain activity in this paper. This technique uses brain activity and computer vision models to discover the association between human functional magnetic resonance imaging (fMRI) activity and imagined contents. Decoding models based on neural networks learned using stimulus-induced brain activity in the visual cortex region showed an accurate estimation of the content. We provide a means of revealing subjective mental content by analysis with visual question answering. This result shows that a mental experience relates to brain activity patterns.