MMG: A Learning Game Platform for Understanding and Predicting Human Recall Memory

How humans infer probable information from the limited observed data? How they are able to build on little knowledge about the context in hand? Is the human memory repeatedly constructing and reconstructing the events that are being recalled? These are a few questions that we are interested in answering with our multimodal memory game (MMG) platform that studies human memory and their behaviors while watching and remembering TV dramas for a better recall. Based on the preliminary results of human learning obtained from the MMG games, we attempt to show that the human memory recall improves steadily with the number of game sessions. As an example case, we provide a comparison for the text-to-text and text-image-to-text learning and demonstrate that the addition of image context is useful in improving the learning.

[1]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[2]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[3]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[4]  N. Chater,et al.  Ten years of the rational analysis of cognition , 1999, Trends in Cognitive Sciences.

[5]  Mark Steyvers,et al.  How ideal are we? Incorporating human limitations into Bayesian models of word segmentation , 2010 .

[6]  J. Tenenbaum,et al.  Structure and strength in causal induction , 2005, Cognitive Psychology.

[7]  Wei-Ying Ma,et al.  A Probabilistic Semantic Model for Image Annotation and Multi-Modal Image Retrieva , 2005, ICCV.

[8]  András A. Benczúr,et al.  Multimodal Retrieval by Text-Segment Biclustering , 2008, CLEF.

[9]  Thomas L. Griffiths,et al.  Distributional Cues to Word Boundaries: Context is Important , 2008 .

[10]  Thomas L. Griffiths,et al.  Connecting human and machine learning via probabilistic models of cognition , 2009, INTERSPEECH.

[11]  Sun-Yuan Kung,et al.  Hidden Markov models for character recognition , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[12]  Wei-Ying Ma,et al.  A probabilistic semantic model for image annotation and multi-modal image retrieval , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[13]  Byoung-Tak Zhang,et al.  Hypernetworks: A Molecular Evolutionary Architecture for Cognitive Learning and Memory , 2008, IEEE Computational Intelligence Magazine.

[14]  Petros Maragos,et al.  Multimodal Processing and Interaction, Audio, Video, Text , 2010 .

[15]  András A. Benczúr,et al.  Cross-modal Retrieval by Text and Image Feature Biclustering , 2007, CLEF.

[16]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[17]  Geoffrey E. Hinton,et al.  Deep, Narrow Sigmoid Belief Networks Are Universal Approximators , 2008, Neural Computation.

[18]  Proceedings of the 31st annual Boston University Conference on Language Development , 2007 .

[19]  Byoung-Tak Zhang,et al.  Cognitive learning and the multimodal memory game: Toward human-level machine learning , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[20]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.