Using Neural Imaging and Cognitive Modeling to Infer Mental States while Using an Intelligent Tutoring System

Functional magnetic resonance imaging (fMRI) data were collected while students worked with a tutoring system that taught an algebra isomorph. A cognitive model predicted the distribution of solution times from measures of problem complexity. Separately, a linear discriminant analysis used fMRI data to predict whether or not students were engaged in problem solving. A hidden Markov algorithm merged these two sources of information to predict the mental states of students during problem-solving episodes. The algorithm was trained on data from one day of interaction and tested with data from a later day. In terms of predicting what state a student was in during any 2 second period, the algorithm achieved 87% accuracy on the training data and 83% accuracy on the test data. Further, the prediction accuracy using combined cognitive model and fMRI signal showed superadditivity of accuracies when using either cognitive model or fMRI signal alone.

[1]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[2]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[3]  Indrayana Rustandi,et al.  Modeling fMRI data generated by overlapping cognitive processes with unknown onsets using Hidden Process Models , 2009, NeuroImage.

[4]  Kathryn M. McMillan,et al.  N‐back working memory paradigm: A meta‐analysis of normative functional neuroimaging studies , 2005, Human brain mapping.

[5]  Shunzheng Yu,et al.  Hidden semi-Markov models , 2010, Artif. Intell..

[6]  R. Passingham,et al.  Reading Hidden Intentions in the Human Brain , 2007, Current Biology.

[7]  Albert T. Corbett,et al.  Cognitive Tutor: Applied research in mathematics education , 2007, Psychonomic bulletin & review.

[8]  Dinggang Shen,et al.  Classifying spatial patterns of brain activity with machine learning methods: Application to lie detection , 2005, NeuroImage.

[9]  John R. Anderson,et al.  The change of the brain activation patterns as children learn algebra equation solving. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[11]  Tom Michael Mitchell,et al.  Predicting Human Brain Activity Associated with the Meanings of Nouns , 2008, Science.

[12]  G. Rees,et al.  Predicting the Stream of Consciousness from Activity in Human Visual Cortex , 2005, Current Biology.

[13]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[14]  John R. Anderson,et al.  An information-processing model of the BOLD response in symbol manipulation tasks , 2003, Psychonomic bulletin & review.

[15]  Theodore J. Huppert,et al.  Real-time imaging of human brain function by near-infrared spectroscopy using an adaptive general linear model , 2009, NeuroImage.

[16]  Shawn Betts,et al.  Practice Enables Successful Learning Under Minimal Guidance , 2009 .

[17]  John R. Anderson.  How Can the Human Mind Occur in the Physical Universe , 2007 .