Active glass-type human augmented cognition system considering attention and intention

Human cognition is the result of an interaction of several complex cognitive processes with limited capabilities. Therefore, the primary objective of human cognitive augmentation is to assist and expand these limited human cognitive capabilities independently or together. In this study, we propose a glass-type human augmented cognition system, which attempts to actively assist human memory functions by providing relevant, necessary and intended information by constantly assessing intention of the user. To achieve this, we exploit selective attention and intention processes. Although the system can be used in various real-life scenarios, we test the performance of the system in a person identity scenario. To detect the intended face, the system analyses the gaze points and change in pupil size to determine the intention of the user. An assessment of the gaze points and change in pupil size together indicates that the user intends to know the identity and information about the person in question. Then, the system retrieves several clues through speech recognition system and retrieves relevant information about the face, which is finally displayed through head-mounted display. We present the performance of several components of the system. Our results show that the active and relevant assistance based on users' intention significantly helps the enhancement of memory functions.

[1]  H. Ebbinghaus.  Memory A Contribution Toexperimental Psychology , 1913 .

[2]  Minho Lee,et al.  Probabilistic human intention modeling for cognitive augmentation , 2012, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[3]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[4]  Minho Lee,et al.  Human implicit intent transition detection based on pupillary analysis , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[5]  H. Levene.  Robust tests for equality of variances , 1961 .

[6]  E. Vogel,et al.  Interactions between attention and working memory , 2006, Neuroscience.

[7]  Hermann Ebbinghaus (1885).  Memory: A Contribution to Experimental Psychology , 2013, Annals of Neurosciences.

[8]  M. Tomasello,et al.  Does the chimpanzee have a theory of mind? 30 years later , 2008, Trends in Cognitive Sciences.

[9]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Farrah Wong,et al.  Intention reading towards engineering applications for the elderly and people with disabilities , 2006 .

[11]  Minho Lee,et al.  Recognition of Human's Implicit Intention Based on an Eyeball Movement Pattern Analysis , 2011, ICONIP.

[12]  B. Goldwater.  Psychological significance of pupillary movements. , 1972, Psychological bulletin.

[13]  Minho Lee,et al.  Stereo saliency map considering affective factors and selective motion analysis in a dynamic environment , 2008, Neural Networks.

[14]  Rada Mihalcea,et al.  TextRank: Bringing Order into Text , 2004, EMNLP.

[15]  Minho Lee,et al.  Incremental two-dimensional kernel principal component analysis , 2014, Neurocomputing.

[16]  Jong Bae Kim.  A personal identity annotation overlay system using a wearable computer for augmented reality , 2003, IEEE Trans. Consumer Electron..

[17]  W. James,et al.  The Principles of Psychology. , 1983 .

[18]  Claudio M. Privitera,et al.  The pupil dilation response to visual detection , 2008, Electronic Imaging.

[19]  Seong-Bae Park,et al.  Expanding Knowledge Source with Ontology Alignment for Augmented Cognition , 2011, ICONIP.

[20]  A. Miyake,et al.  Models of Working Memory: Mechanisms of Active Maintenance and Executive Control , 1999 .

[21]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[22]  Minho Lee,et al.  Improving AdaBoost Based Face Detection Using Face-Color Preferable Selective Attention , 2008, IDEAL.

[23]  Wei-Ying Ma,et al.  User Intention Modeling in Web Applications Using Data Mining , 2002, World Wide Web.

[24]  Harry Zhang,et al.  The Optimality of Naive Bayes , 2004, FLAIRS.

[25]  Marcia K. Johnson,et al.  Contextual prerequisites for understanding: Some investigations of comprehension and recall , 1972 .

[26]  F A YOUNG,et al.  Pupillary contraction and dilation in light and darkness. , 1954, Journal of comparative and physiological psychology.

[27]  S. Liversedge,et al.  Saccadic eye movements and cognition , 2000, Trends in Cognitive Sciences.

[28]  Hyung-Min Park,et al.  Target speech enhancement based on degenerate unmixing and estimation technique for real-world applications (Speech and audio processing and translation) , 2010 .

[29]  C. Dunnett.  A Multiple Comparison Procedure for Comparing Several Treatments with a Control , 1955 .

[30]  Changseok Bae,et al.  Design of The Wearable Gadgets for Life-Log Services based on UTC , 2007, IEEE Transactions on Consumer Electronics.

[31]  Cynthia Breazeal,et al.  Social interactions in HRI: the robot view , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[32]  N. Cowan.  The magical number 4 in short-term memory: A reconsideration of mental storage capacity , 2001, Behavioral and Brain Sciences.

[33]  S. Hillyard,et al.  Modulations of sensory-evoked brain potentials indicate changes in perceptual processing during visual-spatial priming. , 1991, Journal of experimental psychology. Human perception and performance.