Kinect Sensor Gesture and Activity Recognition: New Applications for Consumer Cognitive Systems

Cognitive consumer electronics (CE), the fastest-growing sector worldwide that is driven by machine intelligence and cognitive systems, is triggered and enabled by audio- and video-capturing devices, smart sensors, health- and fitness-monitoring devices, security and education electronics, and intelligent systems. Smart consumer sensors and cognitive systems are synergized through the Internet of Things (IoT) for optimal information sharing, communication, real-time updates, data analytics, and enhanced support for decision making. Biometric-based devices, originally intended for large-scale applications in airports, border controls, disaster zones, or refugee migration zones, are enabling a wide range of applications in commercial and consumer sectors as standalone systems or with interconnected sensor networks. This article introduces the applications of Microsoft Kinect in cognitive systems for smart CE, and, using Kinect sensors, a human-behavior cognition technology is presented for gesture and activity recognition. As a novel front end of pervasive cognitive systems, the challenges and applications of a Kinect sensor-based system will be explored in CE, such as smart automobiles, health care, surveillance, and activity recognition.

[1]  Nadia Magnenat-Thalmann,et al.  Continuous body emotion recognition system during theater performances , 2016, Comput. Animat. Virtual Worlds.

[2]  Claudia Linnhoff-Popien,et al.  Gait Recognition with Kinect , 2012 .

[3]  Saraju P. Mohanty,et al.  Pay-Cloak: A Biometric Back Cover for Smartphones: Facilitating secure contactless payments and identity virtualization at low cost to end users. , 2017, IEEE Consumer Electronics Magazine.

[4]  Saeid Nahavandi,et al.  A Review of Vision-Based Gait Recognition Methods for Human Identification , 2010, 2010 International Conference on Digital Image Computing: Techniques and Applications.

[5]  Fabio Tozeto Ramos,et al.  Unsupervised clustering of people from ‘skeleton’ data , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[6]  Pascal Fua,et al.  3D tracking for gait characterization and recognition , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[7]  Bir Bhanu,et al.  Statistical feature fusion for gait-based human recognition , 2004, CVPR 2004.

[8]  Peter Corcoran,et al.  Deep Learning for Consumer Devices and Services: Pushing the limits for machine learning, artificial intelligence, and computer vision. , 2017, IEEE Consumer Electronics Magazine.

[9]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[10]  Jake K. Aggarwal,et al.  View invariant human action recognition using histograms of 3D joints , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[11]  Yingxu Wang,et al.  A Formal Knowledge Representation System (FKRS) for the Intelligent Knowledge Base of a Cognitive Learning Engine , 2011, Int. J. Softw. Sci. Comput. Intell..

[12]  Cordelia Schmid,et al.  Learning realistic human actions from movies , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Himanshu Thapliyal,et al.  A Survey of Affective Computing for Stress Detection: Evaluating technologies in stress detection for better health , 2016, IEEE Consumer Electronics Magazine.

[14]  Larry S. Davis,et al.  Stride and cadence as a biometric in automatic person identification and verification , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[15]  Mark S. Nixon,et al.  Automated person recognition by walking and running via model-based approaches , 2004, Pattern Recognit..

[16]  Marina L. Gavrilova,et al.  User Recognition From Social Behavior in Computer-Mediated Social Context , 2017, IEEE Transactions on Human-Machine Systems.

[17]  Marina L. Gavrilova,et al.  Applying Biometric Principles to Avatar Recognition , 2010, 2010 International Conference on Cyberworlds.

[18]  Marina L. Gavrilova,et al.  Social Behavioral Information Fusion in Multimodal Biometrics , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[19]  Patrick Pérez,et al.  View-Independent Action Recognition from Temporal Self-Similarities , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Ramakant Nevatia,et al.  View and scale invariant action recognition using multiview shape-flow models , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Dimitris Kastaniotis,et al.  Pose-based human action recognition via sparse representation in dissimilarity space , 2014, J. Vis. Commun. Image Represent..

[23]  Marina L. Gavrilova,et al.  Emerging Trends in Security System Design Using the Concept of Social Behavioural Biometrics , 2017 .

[24]  Marina L. Gavrilova,et al.  Joint-Triplet Motion Image and Local Binary Pattern for 3D Action Recognition Using Kinect , 2016, CASA.

[25]  Yingxu Wang,et al.  Cognitive Intelligence: Deep Learning, Thinking, and Reasoning by Brain-Inspired Systems , 2016, Int. J. Cogn. Informatics Nat. Intell..

[26]  Marina L. Gavrilova,et al.  Decision Fusion for Multimodal Biometrics Using Social Network Analysis , 2014, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[27]  Arslan Munir.  Safety Assessment and Design of Dependable Cybercars: For today and the future. , 2017, IEEE Consumer Electronics Magazine.

[28]  Jitendra Malik,et al.  Recognizing action at a distance , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[29]  Marina L. Gavrilova,et al.  DTW-based kernel and rank-level fusion for 3D gait recognition using Kinect , 2015, The Visual Computer.

[30]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Xiaodong Yang,et al.  Effective 3D action recognition using EigenJoints , 2014, J. Vis. Commun. Image Represent..

[32]  Marina L. Gavrilova,et al.  Multimodal Biometrics and Intelligent Image Processing for Security Systems , 2013 .

[33]  Toby Sharp,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR.

[34]  Marina L. Gavrilova,et al.  Biometric-Based User Authentication and Activity Level Detection in a Collaborative Environment , 2015 .

[35]  Yingxu Wang,et al.  Cognitive Informatics and Computational Intelligence: From Information Revolution to Intelligence Revolution , 2015, Int. J. Softw. Sci. Comput. Intell..

[36]  Shushma Patel,et al.  A layered reference model of the brain (LRMB) , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).