ChaLearn looking at people: A review of events and resources

This paper reviews the historic of ChaLearn Looking at People (LAP) events. We started in 2011 (with the release of the first Kinect device) to run challenges related to human action/activity and gesture recognition. Since then we have regularly organized events in a series of competitions covering all aspects of visual analysis of humans. So far we have organized more than 10 international challenges and events in this field. This paper reviews associated events, and introduces the ChaLearn LAP platform where public resources (including code, data and preprints of papers) related to the organized events are available. We also provide a discussion on our main findings and perspectives of ChaLearn LAP activities.

[1]  Joseph A. Paradiso,et al.  The gesture recognition toolkit , 2014, J. Mach. Learn. Res..

[2]  Wei Li,et al.  One-shot learning gesture recognition from RGB-D data using bag of features , 2013, J. Mach. Learn. Res..

[3]  Sergio Escalera,et al.  Challenges in multimodal gesture recognition , 2016, J. Mach. Learn. Res..

[4]  Giorgio Metta,et al.  Keep it simple and sparse: real-time action recognition , 2013, J. Mach. Learn. Res..

[5]  Thad Starner,et al.  MAGIC summoning: towards automatic suggesting and testing of gestures with low probability of false positives during use , 2013, J. Mach. Learn. Res..

[6]  Fernando De la Torre,et al.  Spatio-Temporal Matching for Human Pose Estimation in Video , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Limin Wang,et al.  Action and Gesture Temporal Spotting with Super Vector Representation , 2014, ECCV Workshops.

[8]  Hanqing Lu,et al.  Fusing multi-modal features for gesture recognition , 2013, ICMI '13.

[9]  Ling Shao,et al.  Structure-Preserving Binary Representations for RGB-D Action Recognition , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Venu Govindaraju,et al.  Language-motivated approaches to action recognition , 2013, J. Mach. Learn. Res..

[11]  Petros Maragos,et al.  Dynamic affine-invariant shape-appearance handshape features and classification in sign language videos , 2013, J. Mach. Learn. Res..

[12]  Stéphane Ayache,et al.  Design of an explainable machine learning challenge for video interviews , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[13]  Ling Shao,et al.  Deep Dynamic Neural Networks for Multimodal Gesture Segmentation and Recognition , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Sergio Escalera,et al.  Guest Editors' Introduction to the Special Issue on Multimodal Human Pose Recovery and Behavior Analysis , 2016, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Sergio Escalera,et al.  ChaLearn Looking at People 2015: Apparent Age and Cultural Event Recognition Datasets and Results , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[16]  Sergio Escalera,et al.  ChaLearn Looking at People Challenge 2014: Dataset and Results , 2014, ECCV Workshops.

[17]  Xiu-Shen Wei,et al.  Deep Bimodal Regression for Apparent Personality Analysis , 2016, ECCV Workshops.

[18]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[19]  Yui Man Lui,et al.  Human gesture recognition on product manifolds , 2012, J. Mach. Learn. Res..

[20]  Sergio Escalera,et al.  ChaLearn Looking at People and Faces of the World: Face AnalysisWorkshop and Challenge 2016 , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[21]  Stefanos Zafeiriou,et al.  Robust Correlated and Individual Component Analysis , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  David A. Forsyth,et al.  Discriminative hierarchical part-based models for human parsing and action recognition , 2012, J. Mach. Learn. Res..

[23]  Adrian Hilton,et al.  Visual Analysis of Humans - Looking at People , 2013 .

[24]  Takeshi Oishi,et al.  Advances in Depth Image Analysis and Applications , 2013, Lecture Notes in Computer Science.

[25]  Martha Larson,et al.  Right inflight?: a dataset for exploring the automatic prediction of movies suitable for a watching situation , 2016, MMSys.

[26]  Bodo Rosenhahn,et al.  3D Reconstruction of Human Motion from Monocular Image Sequences , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Yang Gao,et al.  Multi-layered gesture recognition with Kinect , 2015, J. Mach. Learn. Res..

[28]  Sergio Escalera,et al.  Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  AthitsosVassilis,et al.  The ChaLearn gesture dataset (CGD 2011) , 2014 .

[30]  Jian Cheng,et al.  Bayesian Co-Boosting for Multi-modal Gesture Recognition , 2014, Gesture Recognition.

[31]  Georgios Meditskos,et al.  Semantic Event Fusion of Different Visual Modality Concepts for Activity Recognition , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Yu Qiao,et al.  Object-Scene Convolutional Neural Networks for event recognition in images , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[33]  Ruigang Yang,et al.  Real-Time Simultaneous Pose and Shape Estimation for Articulated Objects Using a Single Depth Camera , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Marta Mejail,et al.  Transfer Learning Decision Forests for Gesture Recognition , 2017, Gesture Recognition.

[35]  Maria Pateraki,et al.  Full-Body Pose Tracking—The Top View Reprojection Approach , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Christian Wolf,et al.  ModDrop: Adaptive Multi-Modal Gesture Recognition , 2014, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Limin Wang,et al.  Multi-view Super Vector for Action Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Aleix M. Martínez,et al.  A Model of the Perception of Facial Expressions of Emotion by Humans: Research Overview and Perspectives , 2012, J. Mach. Learn. Res..

[39]  Aleix M. Martínez,et al.  Labeled Graph Kernel for Behavior Analysis , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Ju Yong Chang.  Nonparametric Feature Matching Based Conditional Random Fields for Gesture Recognition from Multi-Modal Video , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Aurélien Garivier,et al.  On the Complexity of Best-Arm Identification in Multi-Armed Bandit Models , 2014, J. Mach. Learn. Res..

[42]  Christian Wolf,et al.  Multi-scale Deep Learning for Gesture Detection and Localization , 2014, ECCV Workshops.

[43]  Sergio Escalera,et al.  ChaLearn Looking at People 2015 challenges: Action spotting and cultural event recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[44]  Isabelle Guyon,et al.  Results and Analysis of the ChaLearn Gesture Challenge 2012 , 2012, WDIA.

[45]  Jean-Luc Dugelay,et al.  Apparent Age Estimation from Face Images Combining General and Children-Specialized Deep Learning Models , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[46]  Bodo Rosenhahn,et al.  Human Pose Estimation from Video and IMUs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Luc Van Gool,et al.  DEX: Deep EXpectation of Apparent Age from a Single Image , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[48]  Jakub Konecný,et al.  One-shot-learning gesture recognition using HOG-HOF features , 2014, J. Mach. Learn. Res..

[49]  Yu Qiao,et al.  Gender and Smile Classification Using Deep Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[50]  Zhe Wang,et al.  Exploring Fisher vector and deep networks for action spotting , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[51]  Alberto Calatroni,et al.  Robust online gesture recognition with crowdsourced annotations , 2014, J. Mach. Learn. Res..

[52]  Pichao Wang,et al.  Large-scale Continuous Gesture Recognition Using Convolutional Neural Networks , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[53]  Hanqing Lu,et al.  DeepBE: Learning Deep Binary Encoding for Multi-label Classification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[54]  Jun Wan,et al.  Explore Efficient Local Features from RGB-D Data for One-Shot Learning Gesture Recognition , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Xin Liu,et al.  Exploiting Feature Hierarchies with Convolutional Neural Networks for Cultural Event Recognition , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[56]  Sergio Escalera,et al.  ChaLearn Looking at People RGB-D Isolated and Continuous Datasets for Gesture Recognition , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[57]  Urbano Nunes,et al.  Probabilistic Social Behavior Analysis by Exploring Body Motion-Based Patterns , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Subramanian Ramanathan,et al.  SALSA: A Novel Dataset for Multimodal Group Behavior Analysis , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  Sudeep Sarkar,et al.  Finding recurrent patterns from continuous sign language sentences for automated extraction of signs , 2012, J. Mach. Learn. Res..

[60]  Nicolas Pugeault,et al.  Sign language recognition using sub-units , 2012, J. Mach. Learn. Res..

[61]  Sergio Escalera,et al.  ChaLearn Joint Contest on Multimedia Challenges Beyond Visual Analysis: An overview , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[62]  Sergio Escalera,et al.  ChaLearn LAP 2016: First Round Challenge on First Impressions - Dataset and Results , 2016, ECCV Workshops.

[63]  W. Marsden.  I and J , 2012 .

[64]  Petros Maragos,et al.  Multimodal gesture recognition via multiple hypotheses rescoring , 2015, J. Mach. Learn. Res..

[65]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  Isabelle Guyon,et al.  The ChaLearn gesture dataset (CGD 2011) , 2014, Machine Vision and Applications.