User Behavior Tracking for Education Assisting System by Using an RGB-D Sensor

It is difficult to track multiple people effectively for a long time in a complex environment because people’s clothes and body shapes may be similar, and their postures may be constantly changing. This paper proposes a novel method for multiple people tracking in crowded places where people can be partially or completely occluded. The people are detected by the deep learning method ConvNet from the color image first, and detection results are integrated with the depth information so that the accurate human areas can be extracted. The accurate personal color information can be extracted then without any background color information. multiple people tracking is proceeded by using particle filter based on the color information. the effectiveness of the proposed method is verified through experiments of tracking multiple children in a classroom.

[1]  Hiroshi Ishiguro,et al.  Laser-Based Tracking of Human Position and Orientation Using Parametric Shape Modeling , 2009, Adv. Robotics.

[2]  Francesc Moreno-Noguer,et al.  3D Human Pose Estimation from a Single Image via Distance Matrix Regression , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Silvio Savarese,et al.  Ieee Transaction on Pattern Analysis and Machine Intelligence 1 a General Framework for Tracking Multiple People from a Moving Camera , 2022 .

[4]  Arnaud Doucet,et al.  Particle filters for state estimation of jump Markov linear systems , 2001, IEEE Trans. Signal Process..

[5]  Bernt Schiele,et al.  Learning People Detectors for Tracking in Crowded Scenes , 2013, 2013 IEEE International Conference on Computer Vision.

[6]  Yi Zhang,et al.  An Improved Camshift-based Particle Filter Algorithm for Real-time Hand Gesture Tracking , 2010 .

[7]  Takeo Kanade,et al.  Panoptic Studio: A Massively Multiview System for Social Motion Capture , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  Li Song.  An improved particle filtering algorithm , 2007 .

[9]  Norihiro Hagita,et al.  Social acceptance of a childcare support robot system , 2015, 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[10]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Ioannis A. Kakadiaris,et al.  Estimating Anthropometry and Pose from a Single Uncalibrated Image , 2001, Comput. Vis. Image Underst..

[12]  Jonathan Tompson,et al.  Efficient ConvNet-based marker-less motion capture in general scenes with a low number of cameras , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Jwu-Sheng Hu,et al.  A spatial-color mean-shift object tracking algorithm with scale and orientation estimation , 2008, Pattern Recognit. Lett..