Identification from ceiling: unconstrained person identification for tabletops using multiview learning

This paper presents novel unconstrained person identification for tabletop systems using a ceiling-mounted depth camera that overlooks a table. Recent state-of-the-art ubicomp, computer-vision, and CSCW studies have tried to recognize a user's activities and actions on a table using a ceiling-mounted device that overlooks the table. However, conventional unconstrained person identification methods such as face identification cannot be used for providing personalized services in such settings. In this study, we focus on a user's soft biometrics that can be captured from the ceiling such as the shoulder length, shape of the head, and posture of the back to achieve unconstrained person identification by using a ceiling-mounted depth camera. We achieve robust person identification by combining the soft biometrics within a framework of multiview learning. Multiview learning allows us to deal effectively with data consisting of features from multiple sources with different data distributions, i.e., multiple soft biometrics in our case. To the best of our knowledge, this is the first study that investigates the feasibility of person identification for tabletop users by a ceiling-mounted depth camera.

[1]  Constantine Stephanidis,et al.  iEat: An Interactive Table for Restaurant Customers' Experience Enhancement , 2013, HCI.

[2]  Vincent Lepetit,et al.  Randomized trees for real-time keypoint recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[3]  Bernt Schiele,et al.  A database for fine grained activity detection of cooking activities , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Paolo Napoletano,et al.  On the use of MKL for cooking action recognition , 2014, Electronic Imaging.

[5]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[6]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Henry A. Kautz,et al.  Inferring activities from interactions with objects , 2004, IEEE Pervasive Computing.

[8]  C. Avendano,et al.  The CIPIC HRTF database , 2001, Proceedings of the 2001 IEEE Workshop on the Applications of Signal Processing to Audio and Acoustics (Cat. No.01TH8575).

[9]  Hideaki Kuzuoka,et al.  Embedded interaction: The accomplishment of actions in everyday and video-mediated environments , 2013, TCHI.

[10]  Tieniu Tan,et al.  Silhouette Analysis-Based Gait Recognition for Human Identification , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Paul Lukowicz,et al.  Recognizing Workshop Activity Using Body Worn Microphones and Accelerometers , 2004, Pervasive.

[12]  Kamin Whitehouse,et al.  Using Height Sensors for Biometric Identification in Multi-resident Homes , 2010, Pervasive.

[13]  Zhen Wang,et al.  uWave: Accelerometer-based Personalized Gesture Recognition and Its Applications , 2009, PerCom.

[14]  Mehmet G nen Bayesian Efficient Multiple Kernel Learning , 2012, ICML 2012.

[15]  Patrick Baudisch,et al.  Bootstrapper: recognizing tabletop users by their shoes , 2012, CHI.

[16]  Mehmet Gönen,et al.  Bayesian Efficient Multiple Kernel Learning , 2012, ICML.

[17]  Yan Huang,et al.  Propagation networks for recognition of partially ordered sequential action , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[18]  S. C. Johnson Hierarchical clustering schemes , 1967, Psychometrika.

[19]  Volker Roth,et al.  The IR ring: authenticating users' touches on a multi-touch display , 2010, UIST '10.

[20]  Takuya Maekawa,et al.  A sensor device for automatic food lifelogging that is embedded in home ceiling light: A preliminary investigation , 2013, 2013 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops.

[21]  Albrecht Schmidt,et al.  The potentials of in-situ-projection for augmented workplaces in production: a study with impaired persons , 2013, CHI Extended Abstracts.

[22]  Atsushi Sugiura,et al.  A user interface using fingerprint recognition: holding commands and data objects on fingers , 1998, UIST '98.

[23]  Stephen J. McKenna,et al.  Combining embedded accelerometers with computer vision for recognizing food preparation activities , 2013, UbiComp.

[24]  James M. Rehg,et al.  A Scalable Approach to Activity Recognition based on Object Use , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[25]  Takuya Maekawa,et al.  Unsupervised Activity Recognition with User's Physical Characteristics Data , 2011, 2011 15th Annual International Symposium on Wearable Computers.

[26]  Robert T. Collins,et al.  Silhouette-based human identification from body shape and gait , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[27]  Frederic Kerber,et al.  Automatic capturing and analysis of manual manufacturing processes with minimal setup effort , 2016, UbiComp Adjunct.

[28]  Andrew W. Moore,et al.  X-means: Extending K-means with Efficient Estimation of the Number of Clusters , 2000, ICML.

[29]  Martin Schmitz,et al.  Permulin: mixed-focus collaboration on multi-view tabletops , 2014, CHI.

[30]  Takuya Maekawa,et al.  Toward practical factory activity recognition: unsupervised understanding of repetitive assembly work in a factory , 2016, UbiComp.

[31]  C. Ogden,et al.  Anthropometric reference data for children and adults: United States, 2007-2010. , 2012, Vital and health statistics. Series 11, Data from the National Health Survey.

[32]  Andrea F. Abate,et al.  2D and 3D face recognition: A survey , 2007, Pattern Recognit. Lett..

[33]  Yujie Dong,et al.  Detecting Periods of Eating During Free-Living by Tracking Wrist Motion , 2014, IEEE Journal of Biomedical and Health Informatics.

[34]  Ricardo Matsumura de Araújo,et al.  Towards skeleton biometric identification using the microsoft kinect sensor , 2013, SAC '13.

[35]  Kazutaka Shimada,et al.  Person Identification Using Top-View Image with Depth Information , 2012, 2012 13th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing.

[36]  Dominik Schmidt,et al.  HandsDown: hand-contour-based user identification for interactive surfaces , 2010, NordiCHI.

[37]  Trevor Hastie,et al.  Kernel Smoothing Methods , 2009 .

[38]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[39]  Claudia Linnhoff-Popien,et al.  Gait Recognition with Kinect , 2012 .

[40]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Patrick Baudisch,et al.  Multitoe: high-precision interaction with back-projected floors based on high-resolution multi-touch input , 2010, UIST.