Localization of Humans, Objects, and Robots Interacting on Load-Sensing Floors

Localization, tracking, and recognition of objects and humans are basic tasks that are of high value in the applications of ambient intelligence. Sensing floors were introduced to address these tasks in a non-intrusive way. To recognize the humans moving on the floor, they are usually first localized, and then a set of gait features are extracted (stride length, cadence, and pressure profile over a footstep). However, recognition generally fails when several people stand or walk together, preventing successful tracking. This paper presents a detection, tracking, and recognition technique which uses objects' weight. It continues working even when tracking individual persons becomes impossible. Inspired by computer vision, this technique processes the floor pressure-image by segmenting the blobs containing objects, tracking them, and recognizing their contents through a mix of inference and combinatorial search. The result lists the probabilities of assignments of known objects to observed blobs. The concept was successfully evaluated in daily life activity scenarii, involving multi-object tracking and recognition on low-resolution sensors, crossing of user trajectories, and weight ambiguity. This technique can be used to provide a probabilistic input for multi-modal object tracking and recognition systems.

[1]  Woontack Woo,et al.  User Identification Using User's Walking Pattern over the ubiFloorII , 2005, CIS.

[2]  Juha Röning,et al.  Methods for person identification on a pressure-sensitive floor: Experiments with multiple classifiers and reject option , 2008, Inf. Fusion.

[3]  Jeha Ryu,et al.  The User Identification System Using Walking Pattern over the ubiFloor , 2003 .

[4]  Jiqing Zhang,et al.  People Identification Using Floor Pressure Sensing and Analysis , 2010, IEEE Sensors Journal.

[5]  C Detrembleur,et al.  Decomposition of the vertical ground reaction forces during gait on a single force plate. , 2013, Journal of musculoskeletal & neuronal interactions.

[6]  Rui Fukui,et al.  High resolution pressure sensor distributed floor for future human-robot symbiosis environments , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Domnic Savio,et al.  Smart Carpet: A Footstep Tracking Interface , 2007, 21st International Conference on Advanced Information Networking and Applications Workshops (AINAW'07).

[8]  David Birchfield,et al.  The Design of a Pressure Sensing Floor for Movement-Based Human Computer Interaction , 2007, EuroSSC.

[9]  Juha Röning,et al.  TOWARDS THE ADAPTIVE IDENTIFICATION OF WALKERS : AUTOMATED FEATURE SELECTION OF FOOTSTEPS USING DISTINCTION-SENSITIVE LVQ , 2004 .

[10]  A. Steinhage,et al.  Large-area wireless sensor system based on smart textiles , 2012, International Multi-Conference on Systems, Sygnals & Devices.

[11]  Mark S. Nixon,et al.  A floor sensor system for gait recognition , 2005, Fourth IEEE Workshop on Automatic Identification Advanced Technologies (AutoID'05).

[12]  Tomomasa Sato,et al.  Dynamic footprint-based person recognition method using a hidden markov model and a neural network: Research Articles , 2004 .

[13]  Chow-Shing Shin,et al.  Distributed Sensing Floor for an Intelligent Environment , 2009, IEEE Sensors Journal.

[14]  Gregory D. Abowd,et al.  The smart floor: a mechanism for natural user identification and tracking , 2000, CHI Extended Abstracts.

[15]  Paul Lukowicz,et al.  Human tracking and identification using a sensitive floor and wearable accelerometers , 2013, 2013 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[16]  Juha Röning,et al.  Combining classifiers with different footstep feature sets and multiple samples for person identification , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[17]  Jukka Vanhala,et al.  TileTrack: Capacitive human tracking using floor tiles , 2009, 2009 IEEE International Conference on Pervasive Computing and Communications.

[18]  Gregory D. Abowd,et al.  User Identification with User's Stepping Pattern over the ubiFloorII , 2008, Int. J. Pattern Recognit. Artif. Intell..

[19]  David Birchfield,et al.  A pressure sensing floor for interactive media applications , 2005, ACE '05.

[20]  Jukka Riekki,et al.  FOOTSTEP IDENTIFICATION FROM PRESSURE SIGNALS USING HIDDEN MARKOV MODELS , 2003 .

[21]  Jaeseok Yun,et al.  User Identification Using Gait Patterns on UbiFloorII , 2011, Sensors.

[22]  Ben Heller,et al.  The Smartfloor: A Large Area Force-measuring Floor for Investigating Dynamic Balance and Motivating Exercise☆ , 2014 .

[23]  Yaakov Bar-Shalom,et al.  Sonar tracking of multiple targets using joint probabilistic data association , 1983 .

[24]  Jin-Woo Jung,et al.  Dynamic-footprint based person identification using mat-type pressure sensor , 2003, Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No.03CH37439).

[25]  Jaakko Suutala,et al.  Persons tracking with Gaussian process joint particle filtering , 2010, 2010 IEEE International Workshop on Machine Learning for Signal Processing.

[26]  William C. Mann,et al.  The Gator Tech Smart House: a programmable pervasive space , 2005, Computer.

[27]  Tomomasa Sato,et al.  Robotic room: Its concept and realization , 1999, Robotics Auton. Syst..

[28]  Hiroshi Ishiguro,et al.  Human tracking using floor sensors based on the Markov chain Monte Carlo method , 2004, ICPR 2004.

[29]  Andreas Braun,et al.  CapFloor - A Flexible Capacitive Indoor Localization System , 2011, EvAAL.

[30]  Li-Chen Fu,et al.  Human Localization via Multi-Cameras and Floor Sensors in Smart Home , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[31]  Hiroshi Ishiguro,et al.  People tracking by fusing different kinds of sensors, floor sensors and acceleration sensors , 2006, 2006 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems.

[32]  Xi Chen,et al.  Advanced approach for indoor identification and tracking using smart floor and pyroelectric infrared sensors , 2014, 2014 5th International Conference on Information and Communication Systems (ICICS).

[33]  Joseph A. Paradiso,et al.  The magic carpet: physical sensing for immersive environments , 1997, CHI Extended Abstracts.

[34]  Dieter Fox,et al.  Bayesian Filtering for Location Estimation , 2003, IEEE Pervasive Comput..

[35]  Albrecht Schmidt,et al.  Context Acquisition Based on Load Sensing , 2002, UbiComp.

[36]  Olivier Simonin,et al.  High resolution pressure sensing using sub-pixel shifts on low resolution load-sensing tiles , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[37]  T. Sato,et al.  Environment-type robot system "RoboticRoom" featured by behavior media, behavior contents, and behavior adaptation , 2004, IEEE/ASME Transactions on Mechatronics.

[38]  Emanuele Frontoni,et al.  Indoor people localization and tracking using an energy harvesting smart floor , 2014, 2014 IEEE/ASME 10th International Conference on Mechatronic and Embedded Systems and Applications (MESA).

[39]  Julian Fiérrez,et al.  Analysis of Time Domain Information for Footstep Recognition , 2010, ISVC.

[40]  F. Livesey,et al.  The ORL active floor [sensor system] , 1997, IEEE Wirel. Commun..

[41]  Gregory D. Abowd,et al.  The Aware Home: A Living Laboratory for Ubiquitous Computing Research , 1999, CoBuild.

[42]  Peter Willett,et al.  PMHT: problems and some solutions , 2002 .

[43]  Tomomasa Sato,et al.  Multiple people tracking by integrating distributed floor pressure sensors and RFID system , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[44]  Rita Cucchiara,et al.  Sensing floors for privacy-compliant surveillance of wide areas , 2013, 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[45]  Tae-Sung Lee,et al.  Smart Location Tracking System using FSR ( Force Sensing Resistor ) , 2004 .

[46]  Henk A. P. Blom,et al.  Joint Particle Filtering of Multiple Maneuvering Targets From Unassociated Measurements , 2006, J. Adv. Inf. Fusion.

[47]  Jeremy R. Cooperstock,et al.  Probabilistic Tracking of Pedestrian Movements via In-Floor Force Sensing , 2010, 2010 Canadian Conference on Computer and Robot Vision.

[48]  Julian Fiérrez,et al.  Comparative Analysis and Fusion of Spatiotemporal Information for Footstep Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Jeremy R. Cooperstock,et al.  Contact sensing and interaction techniques for a distributed, multimodal floor display , 2010, 2010 IEEE Symposium on 3D User Interfaces (3DUI).

[50]  Tomomasa Sato,et al.  Dynamic footprint‐based person recognition method using a hidden markov model and a neural network , 2004, Int. J. Intell. Syst..

[51]  Nicholas W. D. Evans,et al.  Assessment of a Footstep Biometric Verification System , 2009, Handbook of Remote Biometrics.

[52]  Hanan Samet,et al.  A general approach to connected-component labeling for arbitrary image representations , 1992, JACM.

[53]  J. Riekki,et al.  Learning Vector Quantization in Footstep Identification , 2003 .

[54]  Juha Röning,et al.  Gaussian Process Person Identifier Based on Simple Floor Sensors , 2008, EuroSSC.

[55]  Olivier Simonin,et al.  Intelligent Tiles - Putting Situated Multi-Agents Models in Real World , 2009, ICAART.

[56]  Gang Qian,et al.  People Identification Using Gait Via Floor Pressure Sensing and Analysis , 2008, EuroSSC.