Weight Estimation from an RGB-D camera in top-view configuration

The development of so-called soft-biometrics aims at providing information related to the physical and behavioural characteristics of a person. This paper focuses on body weight estimation based on the observation from a top-view RGB-D camera. In fact, the capability to estimate the weight of a person can be of help in many different applications, from health-related scenarios, to business intelligence and retail analytics. To deal with this issue, a TVWE (Top-View Weight Estimation) framework is proposed with the aim of predicting the weight. The approach relies on the adoption of Deep Neural Networks (DNNs) that have been trained on depth data. Each network has also been modified in their top section to replace classification with prediction inference. The performance of five state-of-art DNNs have been compared, namely VGG16, ResNet, Inception, DenseNet and Efficient-Net. In addition, a convolutional auto-encoder has also been included for completeness. Considering the limited literature in this domain, the TVWE framework has been evaluated on a new publicly available dataset: “VRAI Weight estimation Dataset”, which also collects, for each subject, labels related to weight, gender, and height. The experimental results have demonstrated that the proposed methods are suitable for this task, bringing different and significant insights for the application of the solution in different domains.

[1]  Tan Halim Supranata,et al.  Body Weight Measurement Using Image Processing Based on Body Surface Area and Elliptical Tube Volume , 2018, 2018 10th International Conference on Information Technology and Electrical Engineering (ICITEE).

[2]  Pedro F. Felzenszwalb.  Learning models for object recognition , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Peter H. N. de With,et al.  Employing a RGB-D sensor for real-time tracking of humans across multiple re-entries in a smart environment , 2012, IEEE Transactions on Consumer Electronics.

[4]  Andres Erazo,et al.  Artificial neural networks and digital image processing: An approach for indirect weight measurement , 2017, 2017 IEEE Second Ecuador Technical Chapters Meeting (ETCM).

[5]  Guanghui Teng,et al.  Research and development of pig weight estimation system based on image , 2011, 2011 International Conference on Electronics, Communications and Control (ICECC).

[6]  Andreas Nüchter,et al.  Neural network-based visual body weight estimation for drug dosage finding , 2016, SPIE Medical Imaging.

[7]  Emanuele Frontoni,et al.  A business application of RTLS technology in Intelligent Retail Environment: Defining the shopper's preferred path and its segmentation , 2019, Journal of Retailing and Consumer Services.

[8]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Ye Liu,et al.  Detecting and tracking people in real time with RGB-D camera , 2015, Pattern Recognit. Lett..

[10]  Sung-Jea Ko,et al.  Robust people counting system based on sensor fusion , 2012, IEEE Transactions on Consumer Electronics.

[11]  Zheng Liu,et al.  RGB-D-Based Object Recognition Using Multimodal Convolutional Neural Networks: A Survey , 2019, IEEE Access.

[12]  Ramakant Nevatia,et al.  Segmentation and Tracking of Multiple Humans in Crowded Environments , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Emanuele Frontoni,et al.  Deep understanding of shopper behaviours and interactions using RGB-D vision , 2020, Machine Vision and Applications.

[14]  Boby George,et al.  Continuous Weight Monitoring System for ICU Beds using Air-filled Mattresses/Pads: A Proof of Concept , 2019, 2019 IEEE International Symposium on Medical Measurements and Applications (MeMeA).

[15]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[16]  T. Liang,et al.  Body weight measurement based on bending losses of single-mode fiber optic loops , 2016, International Conference on Advanced Materials for Science and Engineering.

[17]  Jack Wang,et al.  Validation of a 3-dimensional photonic scanner for the measurement of body volumes, dimensions, and percentage body fat. , 2006, The American journal of clinical nutrition.

[18]  Ling Shao,et al.  Enhanced Computer Vision With Microsoft Kinect Sensor: A Review , 2013, IEEE Transactions on Cybernetics.

[19]  Emanuele Frontoni,et al.  People Detection and Tracking from an RGB-D Camera in Top-View Configuration: Review of Challenges and Applications , 2017, ICIAP Workshops.

[20]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[21]  V. Piuri,et al.  Weight estimation from frame sequences using computational intelligence techniques , 2012, 2012 IEEE International Conference on Computational Intelligence for Measurement Systems and Applications (CIMSA) Proceedings.

[22]  Emanuele Frontoni,et al.  Multidisciplinary Pattern Recognition applications: A review , 2020, Comput. Sci. Rev..

[23]  Emanuele Frontoni,et al.  Open-World Person Re-Identification With RGBD Camera in Top-View Configuration for Retail Applications , 2020, IEEE Access.

[24]  Andreas Nüchter,et al.  Libra3D: Body weight estimation for emergency patients in clinical environments with a 3D structured light sensor , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Emanuele Frontoni,et al.  Movements analysis of preterm infants by using depth sensor , 2017, IML.

[26]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Nissan Kunju,et al.  A palmar pressure sensor for measurement of upper limb weight bearing by the hands during transfers by paraplegics , 2013, Journal of medical engineering & technology.

[28]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  D. D. Bois,et al.  A height-weight formula to estimate the surface area of man , 1916 .

[30]  Ramakant Nevatia,et al.  Detection and Tracking of Multiple, Partially Occluded Humans by Bayesian Combination of Edgelet based Part Detectors , 2007, International Journal of Computer Vision.

[31]  Andreas Nüchter,et al.  Body Weight Estimation for Dose-Finding and Health Monitoring of Lying, Standing and Walking Patients Based on RGB-D Data , 2018, Sensors.

[32]  Emanuele Frontoni,et al.  Convolutional Networks for Semantic Heads Segmentation using Top-View Depth Data in Crowded Environment , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[33]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[34]  Michael Beetz,et al.  Leaving Flatland: Efficient real‐time three‐dimensional perception and motion planning , 2009, J. Field Robotics.

[35]  Luc Van Gool,et al.  Robust Multiperson Tracking from a Mobile Platform , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Roberto Pierdicca,et al.  Robust and affordable retail customer profiling by vision and radio beacon sensor fusion , 2016, Pattern Recognit. Lett..

[37]  Saeed Babaeizadeh,et al.  Densely connected convolutional networks and signal quality analysis to detect atrial fibrillation using short single-lead ECG recordings , 2017, 2017 Computing in Cardiology (CinC).

[38]  R. Mosteller.  Simplified calculation of body-surface area. , 1987, The New England journal of medicine.