Concept Towards Segmenting Arm Areas for Robot-Based Dermatological In Vivo Measurements

Dermatological in vivo measurements are used for various purposes, e.g. health care, development and testing of skin care products or claim support in marketing. Especially for the last two purposes, in vivo measurements are extensive due to the quantity and repeatability of the measurement series. Furthermore, they are performed manually and therefore represent a nonnegligible time and cost factor. A solution to this is the implementation of collaborative robotics for the measurement execution. Due to various body shapes and surface conditions, common static control procedures are not applicable. To solve this problem, spatial information obtained from a stereoscopic camera can be integrated into the robot control process. However, the designated measurement area has to be detected and the spatial information processed. Therefore the authors propose a concept towards segmenting arm areas through a CNN-based object detector and their further processing to perform robot-based in vivo measurements. The paper gives an overview of the utilization of RGB-D images in 2D object detectors and describes the selection of a suitable model for the application. Furthermore the creation, annotation and augmentation of a custom dataset is presented.

[1]  Patrice Y. Simard,et al.  Using GPUs for machine learning algorithms , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[2]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[4]  Farid Melgani,et al.  Detecting Cars in UAV Images With a Catalog-Based Approach , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Patrick Haffner,et al.  Support vector machines for histogram-based image classification , 1999, IEEE Trans. Neural Networks.

[6]  Heng Tao Shen,et al.  Exploiting Depth From Single Monocular Images for Object Detection and Semantic Segmentation , 2016, IEEE Transactions on Image Processing.

[7]  Junsong Yuan,et al.  Robust Part-Based Hand Gesture Recognition Using Kinect Sensor , 2013, IEEE Transactions on Multimedia.

[8]  Keechul Jung,et al.  GPU implementation of neural networks , 2004, Pattern Recognit..

[9]  Asifullah Khan,et al.  A survey of the recent architectures of deep convolutional neural networks , 2019, Artificial Intelligence Review.

[10]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[11]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Danfei Xu,et al.  PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Eduardo Ros,et al.  Background Subtraction Based on Color and Depth Using Active Sensors , 2013, Sensors.

[16]  Xiaoliang Wang,et al.  Focal loss dense detector for vehicle surveillance , 2018, 2018 International Conference on Intelligent Systems and Computer Vision (ISCV).

[17]  Kazunori Umeda,et al.  Expandable YOLO: 3D Object Detection from RGB-D Images , 2020, 2020 21st International Conference on Research and Education in Mechatronics (REM).

[18]  C. V. Jawahar,et al.  Cats and dogs , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Yi Zheng,et al.  Parallel RCNN: A deep learning method for people detection using RGB-D images , 2017, 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[20]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[21]  Jitendra Malik,et al.  Deformable part models are convolutional neural networks , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Iasonas Kokkinos,et al.  Accurate Human-Limb Segmentation in RGB-D Images for Intelligent Mobility Assistance Robots , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[23]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Sebastian Thrun,et al.  Real-time identification and localization of body parts from depth images , 2010, 2010 IEEE International Conference on Robotics and Automation.

[25]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[26]  Radu Tudor Ionescu,et al.  Optimizing the Trade-Off between Single-Stage and Two-Stage Deep Object Detectors using Image Difficulty Prediction , 2018, 2018 20th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC).

[27]  Jianxiong Xiao,et al.  Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Majid Mirmehdi,et al.  Detecting humans in RGB-D data with CNNs , 2017, 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA).

[29]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[30]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Esteban Walter Gonzalez Clua,et al.  A Comparison between Background Subtraction Algorithms using a Consumer Depth Camera , 2012, VISAPP.

[33]  Silvio Savarese,et al.  Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[35]  Jianying Zhang,et al.  Face Recognition Based on Local Binary Patterns with Threshold , 2010, 2010 IEEE International Conference on Granular Computing.

[36]  Jitendra Malik,et al.  Learning Rich Features from RGB-D Images for Object Detection and Segmentation , 2014, ECCV.

[37]  Amir Nadeem,et al.  Human Body Parts Estimation and Detection for Physical Sports Movements , 2019, 2019 2nd International Conference on Communication, Computing and Digital systems (C-CODE).

[38]  Toon Goedemé,et al.  Exploring RGB+Depth Fusion for Real-Time Object Detection , 2019, Sensors.

[39]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[40]  Yang Hongping,et al.  A Human Body Part Segmentation Method Based on Markov Random Field , 2012, 2012 International Conference on Control Engineering and Communication Technology.

[41]  Shahram Payandeh,et al.  Localization and identification of body extremities based on data from multiple depth sensors , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[42]  Xiangyu Zhang,et al.  Light-Head R-CNN: In Defense of Two-Stage Object Detector , 2017, ArXiv.

[43]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[44]  Gang Zeng,et al.  2.5D Convolution for RGB-D Semantic Segmentation , 2019, 2019 IEEE International Conference on Image Processing (ICIP).