Item Ownership Relationship Semantic Learning Strategy for Personalized Service Robot

In order to satisfy the robotic personalized service requirements that can select exclusive items to perform inference and planning according to different service individuals, the service robots need to have the ability to independently obtain the ownership relationship between humans and their carrying items. In this work, we present a novel semantic learning strategy for item ownership. Firstly, a human-carrying-items detection network based on human posture estimation and object detection model is used. Then, the transferred convolutional neural network is used to extract the characteristics of the objects and the back-end classifier to recognize the object instance. At the same time, the face detection and recognition model are used to identify the service individual. Finally, on the basis of the former two, the active learning of ownership items is completed. The experimental results show that the proposed ownership semantic learning strategy can determine the ownership relationship of private goods accurately and efficiently. The solution of this problem can improve the intelligence level of robot life service.

[1]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Xue Li,et al.  Learning to Recognize Hand-Held Objects from Scratch , 2016, PCM.

[3]  Tai-Jiang Mu,et al.  Synthesizing Robot Programs with Interactive Tutor Mode , 2018, Int. J. Autom. Comput..

[4]  Sanja Fidler,et al.  Detect What You Can: Detecting and Representing Objects Using Holistic Models and Body Parts , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Guohui Tian,et al.  Learning to Transform Service Instructions into Actions with Reinforcement Learning and Knowledge Base , 2018, Int. J. Autom. Comput..

[6]  Ravi Kumar Mandava,et al.  Near Optimal PID Controllers for the Biped Robot While Walking on Uneven Terrains , 2018, Int. J. Autom. Comput..

[7]  Tianmiao Wang,et al.  Current Researches and Future Development Trend of Intelligent Robot: A Review , 2018, Int. J. Autom. Comput..

[8]  Hatice Kose-Bagci,et al.  Improving human-robot interaction based on joint attention , 2016, Applied Intelligence.

[9]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[10]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Zhijing Li,et al.  A Position and Torque Switching Control Method for Robot Collision Safety , 2018, Int. J. Autom. Comput..

[12]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[13]  David A. McAllester,et al.  A discriminatively trained, multiscale, deformable part model , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Jitendra Malik,et al.  Using k-Poselets for Detecting People and Localizing Their Keypoints , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Luis E. Ortiz,et al.  Parsing clothing in fashion photographs , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Shuo Yang,et al.  WIDER FACE: A Face Detection Benchmark , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[18]  Anil A. Bharath,et al.  A dataset for Hand-Held Object Recognition , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[19]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Hao Li,et al.  Large-scale 3D Semantic Mapping Using Stereo Vision , 2018, Int. J. Autom. Comput..

[21]  Shuang Wang,et al.  RGB-D Hand-Held Object Recognition Based on Heterogeneous Feature Fusion , 2015, Journal of Computer Science and Technology.

[22]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[23]  Anil A. Bharath,et al.  Small Hand-held Object Recognition Test (SHORT) , 2014, IEEE Winter Conference on Applications of Computer Vision.

[24]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Wei Wang,et al.  Virtual plate based controlling strategy of toy play for robot’s communication development in JA space , 2019, Int. J. Autom. Comput..

[26]  Bernt Schiele,et al.  Articulated people detection and pose estimation: Reshaping the future , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Peter V. Gehler,et al.  DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Hao Wu,et al.  Cloud robot: semantic map building for intelligent service task , 2018, Applied Intelligence.

[29]  Yunchao Wei,et al.  Multiple-Human Parsing in the Wild , 2017 .

[30]  Guohui Tian,et al.  A Selective Attention Guided Initiative Semantic Cognition Algorithm for Service Robot , 2018, Int. J. Autom. Comput..

[31]  Ke Gong,et al.  Look into Person: Self-Supervised Structure-Sensitive Learning and a New Benchmark for Human Parsing , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Shuicheng Yan,et al.  Human Parsing with Contextualized Convolutional Neural Network , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Jun-Wei Hsieh,et al.  Handheld object detection and its related event analysis using ratio histogram and mixture of HMMs , 2014, J. Vis. Commun. Image Represent..

[35]  C. L. Philip Chen,et al.  A survey of human-centered intelligent robots: issues and challenges , 2017, IEEE/CAA Journal of Automatica Sinica.

[36]  Bernt Schiele,et al.  DeeperCut: A Deeper, Stronger, and Faster Multi-person Pose Estimation Model , 2016, ECCV.