A RGB-D sensor and speech interaction based object manipulation system with improved CBR-BDI reasoning

We design a kind of object manipulation system, which is based on Robot Operating System (ROS) distributed processing framework. This system can communicate with human beings; can percept the 3D environment by RGB-D sensor; has the ability of reasoning; can transfer the natural language intention to machine instruction to control the movement of manipulator. In particular, an improved Case-Based Reasoning-Belief-Desire-Intention (CBR-BDI) reasoning method is proposed. Three parts are added on the CBR-BDI reasoning system, which are map matching, desire analysis and guidance. Such an improvement makes the reasoning engine can carry out in-depth reasoning actively when the user's intention is incomplete and/or mismatched with actual scene. The system could analysis the information which user inputted, interact with human and environment, guide user by conversation, improve user's beliefs, finally get the standardization of complete intention and transfer the intention to machine instructions. Experiment results prove the validity and practicability of our proposed method.

[1]  Sharath Pankanti,et al.  An Extensible Language Interfacefor Robot Manipulation , 2012, AGI.

[2]  Christian Bird,et al.  Products, developers, and milestones: how should I build my N-Gram language model , 2015, ESEC/SIGSOFT FSE.

[3]  Federico Tombari,et al.  A combined texture-shape descriptor for enhanced 3D feature matching , 2011, 2011 18th IEEE International Conference on Image Processing.

[4]  Surjeet Dalal,et al.  Designing CBR-BDI Agent for implementing Supply Chain system , 2013 .

[5]  Mick P. Couper,et al.  Using Text-to-speech (TTS) for Audio Computer-assisted Self-interviewing (ACASI) , 2016 .

[6]  Ashok K. Goel,et al.  A Case-Based Approach To Imitation Learning in Robotic Agents , 2014 .

[7]  Florence March,et al.  2016 , 2016, Affair of the Heart.

[8]  Huasong Min,et al.  Experience mixed the modified artificial potential field method , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Maja J. Mataric,et al.  Interpreting instruction sequences in spatial language discourse with pragmatics towards natural human-robot interaction , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Javier Bajo,et al.  Context-aware multiagent system: Planning home care tasks , 2013, Knowledge and Information Systems.

[11]  F. Jung,et al.  Products , 1968, ADHESION ADHESIVES&SEALANTS.

[12]  Advait Jain,et al.  EL-E: an assistive mobile manipulator that autonomously fetches objects from flat surfaces , 2010, Auton. Robots.

[13]  Cristina Urdiales,et al.  Implicit robot coordination using Case-Based Reasoning behaviors , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Héctor M. Pérez Meana,et al.  Speaker recognition using Mel frequency Cepstral Coefficients (MFCC) and Vector quantization (VQ) techniques , 2012, CONIELECOMP 2012, 22nd International Conference on Electrical Communications and Computers.

[15]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[16]  Xia Shen,et al.  Path planning of mobile robot by mixing experience with modified artificial potential field method , 2015 .

[17]  Luís A. Alexandre.  3D Descriptors for Object and Category Recognition: a Comparative Evaluation , 2012 .

[18]  Aditya Ghose,et al.  Case-Based BDI Agents: An Effective Approach For Intelligent Search On the World Wide Web , 1999 .

[19]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[20]  Javier Bajo,et al.  A multi-agent system for web-based risk management in small and medium business , 2012, Expert Syst. Appl..

[21]  Zoltan-Csaba Marton,et al.  Tutorial: Point Cloud Library: Three-Dimensional Object Recognition and 6 DOF Pose Estimation , 2012, IEEE Robotics & Automation Magazine.

[22]  Liu Qun.  Chinese Lexical Analysis Using Cascaded Hidden Markov Model , 2004 .

[23]  Alexander I. Rudnicky,et al.  Pocketsphinx: A Free, Real-Time Continuous Speech Recognition System for Hand-Held Devices , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[24]  Radu Bogdan Rusu,et al.  Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments , 2010, KI - Künstliche Intelligenz.