Motion intention recognition in robot assisted applications

Acquiring, representing and modelling human skills is one of the key research areas in teleoperation, programming-by-demonstration and human-machine collaborative settings. The problems are challenging mainly because of the lack of a general mathematical model to describe human skills. One of the common approaches is to divide the task that the operator is executing into several subtasks or low-level subsystems in order to provide manageable modelling. In this paper we consider the use of a Layered Hidden Markov Model (LHMM) to model human skills. We evaluate a gesteme classifier that classifies motions into basic action-primitives, or gestemes. The gesteme classifiers are then used in a LHMM to model a teleoperated task. The proposed methodology uses three different HMM models at the gesteme level: one-dimensional HMM, multi-dimensional HMM and multi-dimensional HMM with Fourier transform. The online and off-line classification performance of these three models is evaluated with respect to the number of gestemes, the influence of the number of training samples, the effect of noise and the effect of the number of observation symbols. We also apply the LHMM to data recorded during the execution of a trajectory tracking task in 2D and 3D with a mobile manipulator in order to provide qualitative as well as quantitative results for the proposed approach. The results indicate that the LHMM is suitable for modelling teleoperative trajectory-tracking tasks and that the difference in classification performance between one and multidimensional HMMs for gesteme classification is small. It can also be seen that the LHMM is robust with respect to misclassifications in the underlying gesteme classifiers.

[1]  Allison M. Okamura,et al.  Effect of virtual fixture compliance on human-machine cooperative manipulation , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Paolo Fiorini,et al.  Hybrid HMM/SVM model for the analysis and segmentation of teleoperation tasks , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[3]  Ming Ouhyoung,et al.  A real-time continuous gesture recognition system for sign language , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[4]  Gerhard Rigoll,et al.  Performance evaluation of a new hybrid modeling technique for handwriting recognition using identical on-line and off-line data , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[5]  Eric Horvitz,et al.  Layered representations for learning and inferring office activity from multiple sensory channels , 2004, Comput. Vis. Image Underst..

[6]  Rüdiger Dillmann,et al.  Understanding users intention: programming fine manipulation tasks by demonstration , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Steve Renals,et al.  Dynamic Bayesian networks for meeting structuring , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[8]  Samy Bengio,et al.  Modeling Individual and Group Actions in Meetings: A Two-Layer HMM Framework , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[9]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[10]  Fredrik Gustafsson,et al.  Adaptive filtering and change detection , 2000 .

[11]  Gregory D. Hager,et al.  Human-Machine Collaborative Systems for Microsurgical Applications , 2005, Int. J. Robotics Res..

[12]  Shih-Fu Chang,et al.  Unsupervised discovery of multilevel statistical video structures using hierarchical hidden Markov models , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[13]  Gregory D. Hager,et al.  Building a task language for segmentation and recognition of user input to cooperative manipulation systems , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[14]  Redwan Alqasemi,et al.  Telemanipulation Assistance Based on Motion Intention Recognition , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[15]  Allison M. Okamura,et al.  Recognition of operator motions for real-time assistance using virtual fixtures , 2003, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings..

[16]  Danica Kragic,et al.  Adaptive Virtual Fixtures for Machine-Assisted Teleoperation Tasks , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[17]  Russell H. Taylor,et al.  Medical Robotic Systems in Computer-Integrated Surgery , 2003 .

[18]  Blake Hannaford,et al.  Multi-dimensional hidden Markov model of telemanipulation tasks with varying outcomes , 1990, 1990 IEEE International Conference on Systems, Man, and Cybernetics Conference Proceedings.

[19]  J. Edward Colgate,et al.  Cobot architecture , 2001, IEEE Trans. Robotics Autom..

[20]  Allison M. Okamura,et al.  Activation cues and force scaling methods for virtual fixtures , 2003, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings..

[21]  Yoram Singer,et al.  The Hierarchical Hidden Markov Model: Analysis and Applications , 1998, Machine Learning.

[22]  Rüdiger Dillmann,et al.  Building elementary robot skills from human demonstration , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[23]  J. FREDRIC.  On the Use of Windows for Harmonic Analysis with the Discrete Fourier Transform , .