Multidimensional Time-Series Shapelets Reliably Detect and Classify Contact Events in Force Measurements of Wiping Actions

The vision of service robots that autonomously manipulate objects as skillfully and flexibly as humans is still an open challenge. Findings from cognitive psychology suggest that the human brain structures manipulation actions along representations of contact events and their perceptually distinctive sensory signals. In this letter, we investigate how to reliably detect and classify contact events during robotic wiping actions. We present an algorithm that learns the distinct shapes of force measurements during contact events using multidimensional time-series shapelets. We evaluate our approach on a dataset consisting of 460 real-world robot wiping episodes that we collected using a table-mounted robot with a wrist-mounted force/torque sensor. Our approach shows good performance with tenfold cross validation yielding 97.5% precision and 99.3% recall, and can also be used for online contact event detection and classification.

[1]  Blake Hannaford,et al.  Hidden Markov Model Analysis of Force/ Torque Information in Telemanipulation , 1989, ISER.

[2]  Daniel Leidner,et al.  Robotic Agents Representing, Reasoning, and Executing Wiping Tasks for Daily Household Chores , 2016, AAMAS.

[3]  Vince D. Calhoun,et al.  Shapelet Ensemble for Multi-dimensional Time Series , 2015, SDM.

[4]  Anthony Bagnall,et al.  A Shapelet Transform for Time Series Classification , 2015 .

[5]  Berthold Bäuml,et al.  Robust material classification with a tactile skin using deep learning , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[7]  Daniel Leidner,et al.  Inferring the effects of wiping motions based on haptic perception , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[8]  Darwin G. Caldwell,et al.  Imitation Learning of Positional and Force Skills Demonstrated via Kinesthetic Teaching and Haptic Input , 2011, Adv. Robotics.

[9]  Geir Hovland,et al.  Hidden Markov Models as a Process Monitor in Robotic Assembly , 1998, Int. J. Robotics Res..

[10]  Lars Schmidt-Thieme,et al.  Fast classification of univariate and multivariate time series through shapelet discovery , 2016, Knowledge and Information Systems.

[11]  Geir Hovland,et al.  Frequency-domain force measurements for discrete event contact recognition , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[12]  Dan Roth,et al.  Efficient Pattern-Based Time Series Classification on GPU , 2012, 2012 IEEE 12th International Conference on Data Mining.

[13]  Paolo Fiorini,et al.  Hybrid HMM/SVM model for the analysis and segmentation of teleoperation tasks , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[14]  Eamonn J. Keogh,et al.  Logical-shapelets: an expressive primitive for time series classification , 2011, KDD.

[15]  Shigeki Sugano,et al.  Tactile object recognition using deep learning and dropout , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[16]  G. Knoblich,et al.  Predicting the Effects of Actions: Interactions of Perception and Action , 2001, Psychological science.

[17]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Miles C. Bowman,et al.  Control strategies in object manipulation tasks , 2006, Current Opinion in Neurobiology.

[19]  Joris De Schutter,et al.  Contact State Segmentation Using Particle Filters for Programming by Human Demonstration in Compliant Motion Tasks , 2006, ISER.

[20]  Philip S. Yu,et al.  Extracting Interpretable Features for Early Classification on Time Series , 2011, SDM.

[21]  John Kenneth Salisbury,et al.  Application of Change Detection to Dynamic Contact Sensing , 1994, Int. J. Robotics Res..

[22]  Eamonn J. Keogh,et al.  Time Series Classification under More Realistic Assumptions , 2013, SDM.

[23]  Helge J. Ritter,et al.  Tactile Convolutional Networks for Online Slip and Rotation Detection , 2016, ICANN.

[24]  Ernesto Staffetti,et al.  Contact-State Classification in Human-Demonstrated Robot Compliant Motion Tasks Using the Boosting Algorithm , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Daniel Leidner,et al.  Classifying compliant manipulation tasks for automated planning in robotics , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[26]  Eamonn J. Keogh,et al.  Fast Shapelets: A Scalable Algorithm for Discovering Time Series Shapelets , 2013, SDM.

[27]  B. Hommel.  Action control according to TEC (theory of event coding) , 2009, Psychological research.

[28]  Eamonn J. Keogh,et al.  Time series shapelets: a new primitive for data mining , 2009, KDD.