Inferring Semantic State Transitions During Telerobotic Manipulation

Human teleoperation of robots and autonomous operations go hand in hand in today's service robots. While robot teleoperation is typically performed on low to medium levels of abstraction, automated planning has to take place on a higher abstraction level, i.e. by means of semantic reasoning. Accordingly, an abstract state of the world has to be maintained in order to enable an operator to switch seamlessly between both operational modes. We propose a novel approach that combines simulation based geometric tracking and semantic state inference by means of so called State Inference Entities to overcome this issue. We also demonstrate how Evolutionary Strategies can be employed to refine simulation parameters. All experiments are demonstrated in real-world experiments conducted with the humanoid robot Rollin’ Justin.

[1]  Tim Laue,et al.  Automatic Parameter Optimization for a Dynamic Robot Simulation , 2009, RoboCup.

[2]  Michel Jouvet,et al.  The paradox of sleep : the story of dreaming , 1999 .

[3]  Leslie Pack Kaelbling,et al.  Integrated task and motion planning in belief space , 2013, Int. J. Robotics Res..

[4]  Naive Physik : Arbeiten aus dem Institut für angewandte Psychologie in Berlin : theoretische und experimentelle Untersuchungen über die Fähigkeit zu intelligentem Handeln , 1923 .

[5]  Du Q. Huynh,et al.  Metrics for 3D Rotations: Comparison and Analysis , 2009, Journal of Mathematical Imaging and Vision.

[6]  Edwin Olson,et al.  AprilTag 2: Efficient and robust fiducial detection , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[7]  Florian Schmidt,et al.  Rollin' Justin - Mobile platform with variable base , 2009, 2009 IEEE International Conference on Robotics and Automation.

[8]  Janusz,et al.  Geometrical Methods in Robotics , 1996, Monographs in Computer Science.

[9]  Mary-Anne Williams,et al.  A Generic Framework for Approximate Simulation in Commonsense Reasoning Systems , 2007, AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning.

[10]  Michael Beetz,et al.  Fast temporal projection using accurate physics-based geometric reasoning , 2013, 2013 IEEE International Conference on Robotics and Automation.

[11]  Daniel Sebastian Leidner,et al.  Cognitive Reasoning for Compliant Robot Manipulation , 2018, Springer Tracts in Advanced Robotics.

[12]  Neal Y. Lii,et al.  Toward scalable intuitive telecommand of robots for space deployment with METERON SUPVIS Justin , 2017 .

[13]  Patrick J. Hayes,et al.  The Naive Physics Manifesto , 1990, The Philosophy of Artificial Intelligence.

[14]  Michael Beetz,et al.  Simulation-based temporal projection of everyday robot object manipulation , 2011, AAMAS.

[15]  Michael A. Goodrich,et al.  Teleoperation and Beyond for Assistive Humanoid Robots , 2013 .

[16]  Leslie Pack Kaelbling,et al.  From Skills to Symbols: Learning Symbolic Representations for Abstract High-Level Planning , 2018, J. Artif. Intell. Res..

[17]  Donald Michie,et al.  Expert systems in the micro-electronic age , 1979 .

[18]  Patrick J. Hayes,et al.  The second naive physics manifesto , 1995 .

[19]  Stuart J. Russell,et al.  Combined Task and Motion Planning for Mobile Manipulation , 2010, ICAPS.

[20]  Daniel Leidner,et al.  Bridging the Gap Between Supervised Autonomy and Teleoperation , 2018, HRI 2018.

[21]  Zoltan-Csaba Marton,et al.  Depth-based tracking with physical constraints for robot manipulation , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Michael Beetz,et al.  Grounding the Interaction: Anchoring Situated Discourse in Everyday Human-Robot Interaction , 2012, Int. J. Soc. Robotics.

[23]  Michael Beetz,et al.  Parameterizing actions to have the appropriate effects , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Anne Auger,et al.  Evolution Strategies , 2018, Handbook of Computational Intelligence.

[25]  Daniel Leidner,et al.  Object-centered hybrid reasoning for whole-body mobile manipulation , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Andrew Howard,et al.  Design and use paradigms for Gazebo, an open-source multi-robot simulator , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[27]  Daniel Leidner,et al.  Things are made for what they are: Solving manipulation tasks by using functional object classes , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[28]  Michael Beetz,et al.  Deeper Understanding of Vague Instructions through Simulated Execution , 2017, AAMAS.

[29]  Hans-Paul Schwefel,et al.  Evolution strategies – A comprehensive introduction , 2002, Natural Computing.

[30]  Alin Albu-Schäffer,et al.  KONTUR-2: Force-feedback teleoperation from the international space station , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[31]  Antonio Bolufé Röhler,et al.  Measuring the curse of dimensionality and its effects on particle swarm optimization and differential evolution , 2014, Applied Intelligence.

[32]  Jessica B. Hamrick,et al.  Simulation as an engine of physical scene understanding , 2013, Proceedings of the National Academy of Sciences.

[33]  Michael Beetz,et al.  Using Physics- and Sensor-based Simulation for High-Fidelity Temporal Projection of Realistic Robot Behavior , 2009, ICAPS.

[34]  Xin Yao,et al.  Fast Evolution Strategies , 1997, Evolutionary Programming.

[35]  Dan Ryan,et al.  Traded Control with Autonomous Robots as Mixed Initiative Interaction , 1997 .

[36]  Samad Hayati,et al.  Design and implementation of a robot control system with traded and shared control capability , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[37]  Robert C. Moore,et al.  Formal Theories of the Commonsense World , 1985 .