KnowRobSIM — Game Engine-Enabled Knowledge Processing Towards Cognition-Enabled Robot Control

AI knowledge representation and reasoning methods consider actions to be blackboxes that abstract away from how they are executed. This abstract view does not suffice for the decision making capabilities required by robotic agents that are to accomplish manipulation tasks. Such robots have to reason about how to pour without spilling, where to grasp a pot, how to open different containers, and so on. To enable such reasoning it is necessary to consider how objects are perceived, how motions can be executed and parameterized, and how motion parameterization affects the physical effects of actions. To this end, we propose to complement and extend symbolic reasoning methods with KnowRobSIM, an additional reasoning infrastructure based on modern game engine technology, including the subsymbolic world modeling through data structures, action simulation based on physics engine, and world scene rendering. We demonstrate how KnowRobSIM can perform powerful reasoning, prediction, and learning tasks that are required for informed decision making in object manipulation.

[1]  Paul Oliver.  Unreal Engine 4 Elemental , 2012, SIGGRAPH '12.

[2]  Michael Beetz,et al.  Know Rob 2.0 — A 2nd Generation Knowledge Processing Framework for Cognition-Enabled Robotic Agents , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Jeffrey C. Trinkle,et al.  Grasp analysis as linear matrix inequality problems , 2000, IEEE Trans. Robotics Autom..

[4]  Jeffrey Mark Siskind,et al.  Reconstructing force-dynamic models from video sequences , 2003, Artif. Intell..

[5]  Drew McDermott,et al.  Robot Planning , 1991, AI Mag..

[6]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[7]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[8]  Michael Beetz,et al.  Learning and Reasoning with Action-Related Places for Robust Mobile Manipulation , 2014, J. Artif. Intell. Res..

[9]  G. Hesslow.  The current status of the simulation theory of cognition , 2012, Brain Research.

[10]  J. Tenenbaum,et al.  Mind Games: Game Engines as an Architecture for Intuitive Physics , 2017, Trends in Cognitive Sciences.

[11]  Michael Beetz,et al.  A cloud service for robotic mental simulations , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Chenxi Liu,et al.  ScaleNet: Guiding Object Proposal Generation in Supermarkets and Beyond , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[13]  Ernest Davis,et al.  The Scope and Limits of Simulation in Cognitive Models , 2015, ArXiv.

[14]  Gavin Moran.  Pushing photorealism in "a boy and his kite" , 2015, SIGGRAPH 2015.

[15]  Mihai Polceanu,et al.  Computational mental simulation: A review , 2016, Comput. Animat. Virtual Worlds.

[16]  Miles C. Bowman,et al.  Control strategies in object manipulation tasks , 2006, Current Opinion in Neurobiology.

[17]  J. Feldman,et al.  Embodied meaning in a neural theory of language , 2004, Brain and Language.

[18]  M. Arbib.  Action to language via the mirror neuron system , 2006 .

[19]  Tom Ziemke,et al.  Finding Your Way from the Bed to the Kitchen: Reenacting and Recombining Sensorimotor Episodes Learned from Human Demonstration , 2016, Front. Robot. AI.

[20]  Justus H. Piater,et al.  Integrating multi-purpose natural language understanding, robot's memory, and symbolic planning for task execution in humanoid robots , 2018, Robotics Auton. Syst..

[21]  Sanem Sariel,et al.  Cognition-Enabled Robot Manipulation in Human Environments: Requirements, Recent Work, and Open Problems , 2017, IEEE Robotics & Automation Magazine.