Actions & Imagined Actions in Cognitive robots

Natural/Artificial systems that are capable of utilizing thoughts at the service of their actions are gifted with the profound opportunity to mentally manipulate the causal structure of their physical interactions with the environment. A cognitive robot can in this way virtually reason about how an unstructured world should ‘change’, such that it becomes a little bit more conducive towards realization of its internal goals. In this article, we describe the various internal models for real/mental action generation developed in the GNOSYS Cognitive architecture and demonstrate how their coupled interactions can endow the GNOSYS robot with a preliminary ability to virtually manipulate neural activity in its mental space in order to initiate flexible goal directed behavior in its physical space. Making things more interesting (and computationally challenging) is the fact that the environment in which the robot seeks to achieve its goals consists of specially crafted ‘stick and ball’ versions of real experimental scenarios from animal reasoning (like tool use in chimps, novel tool construction in Caledonian crows, the classic trap tube paradigm and their possible combinations). We specifically focus on the progressive creation of the following internal models in the behavioral repertoire of the robot a) a passive motion paradigm based forward inverse model for mental simulation / real execution of goal directed arm (and arm+tool) movements; b) a spatial mental map of the playground; c) an internal model representing the causality of pushing objects and further learning to push intelligently in order to avoid randomly placed traps in the trapping groove. After presenting the computational architecture for the internal models, we demonstrate how the robot can use them to mentally compose a sequence of ‘Push-MoveReach’ in order to Grasp (an otherwise unreachable) ball in its playground.