Everyday robotic action: lessons from human action control

Robots are increasingly capable of performing everyday human activities such as cooking, cleaning, and doing the laundry. This requires the real-time planning and execution of complex, temporally extended sequential actions under high degrees of uncertainty, which provides many challenges to traditional approaches to robot action control. We argue that important lessons in this respect can be learned from research on human action control. We provide a brief overview of available psychological insights into this issue and focus on four principles that we think could be particularly beneficial for robot control: the integration of symbolic and subsymbolic planning of action sequences, the integration of feedforward and feedback control, the clustering of complex actions into subcomponents, and the contextualization of action-control structures through goal representations.

[1]  R Klein,et al.  Nonhierarchical control of rapid movement sequences: a comment on Rosenbaum, Kenny, and Derr. , 1983, Journal of experimental psychology. Human perception and performance.

[2]  S. T. Klapp,et al.  REACTION TIME ANALYSIS Of PROGRAMMED CONTROL , 1977, Exercise and sport sciences reviews.

[3]  M. Goodale,et al.  The visual brain in action , 1995 .

[4]  A. Clark Being There: Putting Brain, Body, and World Together Again , 1996 .

[5]  D. Spalding The Principles of Psychology , 1873, Nature.

[6]  Angelo Cangelosi,et al.  A cognitive robotic model of grasping , 2009, EpiRob.

[7]  David Badre,et al.  Cognitive control, hierarchy, and the rostro–caudal organization of the frontal lobes , 2008, Trends in Cognitive Sciences.

[8]  Moritz Tenorth,et al.  Knowledge Processing for Autonomous Robots , 2011 .

[9]  Donald A. Norman,et al.  Simulating a Skilled Typist: A Study of Skilled Cognitive-Motor Performance , 1982, Cogn. Sci..

[10]  Moritz Tenorth,et al.  Knowledge Processing for Autonomous Robot Control , 2012, AAAI Spring Symposium: Designing Intelligent Robots.

[11]  R. Cohen,et al.  Where grasps are made reveals how grasps are planned: generation and recall of motor plans , 2004, Experimental Brain Research.

[12]  A. Noë,et al.  A sensorimotor account of vision and visual consciousness. , 2001, The Behavioral and brain sciences.

[13]  Caroline Palmer,et al.  Incremental planning in sequence production. , 2003, Psychological review.

[14]  Iris van Rooij,et al.  Hierarchies in Action and Motor Control , 2012, Journal of Cognitive Neuroscience.

[15]  Bernhard Hommel,et al.  Grounding Attention in Action Control: The Intentional Control of Selection , 2010 .

[16]  Leona E. Tyler,et al.  The effect of practice. , 1947 .

[17]  Moritz Tenorth,et al.  CRAM — A Cognitive Robot Abstract Machine for everyday manipulation in human environments , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Wolfgang Maass,et al.  Movement Generation with Circuits of Spiking Neurons , 2005, Neural Computation.

[19]  L. Barsalou,et al.  Whither structured representation? , 1999, Behavioral and Brain Sciences.

[20]  Michael A. Khan,et al.  The Effect of Practice on the Control of Rapid Aiming Movements: Evidence for an Interdependency between Programming and Feedback Processing , 1998 .

[21]  J. Pratt,et al.  Practice and Component Submovements: The Roles of Programming and Feedback in Rapid Aimed Limb Movements. , 1996, Journal of motor behavior.

[22]  Brian Bruya Effortless attention : a new perspective in the cognitive science of attention and action , 2010 .

[23]  Stephen Monsell,et al.  Motor Programs and Hierarchical Organization in the Control of Rapid Speech , 1988 .

[24]  C. Prablanc,et al.  Automatic control during hand reaching at undetected two-dimensional target displacements. , 1992, Journal of neurophysiology.

[25]  D A Rosenbaum,et al.  Hierarchical control of rapid movement sequences. , 1983, Journal of experimental psychology. Human perception and performance.

[26]  S. Vijayakumar,et al.  The role of feed-forward and feedback processes for closed-loop prosthesis control , 2011, Journal of NeuroEngineering and Rehabilitation.

[27]  R. Daniloff,et al.  Coarticulation of lip rounding. , 1968, Journal of speech and hearing research.

[28]  T. Shallice,et al.  Hierarchical schemas and goals in the control of sequential behavior. , 2006, Psychological review.

[29]  K. Harris,et al.  Anticipatory coarticulation: some implications from a study of lip rounding. , 1979, The Journal of the Acoustical Society of America.

[30]  R. D Seidler,et al.  Feedforward and feedback processes in motor control , 2004, NeuroImage.

[31]  Panos E. Trahanias,et al.  Temporal Cognition: A Key Ingredient of Intelligent Systems , 2011, Front. Neurorobot..

[32]  L. Barsalou,et al.  Steering a course for embodied representation , 2000 .

[33]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[34]  C H Nakatani,et al.  A corpus-based study of repair cues in spontaneous speech. , 1994, The Journal of the Acoustical Society of America.

[35]  K. Lashley The problem of serial order in behavior , 1951 .

[36]  R. J. Weber,et al.  The parameter remapping effect in human performance: Evidence from tongue twisters and finger fumblers☆ , 1986 .

[37]  Angelo Cangelosi,et al.  The Mechanics of Embodiment: A Dialog on Embodiment and Computational Modeling , 2011, Front. Psychology.

[38]  C. Prablanc,et al.  Large adjustments in visually guided reaching do not depend on vision of the hand or perception of target displacement , 1986, Nature.

[39]  M. Arbib,et al.  Grasping objects: the cortical mechanisms of visuomotor transformation , 1995, Trends in Neurosciences.

[40]  David A Rosenbaum,et al.  The Cinderella of psychology: the neglect of motor control in the science of mental life and behavior. , 2005, The American psychologist.

[41]  D. Rosenbaum,et al.  Coordination of locomotion and prehension , 2006, Experimental Brain Research.

[42]  R. Schmidt A schema theory of discrete motor skill learning. , 1975 .

[43]  D. Plaut,et al.  Doing without schema hierarchies: a recurrent connectionist approach to normal and impaired routine sequential action. , 2004, Psychological review.

[44]  C. Eriksen,et al.  Implicit speech: Mechanism in perceptual encoding? , 1970 .

[45]  David A. Rosenbaum,et al.  Successive Approximations to a Model of Human Motor Programming , 1988 .

[46]  Martijn Wisse,et al.  Optimization of feedforward controllers to minimize sensitivity to model inaccuracies , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[47]  S. Glover,et al.  Separate visual representations in the planning and control of action , 2004, Behavioral and Brain Sciences.

[48]  G. Aschersleben,et al.  The Theory of Event Coding (TEC): a framework for perception and action planning. , 2001, The Behavioral and brain sciences.

[49]  B. Hommel Action control according to TEC (theory of event coding) , 2009, Psychological research.

[50]  James L. Lyons,et al.  Goal-directed aiming: two components but multiple processes. , 2010, Psychological bulletin.

[51]  B. Hommel,et al.  Intentional control of attention: action planning primes action-related stimulus dimensions , 2007, Psychological research.

[52]  Klas Nilsson,et al.  Industrial Robotics , 2008, Springer Handbook of Robotics.

[53]  C. Dugas,et al.  On the type of information used to control and learn an aiming movement after moderate and extensive training , 1987 .

[54]  M. Washburn Movement and Mental Imagery , 2009 .

[55]  G. Miller,et al.  Plans and the structure of behavior , 1960 .

[56]  George Houghton,et al.  The problem of serial order: a neural network model of sequence learning and recall , 1990 .

[57]  F. M. Henry,et al.  Increased Response Latency for Complicated Movements and A “Memory Drum” Theory of Neuromotor Reaction , 1960 .

[58]  Donald R. Gentner,et al.  Skilled Finger Movements in Typing. , 1981 .

[59]  Richard Fikes,et al.  STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving , 1971, IJCAI.

[60]  W. James,et al.  The Principles of Psychology. , 1983 .

[61]  L. A. Jeffress,et al.  Cerebral Mechanisms in Behavior , 1953 .

[62]  John R. Anderson,et al.  Rules of the Mind , 1993 .

[63]  B. Hommel,et al.  Intentional weighting: a basic principle in cognitive control , 2012, Psychological research.

[64]  Mark Steedman,et al.  Exploration and Planning in a Three-Level Cognitive Architecture , 2008 .

[65]  L. Barsalou Flexibility, Structure, and Linguistic Vagary in Concepts: Manifestations of a Compositional System of Perceptual Symbols , 2019, Theories of Memory.

[66]  Steven A. Jax,et al.  The problem of serial order in behavior: Lashley's legacy. , 2007, Human movement science.

[67]  Carol A. Fowler,et al.  Coarticulation and theories of extrinsic timing , 1980 .

[68]  Alan D Samples,et al.  SOAR Architecture , 1985 .

[69]  Bernhard Hommel,et al.  The Functional and Neural Mechanism of Action Preparation: Roles of EBA and FFA in Voluntary Action Control , 2011, Journal of Cognitive Neuroscience.

[70]  Agnieszka Wykowska,et al.  How you move is what you see: action planning biases selection in visual search. , 2009, Journal of experimental psychology. Human perception and performance.

[71]  S Serge Larochelle,et al.  Some aspects of movements in skilled typewriting , 1982 .