A Novel Task for the Investigation of Action Acquisition

We present a behavioural task designed for the investigation of how novel instrumental actions are discovered and learnt. The task consists of free movement with a manipulandum, during which the full range of possible movements can be explored by the participant and recorded. A subset of these movements, the ‘target’, is set to trigger a reinforcing signal. The task is to discover what movements of the manipulandum evoke the reinforcement signal. Targets can be defined in spatial, temporal, or kinematic terms, can be a combination of these aspects, or can represent the concatenation of actions into a larger gesture. The task allows the study of how the specific elements of behaviour which cause the reinforcing signal are identified, refined and stored by the participant. The task provides a paradigm where the exploratory motive drives learning and as such we view it as in the tradition of Thorndike [1]. Most importantly it allows for repeated measures, since when a novel action is acquired the criterion for triggering reinforcement can be changed requiring a new action to be discovered. Here, we present data using both humans and rats as subjects, showing that our task is easily scalable in difficulty, adaptable across species, and produces a rich set of behavioural measures offering new and valuable insight into the action learning process.

[1]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[2]  Sara Fripp A learning curve. , 2014, Midwives.

[3]  Yael Niv,et al.  Operant Conditioning , 1971 .

[4]  Christopher D. Adams,et al.  Instrumental Responding following Reinforcer Devaluation , 1981 .

[5]  G. Peterson A day of great illumination: B. F. Skinner's discovery of shaping. , 2004, Journal of the experimental analysis of behavior.

[6]  J. Bruner Organization of early skilled action. , 1973, Child development.

[7]  K. Lattal,et al.  Response acquisition with delayed reinforcement. , 1990, Journal of experimental psychology. Animal behavior processes.

[8]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[9]  Zoubin Ghahramani,et al.  Perspectives and problems in motor learning , 2001, Trends in Cognitive Sciences.

[10]  R. Morris Developments of a water-maze procedure for studying spatial learning in the rat , 1984, Journal of Neuroscience Methods.

[11]  E. Thelen The (re)discovery of motor development: Learning new things from an old field. , 1989 .

[12]  Allen and Rosenbloom Paul S. Newell,et al.  Mechanisms of Skill Acquisition and the Law of Practice , 1993 .

[13]  B. Hommel,et al.  Contiguity and contingency in action-effect learning , 2004, Psychological research.

[14]  Response acquisition with immediate and delayed conditioned reinforcement , 2005, Behavioural Processes.

[15]  W. J. Griffiths,et al.  Free-Operant Acquisition with Delayed Reinforcement , 1992 .

[16]  J. Stevens,et al.  Animal Intelligence , 1883, Nature.

[17]  A. Poling,et al.  The effects of behavioral history on response acquisition with immediate and delayed reinforcement. , 2004, Journal of the experimental analysis of behavior.

[18]  B. Skinner Contingencies of reinforcement : a theoretical analysis , 1969 .

[19]  E. Gibson Exploratory behavior in the development of perceiving, acting, and the acquiring of knowledge. , 1988 .

[20]  M. Jüptner,et al.  A review of differences between basal ganglia and cerebellar control of movements as revealed by functional imaging studies. , 1998, Brain : a journal of neurology.

[21]  Laurent Madelain,et al.  Reinforcing saccadic amplitude variability. , 2011, Journal of the experimental analysis of behavior.

[22]  Daniel A. Braun,et al.  Motor Task Variation Induces Structural Learning , 2009, Current Biology.

[23]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[24]  J. Bruner,et al.  The Coordination of Visual Observation and Instrumental Behavior in Early Infancy , 1973, Perception.

[25]  Richard S. Sutton,et al.  Reinforcement Learning with Replacing Eligibility Traces , 2005, Machine Learning.

[26]  J. Piek The role of variability in early motor development , 2002 .

[27]  Anthony Dickinson,et al.  The 28th Bartlett Memorial Lecture Causal Learning: An Associative Analysis , 2001, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[28]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[29]  Marco Mirolli,et al.  Intrinsically Motivated Learning in Natural and Artificial Systems , 2013 .

[30]  A. Neuringer,et al.  Operant variability and voluntary action. , 2010, Psychological review.

[31]  R. Ivry,et al.  Cerebellar involvement in anticipating the consequences of self-produced actions during bimanual movements. , 2005, Journal of neurophysiology.

[32]  A. Neuringer,et al.  Reinforcement of variations and repetitions along three independent response dimensions , 2002, Behavioural Processes.

[33]  E. Thorndike,et al.  Acquisition of a nose-poke response in rats as an operant , 1993 .

[34]  R. Leach The learning curve , 1992 .

[35]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.