A computational model of selection by consequences.

Darwinian selection by consequences was instantiated in a computational model that consisted of a repertoire of behaviors undergoing selection, reproduction, and mutation over many generations. The model in effect created a digital organism that emitted behavior continuously. The behavior of this digital organism was studied in three series of computational experiments that arranged reinforcement according to random-interval (RI) schedules. The quantitative features of the model were varied over wide ranges in these experiments, and many of the qualitative features of the model also were varied. The digital organism consistently showed a hyperbolic relation between response and reinforcement rates, and this hyperbolic description of the data was consistently better than the description provided by other, similar, function forms. In addition, the parameters of the hyperbola varied systematically with the quantitative, and some of the qualitative, properties of the model in ways that were consistent with findings from biological organisms. These results suggest that the material events responsible for an organism's responding on RI schedules are computationally equivalent to Darwinian selection by consequences. They also suggest that the computational model developed here is worth pursuing further as a possible dynamic account of behavior.

[1]  J. Donahoe,et al.  The S-R issue: its status in behavior analysis and in Donahoe and Palmer's learning and complex behavior. , 1997, Journal of the experimental analysis of behavior.

[2]  Jens G. Reich.  C curve fitting and modeling for scientists and engineers , 1992 .

[3]  R. Herrnstein.  On the law of effect. , 1970, Journal of the experimental analysis of behavior.

[4]  J. J. McDowell,et al.  Falsification of matching theory: changes in the asymptote of Herrnstein's hyperbola as a function of water deprivation. , 1999, Journal of the experimental analysis of behavior.

[5]  Stephen Wolfram,et al.  Cellular Automata And Complexity , 1994 .

[6]  J W Donahoe,et al.  A selectionist approach to reinforcement. , 1993, Journal of the experimental analysis of behavior.

[7]  J. Elman,et al.  Rethinking Innateness: A Connectionist Perspective on Development , 1996 .

[8]  J. J. McDowell,et al.  Confirmation of linear system theory prediction: Changes in Herrnstein's k as a function of changes in reinforcer magnitude. , 1984, Journal of the experimental analysis of behavior.

[9]  John R. Koza,et al.  Hidden Order: How Adaptation Builds Complexity. , 1995, Artificial Life.

[10]  W. Baum.  Selection by consequences is a good idea , 1988, Behavioral and Brain Sciences.

[11]  Stephen Wolfram,et al.  A New Kind of Science , 2003, Artificial Life.

[12]  J. J. McDowell,et al.  Application of Herrnstein's hyperbola to time allocation of naturalistic human behavior maintained by naturalistic social reinforcement. , 1992, Journal of the experimental analysis of behavior.

[13]  G. Madden,et al.  Units of Interaction, Evolution, and Replication: Organic and Behavioral Parallels , 1995, The Behavior analyst.

[14]  W. Press,et al.  Numerical Recipes: The Art of Scientific Computing , 1987 .

[15]  S. Glenn,et al.  Functions of the Environment in Behavioral Evolution , 1994, The Behavior analyst.

[16]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[17]  J. J. McDowell,et al.  Falsification of matching theory's account of single-alternative responding: Herrnstein's k varies with sucrose concentration. , 2000, Journal of the experimental analysis of behavior.

[18]  David C. Palmer,et al.  Learning and Complex Behavior , 1993 .

[19]  J. J. McDowell,et al.  Matching Theory in Natural Human Environments , 1988, The Behavior analyst.

[20]  J. Kagel,et al.  Maximization theory in behavioral psychology , 1981, Behavioral and Brain Sciences.

[21]  W Vaughan,et al.  Melioration, matching, and maximization. , 1981, Journal of the experimental analysis of behavior.

[22]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .

[23]  Tony Curzon Price,et al.  Emergence: From Chaos to Order by John H. Holland , 1998, J. Artif. Soc. Soc. Simul..

[24]  D. Hull,et al.  A general account of selection: Biology, immunology, and behavior , 2001, Behavioral and Brain Sciences.

[25]  Computational behavior dynamics: an alternative description of Nevin (1969). , 1992, Journal of the experimental analysis of behavior.

[26]  B. F. Skinner,et al.  Selection by consequences , 1984, Behavioral and Brain Sciences.

[27]  J. J. McDowell,et al.  Confirmation of linear system theory prediction: Rate of change of Herrnstein's kappa as a function of response-force requirement. , 1985, Journal of the experimental analysis of behavior.