Choice, optimal foraging, and the delay-reduction hypothesis

Abstract Behaving organisms are continually choosing. Recently the theoretical and empirical study of decision making by behavioral ecologists and experimental psychologists have converged in the area of foraging, particularly food acquisition. This convergence has raised the interdisciplinary question of whether principles that have emerged from the study of decision making in the operant conditioning laboratory are consistent with decision making in naturally occurring foraging. One such principle, the “parameter-free delay-reduction hypothesis, ” developed in studies of choice in the operant conditioning laboratory, states that the effectiveness of a stimulus as a reinforcer may be predicted most accurately by calculating the decrease in time to food presentation correlated with the onset of the stimulus, relative to the length of time to food presentation measured from the onset of the preceding stimulus. Since foraging involves choice, the delay-reduction hypothesis may be extended to predict aspects of foraging. We discuss the strategy of assessing parameters of foraging with operant laboratory analogues to foraging. We then compare the predictions of the delay-reduction hypothesis with those of optimal foraging theory, developed by behavioral ecologists, showing that, with two exceptions, the two positions make comparable predictions. The delay-reduction hypothesis is also compared to several contemporary pscyhological accounts of choice. Results from several of our experiments with pigeons, designed as operant conditioning simulations of foraging, have shown the following: The more time subjects spend searching for or traveling between potential food sources, the less selective they become, that is, the more likely they are to accept the less preferred outcome; increasing time spent procuring (“handling”) food increases selectivity; how often the preferred outcome is available has a greater effect on choice then how often the less preferred outcome is available; subjects maximize reinforcement whether it is the rate, amount, or probability of reinforcement that is varied; there are no significant differences between subjects performing under different types of deprivation (open vs. closed economies). These results are all consistent with the delay-reduction hypothesis. Moreover, they suggest that the technology of the operant conditioning laboratory may have fruitful application in the study of foraging, and, in doing so, they underscore the importance of an interdisciplinary approach to behavior.

[1]  J. M. Evvard Is the Appetite of Swine a Reliable Indication of Physiological Needs , 1915 .

[2]  E. Tolman,et al.  The organism and the causal texture of the environment. , 1935 .

[3]  R. M. Elliott,et al.  Behavior of Organisms , 1991 .

[4]  E. Brunswik Probability as a determiner of rat behavior. , 1939 .

[5]  B. Skinner,et al.  Some quantitative properties of anxiety , 1941 .

[6]  A. Maslow,et al.  The expressive component of behavior. , 1949, Psychological review.

[7]  B. Skinner,et al.  Are theories of learning necessary? , 2013, Psychological review.

[8]  R. A. Koelling,et al.  Conditioned aversion to saccharin resulting from exposure to gamma radiation. , 1955, Science.

[9]  C. B. Ferster,et al.  Schedules of reinforcement , 1957 .

[10]  L. Tinbergen The natural control of insects in pinewoods. I. Factors influencing the intensity of predation by songbirds , 1960 .

[11]  D. Falconer,et al.  Introduction to Quantitative Genetics. , 1962 .

[12]  L. Tinbergen The Natural Control of Insects in Pinewoods , 1960 .

[13]  R. Herrnstein SECONDARY REINFORCEMENT AND RATE OF PRIMARY REINFORCEMENT. , 1964, Journal of the experimental analysis of behavior.

[14]  R. Herrnstein APERIODICITY AS A FACTOR IN CHOICE. , 1964, Journal of the experimental analysis of behavior.

[15]  J. Staddon,et al.  Reinforcement as Input: Cyclic Variable-Interval Schedule , 1964, Science.

[16]  P. Rozin,et al.  Novel food preferences in thiamine-deficient rats. , 1966, Journal of Comparative and Physiological Psychology.

[17]  R. Macarthur,et al.  On Optimal Use of a Patchy Environment , 1966, The American Naturalist.

[18]  J. Emlen The Role of Time and Energy in Food Preference , 1966, The American Naturalist.

[19]  W. K. Honig Operant behavior : areas of research and application , 1966 .

[20]  M. Felton,et al.  The post-reinforcement pause. , 1966, Journal of the experimental analysis of behavior.

[21]  E Fantino,et al.  Preference for mixed- versus fixed-ratio schedules. , 1967, Journal of the experimental analysis of behavior.

[22]  R. Rescorla Probability of shock in the presence and absence of CS in fear conditioning. , 1968, Journal of comparative and physiological psychology.

[23]  P. Killeen On the measurement of reinforcement frequency in the study of preference. , 1968, Journal of the experimental analysis of behavior.

[24]  E. Fantino Choice and rate of reinforcement. , 1969, Journal of the experimental analysis of behavior.

[25]  R.G.B. Brown Seed Selection By Pigeons , 1969 .

[26]  M. Davison Preference for mixed-interval versus fixed-interval schedules. , 1969, Journal of the experimental analysis of behavior.

[27]  R. Lewontin The Units of Selection , 1970, The Structure and Confirmation of Evolutionary Theory.

[28]  Derek Goodwin,et al.  Pigeons and doves of the world , 1970 .

[29]  N Squires,et al.  A model for choice in simple concurrent and concurrent-chains schedules. , 1971, Journal of the experimental analysis of behavior.

[30]  P. Rozin,et al.  Specific hungers and poison avoidance as adaptive specializations of learning. , 1971, Psychological review.

[31]  T. Schoener Theory of Feeding Strategies , 1971 .

[32]  C. George Choix et apprentissage en situation aléatoire , 1971 .

[33]  H P Zeigler,et al.  Patterns of feeding behavior in the pigeon. , 1971, Journal of comparative and physiological psychology.

[34]  G. Collier,et al.  The ecological determinants of reinforcement in the rat. , 1972, Physiology & behavior.

[35]  D. Wahlsten,et al.  Genetic experiments with animal learning: a critical review. , 1972, Behavioral biology.

[36]  M. Davison Preference for mixed-interval versus fixed-interval schedules: number of component intervals. , 1972, Journal of the experimental analysis of behavior.

[37]  William M. Baum Choice in a continuous procedure , 1972 .

[38]  J. Siegel,et al.  Food and water intake and weight regulation in the pigeon. , 1972, Physiology & behavior.

[39]  R. K. Murton,et al.  Ecological Studies of the Feral Pigeon Columba livia Var. II. Flock Behaviour and Social Organization , 1972 .

[40]  E Fantino,et al.  Relative delay of reinforcement and choice. , 1973, Journal of the experimental analysis of behavior.

[41]  Preference for and effects of variable-as opposed to fixed-reinforcer duration. , 1974, Journal of the experimental analysis of behavior.

[42]  R J Herrnstein,et al.  Hunger and contrast in a multiple schedule. , 1974, Journal of the experimental analysis of behavior.

[43]  W. Timberlake,et al.  Response deprivation: An empirical approach to instrumental performance , 1974 .

[44]  J. Gibbon,et al.  Contingency spaces and measures in classical and instrumental conditioning. , 1974, Journal of the experimental analysis of behavior.

[45]  S. B. Kendall,et al.  Preference for intermittent reinforcement. , 1974, Journal of the experimental analysis of behavior.

[46]  N. Mackintosh The psychology of animal learning , 1974 .

[47]  W. Harris,et al.  Conditioned behavior in Drosophila melanogaster. , 1974, Proceedings of the National Academy of Sciences of the United States of America.

[48]  John Maynard Smith Models in ecology , 1974 .

[49]  H. Pulliam,et al.  On the Theory of Optimal Diets , 1974, The American Naturalist.

[50]  J. Altmann,et al.  Observational study of behavior: sampling methods. , 1974, Behaviour.

[51]  D. J. Hall,et al.  Optimal Foraging and the Size Selection of Prey by the Bluegill Sunfish (Lepomis Macrochirus) , 1974 .

[52]  J. Krebs,et al.  Hunting by expectation or optimal foraging: A study of patch use by chickadees , 1974 .

[53]  E A Wasserman,et al.  Pavlovian appetitive contingencies and approach versus withdrawal to conditioned stimuli in pigeons. , 1974, Journal of comparative and physiological psychology.

[54]  F. Mcsweeney Concurrent schedule responding as a function of body weight , 1975 .

[55]  Edmund Fantino,et al.  Self-control and general models of choice. , 1976 .

[56]  E. Charnov Optimal Foraging: Attack Strategy of a Mantid , 1976, The American Naturalist.

[57]  L. Ehrman,et al.  THE GENETICS OF BEHAVIOR , 1979 .

[58]  E. Charnov Optimal foraging, the marginal value theorem. , 1976, Theoretical population biology.

[59]  S E Lea,et al.  Titration of schedule parameters by pigeons. , 1976, Journal of the experimental analysis of behavior.

[60]  J. Hirsch,et al.  Succesful replication of, and selective breeding for, classical conditioning in the blowfly Phormia region , 1977, Animal Behaviour.

[61]  F. Gill,et al.  NONRANDOM FORAGING BY SUNBIRDS IN A PATCHY ENVIRONMENT , 1977 .

[62]  A. Kamil,et al.  Visual Detection of Cryptic Prey by Blue Jays (Cyanocitta cristata) , 1977, Science.

[63]  Body Fat and Physical Fitness , 1977 .

[64]  R. Luce,et al.  The Choice Axiom after Twenty Years , 1977 .

[65]  Graham H. Pyke,et al.  Optimal Foraging: A Selective Review of Theory and Tests , 1977, The Quarterly Review of Biology.

[66]  R. Cowie Optimal foraging in great tits (Parus major) , 1977, Nature.

[67]  J. T. Erichsen,et al.  Optimal prey selection in the great tit (Parus major) , 1977, Animal Behaviour.

[68]  J A Dinsmoor,et al.  Escape, avoidance, punishment: where do we stand? , 1977, Journal of the experimental analysis of behavior.

[69]  J. Hirsch,et al.  Behavior-genetic analysis of Phormia regina: conditioning, reliable individual differences, and selection. , 1977, Proceedings of the National Academy of Sciences of the United States of America.

[70]  J. Gibbon Scalar expectancy theory and Weber's law in animal timing. , 1977 .

[71]  A. Oaten,et al.  Optimal foraging in patches: a case for stochasticity. , 1977, Theoretical population biology.

[72]  W. K. Honig,et al.  Handbook of Operant Behavior , 2022 .

[73]  J. Goss‐Custard Optimal foraging and the size selection of worms by redshank, Tringa totanus, in the field , 1977, Animal Behaviour.

[74]  M. Domjan,et al.  Learning mechanisms in food selection , 1977 .

[75]  J. M. Smith,et al.  Optimization Theory in Evolution , 1978 .

[76]  G. Belovsky,et al.  Diet optimization in a generalist herbivore: the moose. , 1978, Theoretical population biology.

[78]  W. J. Donovan Structure and function of the pigeon visual system , 1978 .

[79]  B. J. Cockrell,et al.  Predator ingestion rate and its bearing on feeding time and the theory of oprimal diets , 1978 .

[80]  H. Anisman,et al.  Psychopharmacology of Aversively Motivated Behavior , 1978, Springer US.

[81]  R. Hughes,et al.  ENERGY MAXIMIZATION IN THE DIET OF THE SHORE CRAB, CARCINUS MAENAS , 1978 .

[82]  J. Krebs,et al.  Behavioural Ecology: An Evolutionary Approach , 1978 .

[83]  P. Taylor,et al.  Test of optimal sampling by foraging great tits , 1978 .

[84]  P. Maclean,et al.  Behavior and neurology of lizards , 1979 .

[85]  S. Lea Foraging and reinforcement schedules in the pigeon: Optimal and non-optimal aspects of choice , 1979, Animal Behaviour.

[86]  J. McNair A generalized model of optimal diets , 1979 .

[87]  Edmund Fantino,et al.  The experimental analysis of behavior : a biological perspective , 1979 .

[88]  J. Allison,et al.  Conservation in behavior , 1979 .

[89]  J. Staddon Operant behavior as adaptation to constraint. , 1979 .

[90]  A C Kamil,et al.  Search Image Formation in the Blue Jay (Cyanocitta cristata) , 1979, Science.

[91]  R. Hughes Optimal Diets under the Energy Maximization Premise: The Effects of Recognition Time and Learning , 1979, The American Naturalist.

[92]  H. Zeigler,et al.  Food preferences in the pigeon (Columba livia) , 1979, Physiology & Behavior.

[93]  La valeur adaptative des comportements aléatoires , 1979 .

[94]  J. Waage,et al.  Foraging for patchily-distributed hosts by the parasitoid, Nemeritis canescens , 1979 .

[95]  S. Gould,et al.  The spandrels of San Marco and the Panglossian paradigm: a critique of the adaptationist programme , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[96]  K. Waddington,et al.  Optimal Foraging: On Flower Selection by Bees , 1979, The American Naturalist.

[97]  A. M. Granda,et al.  Neural mechanisms of behavior in the pigeon , 1979 .

[98]  Percentage reinforcement and choice. , 1979, Journal of the experimental analysis of behavior.

[99]  S R Hursh,et al.  Economic concepts for the analysis of behavior. , 1980, Journal of the experimental analysis of behavior.

[100]  T. Caraco,et al.  An empirical demonstration of risk-sensitive foraging preferences , 1980, Animal Behaviour.

[101]  D. H. Morse,et al.  Behavioral Mechanisms in Ecology , 1980 .

[102]  P. Giller THE CONTROL OF HANDLING TIME AND ITS EFFECTS ON THE FORAGING STRATEGY OF A HETEROPTERAN PREDATOR, NOTONECTA , 1980 .

[103]  M. Sokolowski Foraging strategies ofDrosophila melanogaster: A chromosomal analysis , 1980, Behavior genetics.

[104]  Alan C. Kamil,et al.  Foraging behavior: ecological, ethological, and psychological approaches , 1980 .

[105]  A. Ōkubo,et al.  Di?usion and ecological problems: mathematical models , 1980 .

[106]  H. Terrace,et al.  Autoshaping and Conditioning Theory , 1980 .

[107]  D. J. Howell,et al.  Optimal Foraging in Glossophagine Bats: When to Give Up , 1980, The American Naturalist.

[108]  John R. Krebs,et al.  Optimal Foraging and Cryptic Prey , 1980 .

[109]  Thomas Caraco,et al.  ON FORAGING TIME ALLOCATION IN A STOCHASTIC ENVIRONMENT , 1980 .

[110]  J. McNair A stochastic foraging model with predator training effects. II. Optimal diets , 1981 .

[111]  John S. Young Discrete-trial choice in pigeons: Effects of reinforcer magnitude. , 1981, Journal of The Experimental Analysis of Behavior.

[112]  J. E. Mazur Optimization theory fails to predict performance of pigeons in a two-response situation. , 1981, Science.

[113]  E. Fantino Is maximization theory general, and is it refutable? , 1981, Behavioral and Brain Sciences.

[114]  Y. Iwasa,et al.  Prey Distribution as a Factor Determining the Choice of Optimal Foraging Strategy , 1981, The American Naturalist.

[115]  Biology of Behaviour: Mechanisms, Functions and Applications , 1981 .

[116]  R. Shull,et al.  Delay or rate of food delivery as determiners of response rate. , 1981, Journal of the experimental analysis of behavior.

[117]  H. M. Jenkins,et al.  Contingency in fear conditioning: A reexamination , 1981 .

[118]  Stephen José Hanson,et al.  Central place foraging in Rattus norvegicus , 1981, Animal Behaviour.

[119]  N. Davies,et al.  Owners and satellites: the economics of territory defence in the Pied Wagtail, Motacilla alba , 1981 .

[120]  J. Krebs,et al.  An introduction to behavioural ecology , 1981 .

[121]  J. Kagel,et al.  Maximization theory in behavioral psychology , 1981, Behavioral and Brain Sciences.

[122]  Donald J. Hall,et al.  The Role of Foraging Profitability and Experience in Habitat Use by the Bluegill Sunfish , 1981 .

[123]  E. Fantino,et al.  The delay-reduction hypothesis of conditioned reinforcement and punishment: Observing behavior. , 1981, Journal of the experimental analysis of behavior.

[124]  S S Pliskoff,et al.  Undermatching and overmatching: The fixed-ratio changeover requirement. , 1981, Journal of the experimental analysis of behavior.

[125]  A I Houston,et al.  How to maximize reward rate on two variable-interval paradigms. , 1981, Journal of the experimental analysis of behavior.

[126]  F. Toates,et al.  Analysis of Motivational Processes , 1981 .

[127]  A. Poling,et al.  Choice as a dependent measure in autoshaping: sensitivity to frequency and duration of food presentation. , 1982, Journal of the experimental analysis of behavior.

[128]  J. Staddon,et al.  Limits to action, the allocation of individual behavior , 1982 .

[129]  W M Baum,et al.  Choice, changeover, and travel. , 1982, Journal of the experimental analysis of behavior.

[130]  M. Rashotte,et al.  Associative influence on the foraging behavior of pigeons (Columba livia). , 1982, Journal of experimental psychology. Animal behavior processes.

[131]  John McNamara,et al.  A sequential approach to risk-taking , 1982, Animal Behaviour.

[132]  M. Davison,et al.  On the effects of component durations and component reinforcement rates in multiple schedules. , 1982, Journal of the experimental analysis of behavior.

[133]  B. L. Brown,et al.  Specification of the stimulus-reinforcer relation in multiple schedules: Delay and probability of reinforcement , 1982 .

[134]  Masato Ito,et al.  Choice behavior of rats in a concurrent-chains schedule: Amount and delay of reinforcement. , 1982, Journal of the experimental analysis of behavior.

[135]  J. Gray,et al.  Précis of The neuropsychology of anxiety: An enquiry into the functions of the septo-hippocampal system , 1982, Behavioral and Brain Sciences.

[136]  David Griffiths TESTS OF ALTERNATIVE MODELS OF PREY CONSUMPTION BY PREDATORS, USING ANT-LION LARVAE , 1982 .

[137]  Choice and multiple reinforcers. , 1982, Journal of the experimental analysis of behavior.

[138]  E. Fantino,et al.  Choice and the relative immediacy of reinforcement. , 1982, Journal of the experimental analysis of behavior.

[139]  M. Domjan The principles of learning and behavior , 1982 .

[140]  P R Killeen,et al.  Incentive theory: II. Models for choice. , 1982, Journal of the experimental analysis of behavior.

[141]  G. Zweers Pecking of the Pigeon (Columba Livia L.) , 1982 .

[142]  K. Waddington Optimal diet theory: sequential and simultaneous encounter models. , 1982 .

[143]  G. Zweers Drinking of the Pigeon (Columba Livia L.) , 1982 .

[144]  Choice, relative reinforcer duration, and the changeover ratio. , 1982, Journal of the experimental analysis of behavior.

[145]  David William Stephens Stochasticity in foraging theory: risk and information , 1982 .

[146]  E Fantino,et al.  Choice and foraging. , 1982, Journal of the experimental analysis of behavior.

[147]  Foraging in a simulated natural environment: There's a rat loose in the lab. , 1982, Journal of the experimental analysis of behavior.

[148]  N. Rowland Failure by deprived hamsters to increase food intake: some behavioral and physiological determinants. , 1982, Journal of comparative and physiological psychology.

[149]  T. Caraco Aspects of risk-aversion in foraging white-crowned sparrows , 1982, Animal Behaviour.

[150]  J. Dinsmoor Observing and conditioned reinforcement , 1983, Behavioral and Brain Sciences.

[151]  S. Shettleworth Memory in Food-Hoarding Birds , 1983 .

[152]  H A Metz,et al.  Continuous-time Markov chains as models for animal behaviour. , 1983, Bulletin of Mathematical Biology.

[153]  M. Abs Physiology and behaviour of the pigeon , 1983 .

[154]  Edmund Fantino,et al.  The delay-reduction hypothesis: Extension to three-alternative choice. , 1983 .

[155]  Stephen José Hanson,et al.  Regulation during challenge: A general model of learned performance under schedule constraint. , 1983 .

[156]  J. Staddon Adaptive behavior and learning , 1983 .

[157]  E Fantino,et al.  Choice: Some quantitative relations. , 1983, Journal of the experimental analysis of behavior.

[158]  Alan C. Kamil,et al.  Optimal Foraging Theory and the Psychology of Learning , 1983 .

[159]  James N. McNair,et al.  A Class of Patch-Use Strategies , 1983 .

[160]  M. Sokolowski,et al.  Drosophila larval foraging behavior. II. Selection in the sibling species,D. melanogaster andD. simulans , 1983, Behavior genetics.

[161]  D. Fulker,et al.  Genetic architecture of olfactory discriminative avoidance conditioning in Drosophila melanogaster. , 1983, Journal of comparative psychology.

[162]  M. Davison,et al.  On the effects of food deprivation and component reinforcer rates on multiple-schedule performance. , 1983, Journal of the experimental analysis of behavior.

[163]  The foraging behaviour of lacewing larvae on vertical rods , 1983, Animal Behaviour.

[164]  E. Fantino,et al.  Duration of components and response rates on multiple fixed-ratio schedules , 1983 .

[165]  C. C. Perkins,et al.  Advances in Analysis of Behavior, Vol. 2. Predictability, Correlation, and Contiguity , 1983 .

[166]  R. Mellgren,et al.  Animal cognition and behavior , 1983 .

[167]  D. Rider Preference for mixed versus constant delays of reinforcement: Effect of probability of the short, mixed delay. , 1983, Journal of the experimental analysis of behavior.

[168]  Jeffrey R. Lucas,et al.  The Role of Foraging Time Constraints and Variable Prey Encounter in Optimal Diet Choice , 1983, The American Naturalist.

[169]  M. Sokolowski,et al.  Larval foraging behavior in isofemale lines of Drosophila melanogaster and D. pseudoobscura , 1984 .

[170]  H. Normile,et al.  Relation between food and water intake in the pigeon (Columba livia). , 1984 .

[171]  W. Timberlake A temporal limit on the effect of future food on current performance in an analogue of foraging and welfare. , 1984, Journal of the experimental analysis of behavior.

[172]  C. Barnard,et al.  Prey selection by plovers: Optimal foraging in mixed-species groups , 1984, Animal Behaviour.

[173]  G. A. Clark,et al.  Perspectives in Ornithology , 1984 .

[174]  A. Poling,et al.  Preference in pigeons given a choice between sequences of fixed-ratio schedules: Effects of ratio values and duration of food delivery. , 1984, Journal of the experimental analysis of behavior.

[175]  J. Gibbon,et al.  Timing and time perception. , 1984, Annals of the New York Academy of Sciences.

[176]  Edmund Fantino,et al.  Percentage reward in an operant analogue to foraging , 1985, Animal Behaviour.

[177]  M Davison,et al.  Performance in continuously available multiple schedules. , 1985, Journal of the experimental analysis of behavior.

[178]  C. L. Gass,et al.  Specialization by territorial hummingbirds on experimentally enriched patches of flowers: energetic profitability and learning , 1985 .

[179]  S. Shettleworth Handling time and choice in pigeons. , 1985, Journal of the experimental analysis of behavior.

[180]  Choice, foraging, and reinforcer duration. , 1986, Journal of the experimental analysis of behavior.

[181]  P. Klopfer,et al.  Perspectives in Ethology , 1973, Springer US.