The Integration of Reinforcements over Time

The title of this paper could as well have been “The Integration of Prey over Time.” We are concerned with a problem that is common to the experimental analysis of animal behavior as it occurs in Skinner boxes, and to the empirical investigation of foraging as it occurs in the animal’s natural environment. It is the problem of how a series of discrete events, deliveries of reinforcers or captures of prey, are transformed into an estimate of the density of reinforcements available on a schedule, or the density of prey present in a patch. All popular accounts both of foraging behavior and of behavior under schedules of reinforcement make use of the results of such an integration. Herrnstein’s’.’ matching law and “quantitative law of effect” predict that behavior under schedules of reinforcement will be a function of the reinforcement rates experienced under the schedules. Charnov’s’ marginal value theorem predicts that natural foraging behavior in a patchy environment will be a function of the rates a t which prey can be captured within each patch and within the environment as a whole. The functions concerned are different, but the problem is common to both. You or I , faced with the problem of estimating the rate a t which discrete events occur, would doubtless solve it by counting them, then dividing by the time that had elapsed, or the number of responses we had made. We have no evidence that animals can use this solution. So far as we can tell, the capacities of even the largest-brained birds to discriminate between numbers are quite m ~ d e s t . ~ (Other kinds of “counting” behavior, which appear to be much more a c c ~ r a t e , ” ~ turn out to be no different from the kind of noncounting mechanism we propose below.) We have no evidence at all that animals can manipulate numbers in the kind of way required for division. Besides, this solution ignores an additional, fundamental aspect of the problem: the density that is being estimated is liable to change. I n the laboratory we do, of course, change the schedules of reinforcement that our subjects experience. But, except i n the case of the transition to extinction, we very rarely study the effects of these transitions. Instead, we maintain the same conditions in force until we are sure we have achieved “steady-state” behavior, and look only at