Department of Psychology, St. Cloud State University,. St. Cloud, Minnesota 56301. procedures used may have differential effects on behavior. This is especially ...
1981, 36, 221-229
JOURNAL OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR
NUMBER
2
(sEPTEmBER)
PREFERENCE FOR SIGNALED VERSUS UNSIGNALED REINFORCEMENT DELAY IN CONCURRENT-CHAIN SCHEDULES A. J. M. MARCATTILIO AND RALPH W. RICHARDS ST. CLOUD STATE UNIVERSITY AND COLORADO STATE UNIVERSITY
A concurrent-chain schedule was employed to examine pigeons' preferences for signaled versus unsignaled delay of reinforcement in which the delay durations ranged from zero to ten seconds. In general, pigeons preferred signaled delay over unsignaled delay especially when a variable-interval 30-second schedule operated in each initial link; when a variableinterval 90-second schedule operated in each initial link, these preferences tended toward indifference or were attenuated. In addition, prior training seemed to exert partial control over behavior. Responding in the terminal link was higher under signaled delay than unsignaled delay in a majority of the cases. Moreover, response rates under signaled delay remained fairly constant whereas responding under unsignaled delay was initially high, but decreased systematically with delay durations as short as 2.5 seconds. These results are consistent with a number of other studies demonstrating the significant role of a signal for impending positive stimuli. Key words: signaled reinforcement delay, unsignaled reinforcement delay, choice behavior, concurrent-chain schedules, key peck, pigeons
Over the years, much attention has been allocated to the effects of delayed reinforcement on various aspects of behavior. Early studies by Grice (1948), Perin (1943), and Spence (1947) demonstrated how delayed reinforcement affects the acquisition of new behavior. More recent investigations have examined the effects of delayed reinforcement on maintained response rates (Dews, 1960; Ferster, 1953; Pierce, Hanford, & Zimmerman, 1972; Richards, 1972, 1981; Sizemore & Lattal, 1977, 1978; Williams, 1976), and on postdiscrimination gradients of stimulus control (Richards, 1973; Richards & Hittesdorf, 1976, 1978; Richards & Marcattilio, 1978). Still other inquiries have been concerned with defining a quantitative relationship between responding and the length of the delay interval in choice situations (Chung, 1965; Chung & Herrnstein, 1967; Fantino, 1969). Procedurally, there are many ways in which to manipulate reinforcement delay, and the In addition to the anonomous reviewers, the authors wish to thank J. A. Nevin, E. Fantino, P. Killeen, and S. Harder for their valuable assistance and comments throughout the course of preparing this manuscript. Reprints may be obtained from A. J. M. Marcattilio, Department of Psychology, St. Cloud State University, St. Cloud, Minnesota 56301.
221
procedures used may have differential effects on behavior. This is especially true with regard to whether or not the delay period is signaled (i.e., whether or not an exteroceptive stimulus is presented during the delay interval). For example, Tarpy and Koster (1970) found that rats learned to escape shock much faster when the negative reinforcement delay interval was signaled compared to when it was unsignaled. Results from two separate studies suggest that responding under conditions of signaled delay decreases as the delay interval increases (Pierce et al., 1972) and that similar results are found under conditions of unsignaled delay except that the latter procedure maintains lower response rates especially at shorter delay durations (as short as 3-sec; Williams, 1976). Recently, Richards (1981) confirmed these general findings in a within-subjects design. In addition, these studies also indicate that responding maintained by unsignaled delay procedures is more variable than responding maintained by signaled delay. Despite the wealth of information this research has contributed, no research to date has examined pigeons' preferences for signaled versus unsignaled reinforcement delay. Such was the purpose of the present investigation.
222
A. J. M. MARCATTILIO and RALPH W. RICHARDS
The procedure employed to make this assessment was the concurrent-chain schedule. In this procedure, the pigeon is presented with two concurrently available response keys that are transilluminated by a stimulus associated with the initial link of the chain. When pecking on one of these keys occasionally produces the terminal link stimulus, the other key becomes dark and nonfunctional. Responding on the key associated with the operating terminal link is then reinforced according to some schedule. The initial links are again presented upon completion of the terminal link and the cycle begins anew. A standard dependent variable is the proportion of responses on each key during the initial links which is referred to as the choice proportion. The concurrentchain procedure is a powerful and sensitive technique for determining preference for one food delivery regimen over another (see Fantino, 1977 for review). METHOD
Sub jects Four experimentally naive, adult, female White Carneaux pigeons were maintained at approximately 80% of their free-feeding weights. One bird (P-5736) died after exposure to five conditions, but the data collected up to that point are included in the present analyses. Apparatus The standard three-key operant conditioning chamber (Lehigh Valley Electronics; internal dimensions of 35 cm long, 30 cm wide, and 35 cm high) contained a white houselight (CM 1820) above each key and a yellow pilot light (CM 1829) on the back wall. Access to the mixed grain reinforcer was timed from when the bird's head disrupted a photo beam located just inside the food aperture. Electromechanical scheduling and recording equipment were located in an adjacent room.
Procedure After key pecking was established by the method of successive approximations, the frequency of reinforcement was gradually reduced until a VI 30-sec schedule was attained. During these sessions (50 earned reinforcers per session), the side keys were covered with tape, the center key was illuminated by a
white light, and reinforcement (2-sec access to grain) was not delayed. Subjects were then trained on various concurrent-chain schedules. In these sessions, the center key was covered with tape and the side keys were uncovered. During the initial links, both keys were functional and were illuminated by a white light, but during either terminal link only one key was illuminated and functional. The terminal link on the left (right) key was associated with a red (green) key light for P-5353 and P-5736; key colors were reversed for P-595 and P-4810. Throughout, identical VI 60-sec schedules were associated with each terminal link, and in most conditions the presentation of reinforcement was delayed. During signaled delay periods the key light and house lights were darkened and the yellow pilot light was illuminated. At the end of the delay period, the yellow pilot light was darkened and the reinforcer was presented. During unsignaled delay periods no stimulus change occurred, and the reinforcer was presented as soon as the delay interval expired. Responding during either type of delay period was nonfunctional. After presentation of the reinforcer, the initial links were again presented. A 2-sec changeover delay (COD, cf. Herrnstein, 1961; see also Navarick, 1979) was employed to prevent accidental reinforcement of initial link responding on one key by presentation of the terminal link on the other key. The COD assured that a terminal link was not presented during the first 2 sec after a bird changed the side key on which it was responding. Table 1 shows the conditions, the number of sessions in each condition, the nature of the delay (S = signaled delay, U = unsignaled delay) contingencies in the terminal links, and the terminal link response rates. A VI 30-sec schedule operated in each initial link during the first six (five) conditions for P-5353 (P4810, P-595, P-5736), whereas a VI 90-sec schedule operated in each initial link during the remaining conditions. All initial and terminal link schedules were generated according to a constant probability progression (Catania & Reynolds, 1968). Each condition was maintained for at least 15 sessions and was only changed when visual inspection (conducted every five sessions) indicated that no consistent change in the distribution of initial link responses between the two keys was occurring.
SIGNALED VS. UNSIGNALED DELAY
223
Table 1 Order of conditions, number of sessions, type of delay associated with each terminal link, and average response rates in each terminal link. Terminal Link Terminal Link Delay Contingency Responsce Rates (delay in sec) (response/sec) Bird Cond. Sessions LK RK LK RK P-5353
1 2 3 4 5 6 7 8 9 10 11 12 13 14
P-595 2 3 4 5 6 7 8 9 10 11
12 13 P-4810 2 3 4 5 6 7 8 9 10 11 12 P-5736
1
2 3 4 5
25 15 15 15 20 15 15 20 20 20 20 35 20 20
U-10 S-10 U-10 U-2.5 U-.25 0 0 S-10 0 S-10 S-10
25 30 15 20 15 15 30 20 20 20 25 20 20
25 15 15 20 20 15 15 15 20 20 40 15
25 15 15 20 20
RESULTS Choice proportions and obtained relative immediacy of reinforcement. Figure 1 shows the percent of responses (unfilled circles) and time (filled circles) allocated to the right key during the initial links in which a concurrent VI 30-sec VI 30-sec schedule was operational.
S-10 U-10 S-10 S-2.5 S-.25 0 0 U-10 0 U-5 U-2.5 U-10 0 S-10
.12 .60 .11 .36 .66 1.18 1.27 .90 1.35 .94 .95 .99 1.29 .22
.50 .10 .59 .83 .97 1.12 1.21 .38 1.42 .99 1.16 .42 1.11 1.08
U-10 S-10 S-5 S-.25 0 0 U-10 0 U-10 U-5 U-10 0 U-10
S-10 U-10 U-5 U-.25 0 0 S-10 0 S-10
.15 1.05 1.15 1.29 1.45 1.83 1.29 1.70 1.23 1.07 1.13 1.69 1.09
.79 .08 .19 1.76 2.02 2.18 .91 1.92 .90 .94 1.41 1.81 1.40
U-10 S-10 S-2.5 S-.25 0 0 U-10 0 U-5 U-2.5 U-1.0 U-10
5-10 U-10 U-2.5 U-.25
.54 .09 .45 1.12 1.04 1.63 1.12 1.21
S-10
.13 .57 .78 .77 1.06 1.10 .12 .89 .10 .11 .73 .20
U-10
S-10 U-10 U-2.5 U-.25 0
.07 .66 .80 .88 .48
.64 .05 .04 .43 .50
S-10
0 U-10
S-10
S-2.5 S-.25 0
s-o 0 S-10 S-10
0 0 S-10 0 S-10 S-10
S-10
.92 .85 .98 .92
Also presented is the obtained relative immediacy of reinforcement (triangles) associated with the right key. The delay contingencies, and respective durations, operating in each terminal link are noted at the top of each panel during each condition (noted in parentheses). The shaded areas indicate those conditions in which the right key was associated
A. J. M. MARCATTILIO and RALPH W. RICHARDS
224
80
rc
....-....
60~~~~~~~0 -e~ 20 0000 ~~~~~~~~~~~0
40
w
-
10
()s.io, wi uuio
7
(2)
60
ojw4
A
~
~~
'a w
0
8.1O, U.s0
0(3 S.51U.5
0g000< -^^
*
----------~ A2
~
~
(4) S.25;U.25
()S.0SUo
-
00E
~
~
~
~
w*~0
0
10~~~~~~~~~~~~~~~~~~~~~
A~~~~~~~~~~~~~~
co
igN~~~~~~~~~~~~~~*.
00o
o'0
W
A
(1) S-.10 UnI0
oS42 *
(3OS-)S2A I.5
(2) 5I1
~~~ ~~~~
........
0 *
o
(4) S-.25;U-.25
~A
(5)
Soo;UuO
A
p.~ e
2o:
SESSIONS Fig. 1. Choice proportions and relative immediacy of reinforcement to the right key during conditions in which a VI 30-sec schedule operated in each initial link. S = signaled delay and U = unsignaled delay.
with the signaled delay and the hatched areas indicate conditions in which no delay was in effect. The horizontal line at the 50% level indicates a) no preference between the alternatives presented during the initial links and b) equal obtained durations of the terminal links. P-5353 exhibited a position preference for the left key in Condition 1, but during Conditions 2 through 5 relatively more behavior was allocated to the key associated with signaled delay contingencies. P-595 allocated more be-
havior to the right key in Condition 1 when it was associated with signaled delay. With a reversal of contingencies in Condition 2, P595 continued to allocate more behavior to the right key during the initial sessions but showed little if any preference by the end of Condition 2. However, in Conditions 3 and 4 a definite preference was shown for the left key associated with signaled delay. P-4810 displayed a systematic preference for the key associated with signaled delay during the first
SIGNALED VS. UNSIGNALED DELAY four conditions of training. During Condition 1 a slight preference for the right key and signaled delay was shown by P-5736 although responding was quite variable. In Conditions 2, 3, and 4, however, preferences for the left key associated with signaled delay were evident. In the no delay condition [6 (P-5353); 5 (P-595, P-4810, P-5736)] all four subjects continued to allocate more behavior to the key associated with signaled delay in the three previous
conditions.
As far as relative immediacy of reinforcement is concerned, examination of Figure 1 reveals that during most sessions the relative immediacy of reinforcement obtained in the terminal links was approximately equal. Except for a few sessions in which the subjects responded exclusively on one or the other key, and hence obtained 100%7 of their reinforcers for responding oni that key, the data reveal no longer delay systematic trends for shorter to primary reinforcement. Figure 2 shows the percent of responses (unfilled circles) and time (filled circles), and the obtained relative immediacy of reinforcement (triangles) associated with the right key averaged across the last five sessions of each condition for the three subjects exposed to a concurrent VI 90-sec VI 90-sec schedule during the initial links. As with Figure 1, the delay contingencies and their respective durations are noted for each condition. Again, shaded panels indicate when the right key was associated with signaled delay and hatched panels indicate conditions in which no delay was in effect. For purposes of clarity, the standard error of the mean is written in each panel for percent responses, percent time, and relative reinforcement immediacy from top to bottom respectively. These data can be summarized briefly. In the first condition of concurrent VI 90-sec VI 90-sec training, all three subjects contintued to allocate more behavior to the same key preferred in the previous condition. With subsequent training in these conditions, however, the choice proportions exhibited by P-5353 and P-595 tended toward indifference; P-4810 continued to prefer signaled delay, albeit such preferences were not as extreme as when the VI 30-sec schedules were in operation. Finally, obtained relative immediacy of reinforcement associated with each terminal link remained equal throughout this training. or
225
Terminal link response rates. The last column in Table 1 reports the average response rates in the terminal links for the last five sessions of each condition. In all but two conditions (10 and 11), P-5353 showed substantially higher response rates under signaled delay than unsignaled delay contingencies. When no delay was in effect (Conditions 6, 7, 9, and 13), there was minimal difference in terminal link response rates for this subject. P-595 showed similar patterns of responding in that signaled delay contingencies maintained higher terminal link response rates in all but four conditions. Only in Conditions 4, 7, 9, and 10 was responding higher in the unsignaled delay terminal link. When no delay was in effect, P-595 responded more rapidly in the terminal link that had maintained higher response rates in the immediately preceding condition. With the exception of Condition 4, P-4810 responded substantially higher in the terminal links associated with signaled delay. Under the no delay conditions, there was no difference in terminal link response rates in one (Condition 5) and higlher responding on the right key in the remaining two (Conditions 6 and 8). P-595 showed higher responding when the terminal link was associated with signaled delay in all comparisons and no difference in the one no delay condition presented to this subject. The bottom panel of Figure 3 depicts the terminal link response rates under signaled delay (filled circles) and unsignaled delay (unfilled circles) contingencies averaged across all birds in conditions where equal delay durations (see X-axis) were programmed. The brackets indicate the standard error of the mean. [Data from Richards (1981) have been replotted in the top panel for comparison pturposes and are discussed below.] In general, terminal link response rates were virtually equal under both types of delay contingencies when the delay duration was 0 and .25 seconds. However, when the delay duration was increased to 2.5, 5.0, and 10.0 sec, response rates under unsignaled delay contingencies decreased substantially while responding under signaled delay contingencies remained at high levels. In addition, responding under unsignaled delay contingencies was more variable relative to responding maintained by signaled delay contingencies as indicated by the standard errors of the mean.
226
A. J. M. MARCATTILIO and RALPH W. RICHARDS
c s I
II
rT -I 9--n 1F1 -I0t ''12 13'-1 14 10 -~~~~~E 0 to 04 t I 0 10 S+L5522* 01I10 0-
100 o 1.87 2.9 1.0 .7 4.8 .9 80 1.7 .8
z F
60
z
Iki L
z>40 Y w
52r52 gS2
I
m
0-
.9 .8 1L1 1.3 .7
.6
2.3 1@7
/
2 60
Ic. 11 i
II
7
o0Io
Z
//
P-5353/
0 1 10.i
z
IA
.
0
c
06 1j 7
-
f/I
0
0
I/i
A
.4 1.4 1.7 1.3 .8 1.8 1.8 1.7 .5 .3 .4 S
.4 .5 .7
'10
0.1 10
6
1.3 I.Z
.8
.6
01 5 1 WT0 1.7 2.1 .5 I.5 .8 32 .9 2.4
IA
.9
Wi
>- 40
o30 X 0
6 tFi 60 c 0
0
I
a
v
10
0
to
IV
11
I
10 -10-
I
10 0 10 2.5 1 2.6 .7 5.6 27 17 -1.7 3A 2-3 3.6 1.7 IA 105 .7 2.0 .7 0
zsa W o
o
...
.............
:,
...
i
c°- 40
p"
.. ...
a 200 La
..............
:944,
......
.............
p, ................
p:
...
ip-4810
)I
..........
Fig. 2. Choice proportions and relative immediacy of reinforcement a VI 90-sec schedule operated in each initial link. C
=
condition, S
=
to the right key during conditions in which signaled delay, and U = unsignaled delay.
SIGNALED VS. UNSIGNALED DELAY .01 }
~~RICHARDS (1980)
J
.8-
.6 w
:
.4 _
w
.2 _>q
z
2
0
.5
1.0
2.5
5.0
1.6
w
1.4z
1.2-
;Ia 1.0 I
.6 .4 .2 -U-o O
*
.. 0
.25
2.5
5.0
lo.o
DELAY DURATION (sec)
Fig. 3. Bottom Panel-response rates in the terminal links of the concurrent-chain schedule averaged across birds and conditions in which unsignaled (U) delay duration was equal to signaled (S) delay duration. Top Panel-replot of Richards (1981).
DISCUSSION The purpose of the present investigation was to examine pigeons' preferences for signaled versus unsignaled reinforcement delay in concurrent-chain schedules. Examination of the data presented in Figures 1 and 2 indicate that, in general, pigeons in the present study showed a preference for signaled delay contingencies albeit such preferences were complexly determined. During the first few conditions of training in whiclh a concurrent VI 30-sec VI 30-sec schedule operated during the initial links, all birds exhibited definite preferences for the signaled delay component. As training progressed, however, the effects of one condition carried over to the next. For example, P-5353 exhibited a strong preference for signaled delay associated witlh the right key in Condition 5, and continued to prefer the right key during Conditions 6 and 7 despite the fact that no delay contingencies were operating. A choice proportion around 50% indicating indifference would be expected in the latter two conditions. Other indications of such carry-over effects can be seen in the choice behavior exhibited by P-595 and P-4810 from Condition 4 to Conditions 5 and 6 and by P-5736 from
227
Condition 4 to Condition 5. These carry-over effects are reminiscent of effects of prior training or hysteresis, and whether or not such effects represent a lack of sensitivity to current reinforcement conditions (Davison & Hunter, 1979; Keller & Gollub, 1977) or bias produced by exposure to previous reinforcement conditions (de Villiers, 1977) is a question that can only be answered by future investigations. Hysteresis effects do not explain all the findings, however. As can be seen in Figures 1 and 2, the large preference for signaled delay exhibited early in training tended toward indifference (P-5353, P-595) or became somewhat attenuated (P-4810) concomitant with a change from concurrent VI 30-sec VI 30-sec to concurrent VI 90-sec VI 90-sec in the initial links. There are a number of possible explanations for this observation. For example, it may be that slow responding in the terminal link correlated with unsignaled delay contingencies generalized to the initial link associated with the same response key. That is, induction effects within the chained sclhedule programming unsignaled delay contingencies may have contributed to the greater relative behavioral allocation to the signaled delay schedule. Furthermore, these induction. effects could have dissipated with extended training (transient induction) which would result in trends towards indifference between signaled and unsignaled delay. This is unlikely, however, since time allocation, which is free from such contamination, mirrored response allocation in virtually all conditions. A second possible explanation appealing to changes in relative immediacy of reinforcement is not a viable notion regarding the observed preference structures because as Figures 1 and 2 show, the obtained durations of the terminal links were relatively equal throughout training. Perhaps a more viable speculation is to assume that the exteroceptive stimulus presented during the signaled delay period acquires secondary reinforcement properties because the signal terminates concomitantly with food presentation, and the choice behavior observed reflects a preference for slightly more immediate secondary reinforcement relative to sliglhtly more delayed primary reinforcement. Moreover, just as animals show an attenuation of large preferences for certain schedules of primary reinforcement when the initial links change from very short duration (a few sec-
228
A. J. M. MARCATTILIO and RALPH W. RICHARDS
onds) to very long duration (many minutes) initial links (see Fantino, 1977), pigeons in the present study showed an attenuation of preference for secondary reinforcement with a change from shorter (VI 30-sec) to longer (VI 90-sec) initial link durations. In a series of studies conducted by Fantino and his associates (e.g., Duncan & Fantino, 1970; See Fantino, 1977 for review), it has been found that pigeons usually prefer terminal links in which a single exteroceptive stimulus is presented relative to an equal duration terminal link in which two or more stimuli are employed. That is, pigeons tend to prefer unsegmented over segmented terminal links. Observations of behavioral allocation from the early conditions of the present study suggest that this is not always the case. The signaled delay terminal link-the one with two stimuli -was preferred over the unsignaled delay terminal link in which only one exteroceptive stimulus was presented. At this juncture, a satisfactory explanation of this discrepancy is somewhat elusive given the original intent and procedures of the present study. Procedural differences (e.g., the use of a COD here) would seem too trivial to be responsible for such differences. Locus of stimulation (on-key vs pilot light illumination on the chamber's back wall) also has little intuitive appeal as a potential explanatory mechanism. Consequently, it would seem that systematic research is required to delineate the necessary and sufficient conditions that affect preference between segmented and unsegmented schedules. Such research would fill a large gap in the comprehensive understanding of choice behavior in concurrent-chain schedules. Another striking feature of the present findings is the difference in terminal link response rates maintained by signaled and unsignaled delay contingencies. The data presented in Figure 3 show that the present findings replicate observations recently made by Richards (1981) which indicate that under conditions of no delay (0 sec) and very short delay (.25 sec), there is virtually no difference in responding between components. With slightly longer delay durations, however, responding during unsignaled delay decreases systematically whereas responding during signaled delay remains unchanged (at least up to 10 sec). The present findings are also in agreement with Williams' (1976) suggestion and Richards'
finding that unsignaled delay response rates are more variable than signaled delay response rates. There seem to be two explanations for these effects that are not necessarily mutually exclusive. First of all, it could be hypothesized that signaled delay contingencies maintain higher response rates in the terminal link than unsignaled delay contingencies because of the effects of (immediate) conditioned reinforcement associated with the signal (see above). On the other hand, as Richards (1981) has suggested, unsignaled delay might maintain lower response rates because behavior other than key pecking that occurs during the delay period may be adventitiously reinforced. This behavior then generalizes to nondelay portions of the schedules and competes with key-pecking behavior. Such an explanation would account for the greater variability in unsignaled delay responding. A two-factor notion, of course, would postulate that both mechanisms are responsible for these observations. The present findings are in conjunction with other research showing the importance of signaling reinforcement without the inclusion of the delay period. Williams and Fantino (1978) presented data that showed preference for signaled reinforcement over unsignaled reinforcement (when compared across conditions) in concurrent-chain schedules. Branch (1977) demonstrated that if the outcome of a chained schedule is signaled, responding is higher than when it was unsignaled. Lewis, Lewin, Muehleisen, and Stoyak (1974) showed that pecking was higher during one component of a multiple schedule in which reinforcement was signaled by a tone relative to a second component in which no tone was presented. In a choice situation pigeons also preferred signal to no signal conditions (see also Badia, Ryan, & Harsh, 1981 for similar findings, but Wilkie, 1973 for conflicting results). Whatever variables are responsible for these behavioral effects, the present findings in conjunction with those cited above clearly demonstrate that signaling reinforcement with or without the inclusion of a delay period is a powerful way to control behavior across a wide variety of
settings. REFERENCES Badia, P., R.yan, K., & Harsh, J. Choosing schedules of signaled appetitive events over schedules of un-
SIGNALED VS. UNSIGNALED DELAY signaled ones. Journal of the Experimental Analysis Behavior, 1981, 35, 187-195. Branch, M. N. Signalled and unsignalled percentage reinforcement of performance under a chained schedule. Journal of the Experimental Analysis of Behavior, 1977, 27, 71-83. Catania, A. C. & Reynolds, G. S. A quantitative analysis of the responding maintained by interval schedules of reinforcement. Journal of the Experimental Analysis of Behavior, 1968, 11, 327-383. Chung, S. H. Effects of delayed reinforcement in a concurrent situation. Journal of the Experimental Analysis of Behavior, 1965, 8, 439-444. Chung, S. H., & Herrnstein, R. J. Choice and delay of reinforcement. Journal of the Experimental Analy-
sis of Behavior, 1967, 10, 67-74. Davison, M. C., & Hunter, I. W. Concurrent schedules: Undermatching and control by previous experimental conditions. Journal of the Experimental Analysis of Behavior, 1979, 32, 233-244. de Villiers, P. Choice in concurrent schedules and a quantitative formulation of the law of effect. In W. K. Honig & J. E. R. Staddon (Eds.) Handbook of operant behavior. Englewood Cliffs, N.J.: PrenticeHall, 1977. Dews, P. B. Free-operant behavior under conditions of delayed reiniforcement: I. CRF-type schedules. Journal of the Experimental Analysis of Behavior, 1960, 3, 221-234. Duncan, B., & Fantino, E. Choice for periodic schedules of reinforcement. Journal of the Experimental Analysis of Behavior, 1970, 14, 73-86. Fantino, E. Choice and rate of reinforcement. Journal of the Experimental Analysis of Behavior, 1969, 12,
723-730. Fantino, E. Conditioned reinforcement: Choice and information. In W. K. Honig & J. E. R. Staddon (Eds.) Handbook of operant behavior. Englewood Cliffs, N.J.: Prentice-Hall, 1977. Ferster, C. B. Sustained behavior under delayed reinforcement. Journal of Experimental Psychology, 1953, 45, 218-224. Grice, G. R. The relation of secondary reinforcement to delayed reward in visual discrimination learning. Journal of Experimental Psychology, 1948, 38, 1-16. Herrnstein, R. J. Relative and absolute strength of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior, 1961, 4, 267-272. Keller, J. V., & Gollub, L. R. Duration and rate of reinforcement as determinants of concurrent responding. Journal of Experimental Analysis of Behiavior, 1977, 28, 145-153. Lewis, P., Lewin, L., Muehleisen, P., & Stoyak, M. Preference for signalled reinforcement. Jounral of the Experimental Analysis of Behavior, 1974, 22, 143-150. Navarick, D. J. Free-operant choice behavior: A molecular analysis. Journal of the Experimental Analysis of Behavior, 1979, 32, 213-232.
229
Perin, C. T. A quantitative investigation of the delayof-reinforcement gradient. Journal of Experimental Psychology, 1943, 32, 37-51. Pierce, C. H., Hanford, P. V., & Zimmerman, J. Effects of different delay of reinforcement procedures on variable-interval responding. Journal of the Experimental Analysis of Behavior, 1972, 18, 141-146. Richards, R. W. Reinforcement delay: Some effects on behavioral contrast. Journal of the Experimental Analysis of Behavior, 1972, 17, 381-394. Richards, R. W. Stimulus generalization and delay of reinforcement during one component of a multiple schedule. Journal of the Experimental Analysis of Behavior, 1973, 19, 303-309. Richards, R. W. A comparison of signaled and unsignaled delay of reinforcement. Journal of the Experimental Analysis of Behavior, 1981, 35, 145152. Richards, R. W., & Hittesdorf, W. M. Reinforcement delay: A parametric study of effects within a multiple schedule. Bulletin of the Psychonomic Society, 1976, 7, 303-305. Richards, R. W., & Hittesdorf, W. M. Inhibitory stimulus control under conditions of signaled and unsignaled delay of reinforcement. Psychological Record, 1978, 28, 615-625. Richards, R-. W., & Marcattilio, A. J. Stimulus control and delayed reinforcement. Learning and Motivation, 1978, 9, 54-68. Sizemore, 0. J., & Lattal, K. A. Dependency, temporal contiguity, and response-independent reinforceInent. Journal of the Experimental Analysis of Behavior, 1977, 27, 119-125. Sizemore, 0. J., & Lattal, K. A. Unsignaled delay of reinforcement in variable-interval schedules. Journal of the Experimental Analysis of Behavior, 1978, 30, 169-175. Spence, K. W. The role of secondary reinforcement in delayed reward learning. Psychological Review, 1947, 54, 1-8. Tarpy, R. M., & Koster, E. D. Stimulus facilitation of delayed-reward learning in the rat. Journal of Comparative and Physiological Psychology, 1970, 71, 147151. Wilkie, D. M. Signalled reinforcement in multiple and concurrent schedules. Journal of the Experimental Analysis of Behavior, 1973, 20, 29-36. Williamiis, B. A. The effects of unsignalled delayed reinforceinent. Journal of the Experimental Analysis of Behavior, 1976, 26, 441-449. Williams, B. A., & Fantino, E. Effects on choice of reinforcement delay and conditioned reinforcement. Joturnal of the Experimental Analysis of Behavior, 1978, 29, 77-86.
Received Decembei 8, 1980 Final acceptance April 24, 1981
