EUROPEAN JOURNAL OF BEHAVIOR ANALYSIS
2006, 7, 153 - 157
NUMBER 2 (WINTER 2006)
153
Discriminative Properties of Reinforcement: Basic Findings and Applied Implications Einar T. Ingvarsson
Youngstown State University
SungWoo Kahng
Kennedy Krieger Institute and Johns Hopkins University School of Medicine
Reinforcers may acquire discriminative properties as a result of a history of contingent reinforcement. Such properties may result in response reinstatement or response maintenance when reinforcers are delivered on NCR or DRO schedules. However, these effects are typically small and temporary, and response-elimination procedures that involve the presentation of reinforcement may lead to more permanent response elimination and suppression compared with extinction. There is some evidence that noncontingent reinforcement may aid in the maintenance and generalization of responses that have previously been contingently reinforced. It is also possible that noncontingent presentation of reinforcement prior to or at the beginning of treatment sessions may have discriminative (in addition to motivating) effects. Finally, stimulus control may be transferred from arbitrary cues to reinforcement delivery when the former are faded or removed.
Operant researchers have long recognized the discriminative properties of reinforcement. Reid (1957) and Jenkins (1965) both discussed the potential for reinforcers to acquire discriminative properties as a result of a history of a contingent reinforcement. In the experimental analysis of behavior, reinforcement contingency refers to the conditions under which a dependency exists between a response and a reinforcer such that the former can be said to produce the latter (Catania, 1998). By definition, the probability of responding increases as a result of the reinforcement contingency. One test for the presence of a reinforcement contingency is the presentation of the previously identified reinforcer on a responseindependent (i.e., noncontingent) schedule. A decrease in responding as a result of such an arrangement supports the operant reinforcement hypothesis (Catania & Keller, 1981; Thompson & Iwata, 2005). However, although responding
may decrease, it is not always eliminated under response-independent schedules, and may indeed persist for extended periods (Lattal, 1991). Because a response-reinforcer dependency is not in effect under these conditions, additional explanations for behavioral persistence must be sought. One possible explanation is in terms of discriminative properties of reinforcement created by a previous operant reinforcement contingency. The current paper selectively reviews research pertaining to this hypothesis, and discusses some potential applied implications of discriminative functions of reinforcement. Basic Research Findings Persistence of behavior under response-independent (i.e., noncontingent) schedules of reinforcement has been well documented in the literature. In such research, a particular reinforcer is first used to establish and maintain responding, and then that same reinforcer is delivered on a time-based schedule, independent of participant responding. Rescorla and Skucy (1969)
Address correspondence to Einar T. Ingvarsson, Department of Psychology, Youngstown State University, DeBartolo Hall, One University Plaza, Youngstown, OH 44555, USA. Email:
[email protected] 153
154
Einar T. Ingvarsson and SungWoo Kahng
established bar pressing in rats on a VI schedule, and then compared response reduction under extinction, response-independent reinforcement (yoked to a VI schedule being implemented with a control group), and response-independent reinforcement with limited hold (i.e., reinforcement was not delivered unless at least 5 s had passed from the last response). The results showed the greatest reduction with the extinction group, while responding was maintained at a higher rate with rats who received food pellets independent of responding, with or without the limited hold contingency. Additional evidence was provided by a study by Franks and Lattal (1976), who studied magnitude of response reinstatement in rats during response- independent food delivery as a function of a history of reinforcement. (Reinstatement occurs when a response is initially established via contingent reinforcement, and responding then reoccurs as a result of noncontingent reinforcement delivery following extinction; Lieving & Lattal, 2003.) Initially, high or low rates of bar pressing were established using VR20 and DRL schedules, respectively. Following extinction, food was delivered on a 30-s FT (fixed-time) schedule. Rats that had a history of high response rates showed greater magnitude of response reinstatement during the FT condition, suggesting the influence of antecedent history of contingent reinforcement. It should be noted that while these studies showed some response maintenance under response-independent schedules, these rates were nevertheless much lower than those observed with contingent reinforcement. One possible explanation for response maintenance under response-independent schedules is accidental reinforcement of the target response. Several studies have controlled systematically for this possibility by implementing a DRO schedule (i.e., an omission contingency) in which reinforcers are delivered only following a period during which no responding occurs. Uhl and Garcia (1969) found that although both extinction and a 30-s DRO contingency were effective in reducing lever-pressing in rats, responding was maintained at a higher rate in the DRO condition compared to extinction. This finding supports the discriminative interpretation because the possibility of
accidental reinforcement is eliminated with the DRO contingency. However, if discriminative properties of food delivery are responsible for higher level of responding under DRO schedules, it would be expected that the effect would gradually disappear because food delivery does not signal the availability of contingent reinforcement under these conditions. Indeed, this was shown by Uhl (1973), who found that prolonged experience with DRO (30 min sessions over 7 days) eventually led to equal response elimination compared with extinction. As in the previous study, considerably more responding occurred overall in the DRO condition than in the extinction condition, again suggesting the discriminative properties of food delivery. Spradlin and colleagues (Spradlin, Fixsen, & Girardeau, 1969; Spradlin, Girardeau, & Hom, 1966) found similar response patterns in children and adolescents with severe mental retardation. After establishing a simple response (plunger pulling) on an FR schedule, the participants’ responding was extinguished. When the participants met a DRO criterion, a reinforcer was delivered. The results showed that responding was reliably reinstated following reinforcement delivery for all the children, whereas little responding occurred following control periods in which the participants met the DRO criterion but no reinforcers where delivered. For most of the children, the reinstatement effect decreased and eventually disappeared over time with continued non-reinforcement of the target response. Although the above results are consistent with the hypothesis that non-contingent reinforcement delivery may temporarily reinstate or maintain previously reinforced responding, other processes may be partially responsible. For example, partially extinguished responding might by “disinhibited” through stimulus change or presentation of novel stimuli. However, research has shown that much less responding is occasioned by DRO-contingent presentation of novel stimuli than the presentation of reinforcement on a DRO schedule (Campbell, Phillips, Fixsen, & Crumbaugh, 1968; Spradlin et al., 1969). Therefore, stimulus change per se cannot explain these results. In addition, it is possible that species-specific behavior, potentially respondent in nature, may
Discriminative Properties
account for some of the food-seeking behavior under response-independent schedules seen in rats (Timberlake & Lucas, 1985). However, speciesspecific behavior does not explain the reduction in response rates over time seen in some studies (e.g., Uhl, 1973), which seems more plausibly explained via the deterioration of discriminative properties via continued presentation of reinforcement not contingent on behavior. It is also unclear whether species-specific behavior can explain similar results with humans. Discriminative properties of reinforcement have also been suggested by studies demonstrating stimulus control by specific food pellets correlated with different reinforcement schedules (Cruse, Vitulli, & Dertke, 1966), and by studies demonstrating the discriminative properties of reinforcement schedules in a choice paradigm (Lattal, 1975, 1979). Thus, although other behavioral processes may play a role in reinstatement and response maintenance under non-contingent schedules, the evidence is strong that discriminative functions play a role. Applied Implications The above findings have potential implications for the choice of response-elimination procedures in applied situations. It is possible that any procedures that include the presentation of a reinforcer previously shown to maintain problem behavior (e.g., DRO, NCR, and DRA procedures) may set the occasion for the continued occurrence of problem behavior. (It is unclear whether similar results would occur with a functionally irrelevant reinforcer). If this is the case, extinction may be a preferred method. However, research has suggested that maintenance or reinstatement of problem behavior under these conditions are likely to be relatively small and transitory (Uhl, 1973; Uhl & Garcia, 1969). Further, follow-up measures by Uhl (1973) suggested that DRO resulted in more permanent response elimination or suppression than extinction, shown by the fact that introduction of an NCR schedule resulted in more responding following extinction than following DRO. It thus appears that the DRO schedule resulted in more complete response elimination than extinction. These findings may
155
be understood by presuming that what is reinforced in these situations is a chain of responses, which may for example consist of orienting towards the response manipulanda, moving towards the manipulanda, performing the target response, moving towards the reinforcer, and obtaining and consuming the reinforcer. With DRO and NCR, the occasion is set for the performance of earlier parts of the chain, which may then be allowed to undergo extinction. With extinction, parts of the chain may never or rarely be evoked, and hence may not be extinguished (Spradlin et al., 1969). Response-elimination which includes presentation of reinforcement (e.g., NCR, DRO, and DRA) may thus result in more permanent and complete response elimination. Noncontingent reinforcement may serve to increase the likelihood of the generalization or maintenance of previously reinforced responding in settings in which little or no contingent reinforcement is available. In a study with 7-12 year old children with autism, Koegel and Rincover (1977) established novel responses (e.g., motor imitation, instruction following) via contingent reinforcement. While continuing measurement in an alternative setting and with a different teacher, responding was put on extinction, but noncontingent reinforcers were delivered periodically. The results showed that (a) following extinction, NCR reliably reinstated the response, and (b) when a new response was trained and then put on extinction, more responding was emitted when NCR was periodically delivered than when no NCR was delivered. These findings replicate those of previous basic research (e.g., Spradlin et al, 1969, Uhl & Garcia, 1969) and also suggest that NCR may be a useful procedure in facilitating transfer from a treatment setting in which the target response is reinforced on a dense schedule to a generalization setting in which contingent reinforcement schedules may be less reliable. Because noncontingent reinforcement sets the occasion for previously reinforced responding, it may increase the likelihood that responding comes into contact with reinforcement in the alternative setting. NCR also entails introducing a salient stimulus (the reinforcer) from the training setting into the generalization setting, thereby making the two less discriminable and
156
Einar T. Ingvarsson and SungWoo Kahng
increasing the likelihood of generalization. This practice is consistent with the recommendation by Stokes and Baer (1977) to program common stimuli into different environments to promote generalization. Discriminative properties of reinforcement may also need to be considered in situations in which reinforcers are noncontingently presented just before or at the beginning of treatment sessions. First, reinforcer sampling (Cooper, Heron, & Heward, 1987, p. 266) that occurs at the beginning of sessions may set the occasion for behavior previously contingently reinforced using the same reinforcer. This may be viewed as a positive side effect in educational situations in which the goal is to increase the occurrence of behavior, but could be a confound in some experimental situations. Second, noncontingent presentation of reinforcement (e.g., attention) prior to sessions may serve a discriminative function. Several studies have shown that presession attention may reduce the rate of attention-maintained behavior during subsequent sessions in which behavior is contingently reinforced with attention (e.g., McComas, Thompson, & Johnson, 2003). This suggests that presession attention may serve as an abolishing operation (AO), reducing the value of attention as a reinforcer. However, Roantree and Kennedy (2006) recently demonstrated the apparent response-facilitating effect of presession attention for the attention-maintained stereotypy of a 10-year-old boy with severe mental retardation. These authors interpreted this finding as demonstrating that presession attention may function as an establishing operation, increasing the value of attention as a reinforcer. However, it is also possible that the delivery of noncontingent presession attention came to function as a discriminative stimulus, predicting the availability of contingent reinforcement in the subsequent sessions, hence increasing the probability that the participant would come into contact with contingent reinforcement early and often in the session. More research is needed to determine under what conditions noncontingent presession presentation of reinforcement may be beneficial. Finally, discriminative properties of reinforcement may play a role in applied situations involving discriminated responding on multiple and
mixed schedules. Tiger and Hanley (2005) conducted an experiment with two preschool children who initially manded for attention regardless of whether attention was available as a reinforcer. As a result of multiple-schedule discrimination training (FR1 vs. EXT) - in which manding for attention was differentially reinforced in the presence of arbitrary schedule-correlated stimuli (colored leis) - responding came under schedule control, occurring mostly when attention was available. Finally, when the arbitrary stimuli where removed, stimulus control over manding had transferred to reinforcement delivery, as evidenced by continued discriminated responding under a mixed schedule. A within-session analysis of the data suggested that during the extinction component in the mixed schedule the children would respond infrequently, and each non-reinforced response would typically result in a long pause before the next response. Each reinforced response during the FR1 component was however followed by a flurry of responding. It thus appears that the children learned that delivery of reinforcement signaled the availability of more reinforcement; attention thus came to serve a discriminative function. This finding could have implications for applied situations in which stimulus control is established in multiple schedules associated with arbitrary schedule-correlated stimuli. It is possible that the arbitrary stimuli could be faded, resulting in the transfer of stimulus control to reinforcement delivery. If responding comes under such control, generalization and maintenance may become more likely. Future research should further evaluate under what conditions such transfer occurs. References Campbell, P. A., Phillips, E., Fixsen, D., & Crumbough, C. (1968). Free-operant response reinstatement during extinction and time-contingent (DRO) reward. Psychological Reports, 22, 563-569. Cooper, J. O., Heron, T. E., & Heward, W. L. (1987). Applied Behavior Analysis. Columbus, OH: Merrill / Prentice Hall. Cruse, D. B., Vitulli, W., & Dertke, M. (1966). Discriminative and reinforcing properties of
Discriminative Properties
two types of food pellets. Journal of the Experimental Analysis of Behavior, 9, 293-303. Franks, G. J., & Lattal, K. A. (1976). Antecedent reinforcement schedule training and operant response reinstatement in rats. Animal Learning and Behavior, 4, 374-378. Jenkins, H. M. (1965). Measurement of stimulus control during discriminative operant conditioning. Psychological Bulletin, 64, 365-376. Koegel, R. L., & Rincover, A. (1977). Research on the difference between generalization and maintenance in extra-therapy responding. Journal of Applied Behavior Analysis, 10, 112. Lattal, K. A. (1975). Reinforcement contingencies as discriminative stimuli. Journal of the Experimental Analysis of Behavior, 23, 241-246. Lattal, K. A. (1979). Reinforcement contingencies as discriminative stimuli: II. Effects of changes in stimulus probability. Journal of the Experimental Analysis of Behavior, 31, 15-22. Lattal, K. A. (1991). Scheduling positive reinforcers. In I. H. Iversen & K. A. Lattal (Eds.), Experimental analysis of behavior, part 1 (pp. 87-134). New York: Elsevier. Lieving, G. A., & Lattal, K. A. (2003). Recency, repeatability, and reinforcer entrenchment: An experimental analysis of resurgence. Journal of the Experimental Analysis of Behavior, 80, 217-233. McComas, J. J., Thompson, A., & Johnson, L. (2003). The effects of presession attention on problem behavior maintained by different reinforcers. Journal of Applied Behavior Analysis, 36, 297-307. Reid, R. L. (1957). The role of the reinforcer as a stimulus. British Journal of Psychology, 49, 292-309. Rescorla, R. A., & Skucy, J. C. (1969). Effect of response-independent reinforcers during extinction. Journal of Comparative and Physiological Psychology, 67, 381-389.
157
Roantree, C. F., & Kennedy, C. H. (2006). A paradoxical effect of presession attention on stereotypy: Antecedent attention as an establishing, not an abolishing, operation. Journal of Applied Behavior Analysis, 39, 381-384. Spradlin, J. E., Fixsen, D. L., & Girardeau, F. L. (1969). Reinstatement of an operant response by the delivery of reinforcement during extinction. Journal of Experimental Child Psychology, 7, 96-100. Spradlin, J. E., Girardeau, F. L., & Hom, G. L. (1966). Stimulus properties of reinforcement during extinction of a free operant response. Journal of Experimental Child Psychology, 4, 369-380. Stokes, T. F., & Baer, D. M. (1977). An implicit technology of generalization. Journal of Applied Behavior Analysis, 10, 349-367. Thompson, R. H., & Iwata, B. A. (2005). A review of reinforcement control procedures. Journal of Applied Behavior Analysis, 38, 257278. Tiger, J. H., & Hanley, G. P. (2005). An example of discovery research involving the transfer of stimulus control. Journal of Applied Behavior Analysis, 38, 499-509. Timberlake, W., & Lucas, G. A. (1985). The basis of superstitious behavior: chance contingency, stimulus substitution, of appetitive behavior? Journal of the Experimental Analysis of Behavior, 44, 279-299. Uhl, C. N. (1973). Eliminating behavior with omission and extinction after varying amounts of training. Animal Learning and Behavior, 1, 237-240. Uhl, C. N., & Garcia, E. E. (1969). Comparison of omission with extinction in response elimination with rats. Journal of Comparative and Physiological Psychology, 69, 554-562.
