after the pigeons. Experiment 1 was carried out under approval IRB #13-081 granted by the Florida Institute of. Technology. Experiment 2 was carried out under ...
2017, n/a–n/a
JOURNAL OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR
NONCONTINGENT REINFORCEMENT COMPETES WITH RESPONSE PERFORMANCE MICHAEL E. KELLEY1, CY B. NADLER2, CATALINA REY1, SARAH COWIE3, 1,3 AND CHRISTOPHER A. PODLESNIK 1
THE SCOTT CENTER FOR AUTISM TREATMENT AND FLORIDA INSTITUTE OF TECHNOLOGY
2
DIVISION OF DEVELOPMENTAL AND BEHAVIORAL SCIENCES, CHILDREN’S MERCY KANSAS CITY 3
THE UNIVERSITY OF AUCKLAND
Noncontingent reinforcement is a commonly used procedure to decrease levels of problem behavior. Goals of this intervention are to decrease motivation, responding, and the functional relation between behavior and consequences, but it could also possibly compete with performance of alternative desirable responses. In the current study, we assessed the effects of noncontingent reinforcement arranged from 0% to 100% of sessions on performance of alternative responding across two experiments. Experiment 1 assessed manding (i.e., requests) maintained by attention and tangibles with a child with developmental disabilities and Experiment 2 assessed keypecking maintained by food with six pigeons. We extended previous research by (a) showing that noncontingent reinforcement competes with both the acquisition and maintenance (performance) of an alternative response, (b) extending the generality of the findings across nonhuman and human participants, and (c) eliminating influence of sequence effects through random manipulations of noncontingent value in pigeons. Overall, greater amounts of noncontingent reinforcement competed with both acquisition and maintenance of alternative responding. Key words: noncontingent reinforcement, pigeon, translational research, differential reinforcement of alternative behavior
Noncontingent Reinforcement (NCR) is the delivery of a reinforcing event contingent on the passage of time rather than contingent on a particular response (Rescorla & Scucy, 1969; Vollmer & Iwata, 1991; Vollmer, Iwata, Zarcone, Smith, & Mazaleski, 1993). In application, NCR is often used as a control in functional analyses of problem behavior (control or toy play condition; see Beavers, Iwata, & Lerman, 2013, and Hanley, Iwata, & McCord, 2003) and as a treatment for problem behavior (e.g., Vollmer et al.). Its use as a behavioral treatment stems from basic research demonstrating response-independent presentations The authors thank the members of the Experimental Analysis of Behaviour Research Group for their help in running these experiments and Mike Owens for looking after the pigeons. Experiment 1 was carried out under approval IRB #13-081 granted by the Florida Institute of Technology. Experiment 2 was carried out under approval AEC/2011/RT909 granted by the Animal Ethics Committee of The University of Auckland. Portions of the present data set were presented at the 40th annual meeting of the Association for Behavior Analysis International (ABAI). Send correspondence to: Michael E. Kelley, The Scott Center for Autism Treatment and Florida Institute of Technology, 150 W. University Blvd., Melbourne, FL 32901; mkelley@fit.edu or Christopher A. Podlesnik, cpodlesnik@fit.edu. doi: 10.1002/jeab.255
of reinforcing events (e.g., food) decrease rates of a target response (e.g., Kuroda et al., 2013; Podlesnik & Shahan, 2008). NCR has been shown to be effective for treating problem behavior in a number of applied studies. In addition, many questions have been evaluated over the course of several decades of research about the mechanism responsible for the behavioral effects (e.g., Fisher et al., 1999; Kahng, Iwata, Thompson, & Hanley, 2000; Lalli, Casey, & Kates, 1997; Wallace, Iwata, Hanley, Thompson, & Roscoe, 2012), the effects of parameter manipulations (e.g., Carr, Bailey, Ecott, Lucker, & Weil, 1998; Lindberg, Iwata, Roscoe, Worsdell, & Hanley, 2003), and generality across function and topography (e.g., Fischer, Iwata, & Mazaleski, 1997; Reed et al., 2004; Roscoe, Iwata, & Zhou, 2013; Vollmer et al., 1993). Results of these studies suggest that NCR produces behavior-reducing effects through three mechanisms: (1) altering motivation via a motivating operation (MO; Laraway, Snycerski, Michael, & Poling, 2003), (2) weakening the relation between a response and its consequences via extinction (see Lalli, Casey, & Kates, 1997), and/or (3) arranging a choice situation in which
© 2017 Society for the Experimental Analysis of Behavior 1
2
MICHAEL E. KELLEY et al.
subjects allocated responding in accordance with operating contingences (e.g., Fisher et al. 1999). The latter interpretation is consistent with interpretations based on the matching law. Specifically, NCR reduces target responding by reinforcing a range of unspecified responses other than the target response (e.g., Burgess & Wearden, 1986; Herrnstein, 1970; McDowell, 1988). Despite the efficacy of NCR for reducing levels of problem behavior, some features of this intervention may lessen its appeal in the context of a broader treatment strategy. First, intervention programs often include goals for both target behavior reduction and acquisition or ongoing performance of alternative behavior. NCR does not arrange for the acquisition of an alternative response to produce access to reinforcement. Second, the behavioral outcomes of NCR almost certainly extend beyond the target functional relation. That is, the primary effects of noncontingent access to a functional reinforcer include reduction of (a) the motivation to access the reinforcer, and (b) levels of target responding relevant to that reinforcer. The broader effects of NCR might also include a reduction in the efficacy of that reinforcer in the context of concurrent reinforcement contingencies (Fisher et al., 1999; Goh, Iwata, & DeLeon, 2000; Marcus & Vollmer, 1996). Specifically, implementing NCR should reduce the probability of all responses in the response class (not only the target response), and should interfere with the development of new topographies in that class. In basic studies of choice, availability of alternative sources of reinforcement similarly decreases performance of all functionally equivalent responses (e.g., Bensemann, Lobb, Podlesnik, & Elliffe, 2015; Boelens, Kop, Nagel, & Slangen, 1987; Davison & McCarthy, 1994). The most relevant point for clinicians is that NCR may have unintended side effects beyond addressing the target response in the context of the identified functional relation. Two applied studies have explored the specific hypothesis that NCR interferes with acquisition of alternative behavior. Marcus and Vollmer (1996) suggested that the satiation effect of a MO might abate problem behavior during NCR as a primary effect. They further argued that using that same reinforcer in the
context of any differential-reinforcement procedure might compromise the efficacy of the acquisition of a new topography in the same response class. One participant was exposed to a concurrent FT 20-s (rapidly thinned to FT 3min) FR-1 schedule. The participant nevertheless acquired mands but only upon thinning the FT schedule. They observed similar findings when exposing a second participant to a concurrent FT 1-min FR-1 schedule, and then thinning the FT 1-min schedule to FT 5 min over the course of the phase. As the FT schedule was thinned, mand levels increased. Overall, the data suggested that the NCR interfered very little with acquisition of mands. Goh et al. (2000) tested whether NCR would compete with acquisition of an alternative response in the context of schedules that were more likely to interfere with acquisition—they arranged schedules that were not thinned first or rapidly thinned during differential reinforcement. Functional analysis results suggested that problem behavior was maintained by positive reinforcement in the form of access to tangibles and attention for two participants. NCR plus differential reinforcement of alternative behavior (DRA) reduced problem behavior for both participants. However, neither participant acquired the alternative response during this initial phase, despite ongoing acquisition training. Once the schedules of NCR were sufficiently thinned, albeit at a slower pace than in Marcus and Vollmer (1996), both participants acquired the alternative responses. Results of Goh et al. and Marcus and Vollmer suggest that NCR can interfere with acquisition of alternative behavior, but that the thinning schedule may mitigate interference. Thus, practically speaking, DRA would be best applied subsequent to thinning the NCR schedule. Goh et al. (2000) demonstrated the dynamic interaction between NCR and responding and, specifically, that NCR can have a more generalized effect on behavior (beyond the target behavior in a specific functional relation). In this case, NCR both suppressed problem behavior and competed with acquisition of an alternative response. One limitation of the Goh et al. study is that the schedule for NCR was not subsequently enriched after thinning. That is, Goh et al. thinned the NCR schedule to assess the effects of NCR on acquisition, but did not enrich the schedule to assess the effects
NONCONTINGENT REINFORCEMENT on response performance subsequent to acquisition. Thus, the extent to which NCR might compete with response acquisition, maintenance, and performance during contingencies reversals remained unanswered. The purposes of the current studies were to (a) replicate and extend the Goh et al. study, (b) specifically evaluate the extent to which NCR interferes with performance, and (c) evaluate the potential influence of NCR fading per se on alternative behavior during NCR. First, we exposed one human participant to NCR plus DRA with both NCR schedule thinning and enriching across two contingencies (mands for access to attention and tangibles, respectively). This arrangement assessed the potential dynamic relation between NCR and responding, both within acquisition training (Goh et al.; Marcus & Vollmer, 1996) and ultimate performance of clinically relevant behavior during schedule thinning and enriching. Second, we further explored the generality of this dynamic relation in an animal model by exposing six pigeons to a similar procedure (systematically thinning and enriching NCR values). Finally, we evaluated the effects of random exposures to the different NCR values on acquisition and maintenance of alternative responding in the pigeons to control for sequential exposure to the different NCR values. Experiment 1 Method Participant, setting, and materials. Calvin (a pseudonym), a 6-year-old boy diagnosed with global developmental delays secondary to Spina Bifida, Intermittent Explosive Disorder, and Attention-Deficit Disorder with Hyperactivity (ADHD), participated in this study. He was initially referred for the assessment and treatment of aggressive and disruptive behavior (i.e., hitting, scratching, pushing, pulling, grabbing, stepping on feet, and ripping materials). Calvin was exposed to a functional analysis of problem behavior prior to participating in this study. However, the treatment of his problem behavior was unrelated to his participation in this study, which focused on (1) inserting new mands into Calvin’s repertoire and (2) assessing the extent to which NCR might interfere with initial response acquisition and performance. Thus, we did not program contingencies for problem
3
behavior, including a changeover delay for any behaviors. Subsequent to assessment, Calvin was exposed to two manding protocols that were designed to evaluate the extent to which NCR interfered with acquisition of an alternative response. Therapists conducted sessions in a 4 m x 4 m individual treatment room that contained session-relevant stimuli (e.g., table, task materials). Only the therapist and the participant were present in the room during sessions. Materials included session-specific stimuli that were necessary to implement the independent variable manipulations (e.g., session-relevant toys). Response measurement and interobserver agreement. Prompted (occurring subsequent to a therapist prompt) and unprompted (independent) mands were defined as intelligible requests for the relevant reinforcer (i.e., “play please” for attention or “toy please” for a preferred tangible item). A primary data collector collected data on all sessions. A second data collector simultaneously and independently collected IOA data during 36.3% of sessions. IOA data were calculated by dividing the number of agreements by the number of agreements and disagreements in each 10-s bin of each session. The quotients from each 10-s bin were added together, divided by the number of 10-s bins, and multiplied by 100 to yield a percentage of agreement. IOA averaged 98.4% (range, 83.3% to 100%) for manding during the intervention. General procedures and experimental designs. Calvin was exposed to two distinct phases. First, therapists exposed Calvin to a paired-choice preference assessment (Fisher et al., 1992) to determine preferred stimuli that could be used in subsequent assessments. Next, therapists implemented procedurally identical training sequences for teaching Calvin mands to request attention (chosen as an important skill to learn by Calvin’s mother) and preferred stimuli (chosen from the preference assessment). The training sequence consisted of NCR, schedule thinning, and differential reinforcement. The two training sequences consisted of combined multiple-baseline and reversal designs. The effects of NCR were evaluated by using two alternating attention or tangible conditions, with systematic decreasing and increasing amounts of noncontingent access to reinforcement within each condition (see
4
MICHAEL E. KELLEY et al.
below). Thus, attention and tangible sessions were alternated, and thinning (A phase) and enriching (B phase) of the schedules produced a reversal design. Preference assessment. Therapists exposed Calvin to a paired-choice preference assessment consistent with the procedures described by Fisher et al. (1992). The therapist provided brief access to each stimulus prior to initiating the formal assessment. Next, the therapist exposed Calvin to a choice between two stimuli while saying, “pick one.” Contingent on a choice, the therapist provided 30-s access to leisure items or one piece of an edible item. Contingent on attempts to choose two items, the therapist removed both items, and represented the trial. Contingent on no selection, the therapist removed both items, provided access to each item one at a time, and represented the trial. All stimuli were paired with each other once. The therapist then graphed item selection as percentage of trials selected. NCR plus DRA. We developed procedures based on those described by Goh et al. (2000) to test the effects of the systematic fading of noncontingent reinforcer access on the acquisition and maintenance of functional communication responses. Therapists conducted 5min sessions in the session room described above, alternating between attention and tangible conditions. That is, the therapists implemented training sequences for both the attention and tangible functions in a multipleschedule design; however, for ease of visual inspection, the conditions are graphed separately. Sessions were conducted subsequent to an approximate 2-min condition in which the relevant reinforcers were made available, and then withdrawn prior to the initiation of the session. All contingencies were arranged like those described in Goh et al., 2000. First, therapists conducted baseline phases of 100% NCR (attention was delivered on a noncontingent schedule in the attention condition, and tangibles were delivered on a noncontingent schedule in the tangible condition, respectively, during the NCR portion of the sessions), with functional communication contingencies in place. That is, if Calvin emitted the phrase “play please” in the attention mand condition, the therapist provided 20-s contingent access to attention. If Calvin emitted the phrase “toy
Table 1 NCR procedures from Experiment 1
Schedule Step
# of seconds of NCR per minute of 5-min session
Total # of seconds of NCR per session
% of session with NCR
1 (Rich) 2 3 4 5 6 7 8 9 10 11 12 13 (Thin)
60 of each 60 seconds 55 of each 60 seconds 50 of each 60 seconds 45 of each 60 seconds 40 of each 60 seconds 35 of each 60 seconds 30 of each 60 seconds 25 of each 60 seconds 20 of each 60 seconds 20 of each 90 seconds 20 of each 120 seconds 20 of each 180 seconds 20 of each 300 seconds
300 275 250 225 200 175 150 125 100 80 60 40 20
100 91.67 83.33 75 66.67 58.33 50 41.67 33.33 26.67 20 13.33 6.67
Note. Column 1 indicates the schedule steps. Column 2 depicts the number of seconds of NCR per min of each session. Column 3 indicates the total number of seconds per session with NCR. Column 4 shows the percentage of each session with NCR.
please” in the tangible mand condition, the therapist provided 20-s contingent access to a preferred tangible item. Unprompted mands for irrelevant stimuli (e.g., “pizza please,” or “toy please” during an attention session) produced no programmed consequences. Once rates of functional communication were stable, therapists initiated schedule thinning (see Table 1). Specifically, the therapists initially reduced NCR from 100% to 91.7% of the 5-min sessions in each condition (i.e., NCR for the first 55 s of each min of the session). If the rate of combined problem behaviors (data not shown but available upon request) for the session was less than or equal to 0.5 rpm, the therapist further reduced the amount of NCR to a cumulative duration of 83.3% of the 5-min sessions (i.e., the first 50 s of each min). The therapist systematically thinned NCR durations by 5 s per min within each session until the duration reached 20 s per min (i.e., 33.3% of the session with NCR). From then on, cumulative NCR duration was further thinned across sessions by the delivery of 20-s reinforcement at the beginning of each session and extending the delay until onset of a new 20-s NCR interval from 1 min to 1.5, 2.0, 3.0, 4.0 and 5.0 min (resulting in cumulative NCR durations of 26.7%, 20.0%, 13.3%, and 6.7% of each session, respectively). Once cumulative NCR
NONCONTINGENT REINFORCEMENT
5
Fig. 1. Alternative Responses per Min are shown on the y axis and Percent of Session without NCR on the x axis. Each data path depicts response rates at each NCR level during either schedule thinning (filled data points) or schedule enriching (open data points).
reached 6.7% of the session, the therapist then systematically increased NCR across sessions back to 100% using the same sequence (in reverse) with the same criterion for advancement (i.e., ≤ 0.5 rpm for problem behaviors during the session). If rates of problem behavior exceeded 0.5 rpm in a session, the therapist repeated the session under the same NCR arrangement. This sequence was discontinued within each condition when two full cycles of the full reinforcement thinning/ enriching schedule had been completed. Results and Discussion Figure 1 shows results of the NCR plus DRA analysis. The x-axis shows percentage of time per session without NCR of either attention (left panel) or tangibles (right panel). As the numbers increase to the right on the x-axis, the duration of NCR is progressively lower, which is indicative of relatively thinner schedules. Response rate (responses per min) during each session’s exposure to a particular duration of NCR is depicted on the y-axis. Each of the data paths reflects either scheduling thinning or schedule enriching; thinning 1 refers to the initial schedule thinning, enriching 1 refers to the subsequent enriching of the NCR schedule, thinning 2 and enriching 2 refer to the subsequent thinning and enriching. Response rates covaried with the duration of NCR. That is, as the noncontingent duration of access to both attention and tangibles increased, response rates decreased. As the duration of noncontingent access to both attention and tangibles decreased, response rates increased. These results indicated that initial acquisition and subsequent performance of the alternative response, reflected in
response rates, covaried with the amount of NCR. Clinically, these results should help caregivers make decisions about the amount of noncontingent access to provide, depending on the desired response rates for alternative behavior. Experiment 2 Experiment 2 included a nonhuman model of the procedures described in Experiment 1. In the first assessment, pigeons were exposed to NCR schedule thinning and enriching similar to Experiment 1. We attempted to preserve similar independent variables across species, but used session procedures specific to and appropriate for each species. In the second assessment, pigeons were exposed to the NCR schedule in a randomized sequence. Like Mace et al. (2010), we used an animal analog of a human condition in an effort to elucidate a functional relation without unnecessarily exposing humans to unusual conditions (exposure to randomly implemented NCR schedules). The purposes of the model assessments were two-fold. First, we wished to evaluate an interspecies replication of the procedures to extend the generality of Experiment 1. Secondly, we wished to determine whether the results in Experiment 1 were due to the schedule fade procedures or the actual NCR schedules. The inclusion of pigeons provides the ideal opportunity to evaluate such a hypothesis. Method Subjects. Six homing pigeons (numbered 111 to 116) served as subjects. Experimenters provided the pigeons with (a) water and grit
6
MICHAEL E. KELLEY et al.
at all times and (b) mixed grain (when necessary) to maintain their designated body weights (85% 15 g of their free-feeding body weight). Apparatus. The experimental chambers (which also served to house the pigeons) measured 375 mm high by 375 mm deep by 370 mm wide. The chambers included two wooden perches mounted 50 mm above the cage floor and 95 mm from, and parallel to, the right wall and the door. Three 20-mm diameter plastic keys set 100 mm apart (center-to-center) were located on one wall of the cage. The computer recorded responses to illuminated (yellow or red) keys that exceeded approximately 0.1 N. The chamber included a magazine aperture (measuring 40 mm x 40 mm) beneath the center key (60 mm from the perch). When a response produced a reinforcer, key lights were extinguished, the aperture was illuminated, and the hopper containing wheat was raised for 2 s. A program on an IBM PC-compatible computer (located in an adjacent room) running Med PC® software arranged and recorded all experimental events. Procedure. The pigeons participated in other research protocols (Podlesnik & Kelley, 2014), thus no pretraining was required. Room lights were lit from 12 midnight until 4 pm. Sessions for all six pigeons began at 1:00 am. Pigeons participated in 2-min sessions 7 days per week. The program arranged for responding on the left key to contact reinforcement on a VI 10-s schedule over 10 sessions. The purpose of this baseline condition was to simulate a target response (such as aggression) in which a human might engage. Next, the pigeons’ keypecking contacted extinction over the subsequent two sessions. During both Baseline and Extinction conditions, the right key was inactive and darkened. Following the Extinction condition, the program arranged differential reinforcement in the form of DRA, with no changeover delay. In DRA conditions, both the left and right keys were illuminated, and a concurrent EXT VI 10-s schedule operated across the keys. As in Experiment 1, the program arranged for noncontingent reinforcement delivery (i.e., the magazine light was illuminated and the hopper was raised for 9.5 s independent of responding). To ensure that the hopper
Table 2 NCR procedures from Experiment 2 Systematic
Random
% NCR
Descending Conditions
Ascending Conditions
% NCR
Condition
100 83 67 50 33 17 0
1 2 3, 13 4, 14 5, 15 6, 16 7, 17
23 12, 22 11, 21 10, 20 9, 19 8, 18
17 67 0 83 50 17 100
24 25 26 27 28 29 30
Note. Column 1 indicates the percentage of each session with NCR. Columns 2 and 3 depict the descending and ascending order in which pigeons were exposed to each NCR value in the systematic schedule fades. Columns 4 and 5 indicate the percentage of NCR per session and the conditions for the random sequence of NCR delivery.
did not empty during a reinforcer delivery, consecutive noncontingent reinforcement periods were separated by .5 s, during which the hopper was lowered and the magazine light turned off. Thus, a single NCR delivery period lasted for 10 s. Across Systematic DRA conditions, the percentage of time during which NCR was available was systematically increased or decreased by changing the number of 10-s presentations per min between zero and six (see Table 2). Random DRA conditions arranged the same range of 10-s presentations per min as did systematic conditions, but values were changed across conditions in an unsystematic order (Table 2). The number of 10-s presentations per min in each Random condition was chosen randomly without replacement, using Microsoft Excel’s RANDBETWEEN function. Pigeons were exposed to each DRA condition for three sessions. We chose to expose the pigeons to a fixed number of exposures to each NCR value, as opposed to using a behavioral stability criterion, to expedite the parametric assessment of the effect of NCR and to simulate the rapid change in conditions experienced in clinical populations during treatment fading (Davison & Baum, 2000; Hunter & Davison, 1985). Results and Discussion Figure 2 shows results for Experiment 2 for Pigeons 111–116; data reflect mean response rate calculated from responding in all sessions.
NONCONTINGENT REINFORCEMENT
7
Data in the left panels show response per min when the schedule thinning and enriching was implemented consistent with the procedures in Experiment 1 (which are consistent with procedures typically used in applied research). Across all six pigeons, response rates systematically covaried with the percentage of the session in which NCR was delivered. Data in the right panels show response per min when the schedule of NCR was implemented in a random order. That is, the percentage of time within each session without NCR was chosen randomly, and the data were graphed consistently with an orderly thinning and enriching schedule (for ease of visual inspection and to compare with the enriching and thinning functions). Results for all six pigeons were similar in that response rates systematically covaried with the amount of NCR. When the thinning and enriching was implemented either incrementally (left panel) or randomly (right panel), the results were consistent with each other and the results of Experiment 1. Results of Experiment 2 replicated those of Experiment 1, suggesting a robust, interspecies relationship between NCR and response rates. The random NCR schedule exposures produced similar response rates, suggesting that the amount of NCR, and not the schedule fading per se, influenced response rates. General Discussion
Fig. 2. Alternative Responses per Min are shown on the y axis and Percent of Session without NCR on the x axis. Each data path in the left panel depicts response rates at each NCR level during either schedule thinning (filled data points) or schedule enriching (open data points). Each data path in the right panel depicts response rates at each NCR level during both exposures to the random NCR sequences.
Key pecks toward the extinction key rarely occurred and are not depicted on the figure. The x and y axes depict the independent and dependent variables, respectively, in the same manner as in Experiment 1. Results are similar to those from Experiment 1 in that the response rates covaried with the percentage of time within each session that NCR was present.
We demonstrated that NCR competes with both acquisition (Goh et al., 2000) and overall performance (once schedules were thinned and enriched) of alternative responding in both basic and applied arrangements. Results of Experiment 1 were consistent with Goh et al. in that acquisition of alternative behavior occurred when NCR delivery was sufficiently thinned. We extended Goh et al. in two ways. First, we demonstrated that NCR could compete with response maintenance (performance) in addition to response acquisition. Second, we extended Goh et al. by replicating the current study’s procedures with a nonhuman species (i.e., pigeons). The similarity of the results with both a human and the nonhuman animal models suggests that the covariation of response rates in the presence of different levels of NCR is a robust phenomenon. The inclusion of the randomized sequence of NCR is particularly noteworthy
8
MICHAEL E. KELLEY et al.
because (1) it demonstrates that the effects of manipulating NCR duration were independent of sequence effects, (2) obtaining similar data with humans would be relatively more challenging (Mace et al., 2010), and (3) the direct and systematic replications extend the generality of the findings by including an interspecies replication (Sidman, 1960). This study highlights the value of translational research for extending the generality of basic behavior analytic principles. On a superficial level, similar results across species (despite other significant differences, such as environment, topography, stimuli, reinforcers, etc.) suggest that the dynamic interaction of NCR value and responding is a robust phenomenon. On a deeper level, the use of nonhuman animals provided the opportunity to do a more thorough analysis of the effects of parametrically manipulated NCR values on responding. Specifically, in the applied analysis in Experiment 1, we used a common fading procedure for thinning and enriching the NCR schedules (i.e., starting with a rich schedule and slowly fading). In contrast, we exposed the pigeons to both sequential and random sequences while parametrically manipulating the NCR schedules. The similar response patterns across the basic analyses of Experiment 2 and across Experiments 1 and 2 suggest that the fairly linear relation between the levels of NCR schedules and response rates was not a function of sequential exposure to the NCR schedules, but rather a function of the current NCR schedule. Although we did not expose Calvin to the random sequence, the results of the basic analysis suggest that Calvin’s behavior was also a function of the current NCR schedule. Our results were consistent with those of Goh et al. (2000), and somewhat inconsistent with Marcus and Vollmer (1996). Marcus and Vollmer found that NCR did not effectively compete with acquisition during DRA. However, as discussed by Goh et al., Marcus and Vollmer rapidly thinned the NCR schedules. Thus, it is possible that the levels of NCR were sufficient to abate responding yet insufficient to compete with acquisition training during the rapid schedule fade. Overall, the results of Goh et al., Marcus and Vollmer, and the current study suggest that NCR can interfere with acquisition, but the extent to which interference occurs depends on the schedule of NCR,
how rapidly the schedule is thinned, or a combination of these and other variables. Future research could focus on systematically assessing specific amounts of NCR delivery and its effects on motivation and responding (e.g., Kelley, Shillingsburg, & Bowen, 2017). Although the thinning and enriching schedules influenced behavior on average, levels of responding were influenced by the order of exposure to the NCR manipulations in both Experiments 1 and 2. For Experiment 1, initial acquisition was slower for both attention and tangibles and there were more zero points (particularly for attention) during the first thinning exposure. In Experiment 2, initial acquisition was slower for five out of six pigeons during the first thinning exposure (Pigeon 111 was the exception). This pattern is indicative of a potential repeated-exposure effect (Sidman, 1960). It is interesting to note that all three participants in the Goh et al. (2000) study produced similar response patterns during schedule thinning to our participant in the first exposure to thinning. Because Goh et al. did not subsequently enrich, thin, and enrich the schedules, it is not possible to know whether those participants’ data would have been similar to Calvin’s in the following phases. However, the similarity of the response patterns across studies and species suggests a robust and general phenomenon in that initial acquisition during the first exposure of thinning is likely to be relatively slow. The data presented in this study highlight the dynamic nature of motivation and responding (Kelley et al., 2017). Goh et al. (2000) suggested that access to NCR abolished motivation and abated alternative responding for three participants. Likewise, we found that rate of alternative responding covaried with programmed NCR, suggesting that the amount of NCR is an important variable to consider when establishing an intervention. That is, the programmed operation (deprivation or satiation) may evoke or abate responding as a motivating operation (MO; Laraway et al., 2003). The extent to which responding will be evoked or abated—reflected by response rate—is likely to vary as a function of the amount or duration of NCR. Given the clear dynamic interaction between the amount of NCR and responding, the general conception
9
NONCONTINGENT REINFORCEMENT of “MO present” and “MO absent” often described in applied research is probably overly simplistic. Support of this point comes from basic studies of motivational processes in animals. When food reinforcement maintained operant responding during sessions, greater amounts of food made freely available either during or prior to sessions produced greater decreases in operant responding (e.g., Lattal & Bryan, 1976; Nevin, 1992; Pinkston, Ginsburg, & Lamb, 2009; Podlesnik, Jimenez-Gomez, Ward, & Shahan, 2009). Goh et al. (2000) suggested that noncontingent access to reinforcement abolished motivation to access those reinforcers, and thus competed with response acquisition. However, it is possible that alternating periods of reinforcer access and reinforcer deprivation produced a choice arrangement in which participants allocated responding in accordance with the operating schedules. For example, Fisher et al. (1999) found that participants’ behavior covaried with the presence and absence of NCR. Their analyses suggested that participants responded when reinforcement was not available on a noncontingent basis, and responding was suppressed when NCR was delivered. Thus, their data supported a choice interpretation of the effects of NCR—NCR decreases alternative responding by increasing reinforcement rate for responses other than the alternative response (e.g., Burgess & Wearden, 1986; Herrnstein, 1970; McDowell, 1988). Similarly, pigeons engaged in little to no responding when the hopper was available across all conditions (M = 0.74; SEM = 0.23 to 2.12). For Calvin, the data were not collected in a manner in which responding during attention or tangible could be calculated. The data from the pigeons are consistent with those of Fisher et al.’s (1999) choice interpretation that responding primarily was isolated to times NCR was unavailable. Regardless of the specific behavioral mechanism responsible for response suppression across studies with NCR, results of Goh et al. (2000), the current study, and basic research (e.g., Aoyama, 1998; Bizo, Bogdanov, & Killeen, 1998; DeMarse, Killeen, & Baker, 1999; Herrnstein, 1970) suggest that response rates covary systematically with the availability of alternative sources of reinforcement. Future research may evaluate the
specific influence that a particular NCR schedule has on responding. For example, if NCR implementation favors a choice arrangement, the effects of the NCR schedule are likely to be transitory, and target behavior (or any topography in the response class) may experience an increased probability of occurrence when NCR is removed (e.g., Fisher et al., 1999). On the other hand, if NCR implementation favors a MO interpretation, the effects of NCR may be less transient. It is possible that difference in applied and basic research practice obscure potential behavior mechanisms. For example, reinforcement time (e.g., hopper open) is routinely removed from response rate calculations in basic research, but is usually not removed in applied research. Likewise, in the current study, we did not remove reinforcer access time from our calculations, but future researchers should evaluate the effects of doing so. The behavioral mechanism responsible for the effects of NCR may affect the extent to which NCR should be used in applied arrangements. References Aoyama, K. (1998). Within-session response rate in rats decreases as a function of amount eaten. Physiology Behavior, 64, 765–769. https://doi.org/10.1016/ S0031-9384(98)00118-8 Beavers, G. A., Iwata, B. A. & Lerman, D. C. (2013). Thirty years of research on the functional analysis of problem behavior. Journal of Applied Behavior Analysis, 46, 1–21. https://doi.org/10.1002/jaba.30 Bensemann, J., Lobb, B., Podlesnik, C. A., & Elliffe, D. (2015). Steady-state choice between four alternatives obeys the constant-ratio rule. Journal of the Experimental Analysis of Behavior, 104, 7–19. https://doi.org/10. 1002/jeab.157 Boelens, H., Kop, P. F. M., Nagel, A. L., & Slangen, J. L. (1987). Concurrent schedules: Effects of reinforcement rate and changeover delay on time allocation in a three-alternative procedure. The Quarterly Journal of Experimental Psychology, 39B, 229–244. https://doi.org/ 10.1080/14640748708402266 Bizo, L. A., Bogdanov, S. V., & Killeen, P. (1998). Satiation causes within-session decreases in instrumental responding. Journal of Experimental Psychology: Animal Behavior Processes, 24, 439–452. https://doi.org/10. 1037/0097-7403.24.4.439 Burgess, I. S., & Wearden, J. H. (1986). Superimposition of response-independent reinforcement. Journal of the Experimental Analysis of Behavior, 45, 75–82. https:// doi.org/10.1901/jeab.1986.45-75 Carr, J. E., Bailey, J. S., Ecott, C. L., Lucker, K. D. & Weil, T. M. (1998). On the effects of noncontingent delivery of differing magnitudes of reinforcement. Journal of Applied Behavior Analysis, 31, 313–321. https://doi.org/10.1901/jaba.1998.31-313
10
MICHAEL E. KELLEY et al.
Davison, M., & Baum, W. M. (2000). Choice in a variable environment: Every reinforcer counts. Journal of the Experimental Analysis of Behavior, 74, 1–24. https://doi. org/10.1901/jeab.2000.74-1 Davison, M., & McCarthy, D. (1994). Effects of the discriminability of alternatives in three-alternative concurrent-schedule performance. Journal of the Experimental Analysis of Behavior, 61, 45–63. https://doi. org/10.1901/jeab.1994.61-45 DeMarse, T. B., Killeen, P. R. & Baker, D. (1999). Satiation, capacity, and within-session responding. Journal of the Experimental Analysis of Behavior, 72, 407–423. https://doi.org/10.1901/jeab.1999.72-407 Fischer, S. M., Iwata, B. A., & Mazaleski, J. L. (1997). Noncontingent delivery of arbitrary reinforcers as treatment for self-injurious behavior. Journal of Applied Behavior Analysis, 30, 239–249. https://doi.org/10. 1901/jaba.1997.30-239 Fisher, W., Piazza, C. C., Bowman, L. G., Hagopian, L. P., Owens, J. C., & Slevin, I. (1992). A comparison of two approaches for identifying reinforcers for persons with severe and profound disabilities. Journal of Applied Behavior Analysis, 25, 491–498. https://doi.org/10. 1901/jaba.1992.25-491 Fisher, W. W., Thompson, R. H., DeLeon, I. G., Piazza, C. C., Kuhn, D. E., Rodriguez-Catter, V., & Adelinis, J. D. (1999). Noncontingent reinforcement: Effects of satiation versus choice responding. Research in Developmental Disabilities, 20, 411–427. https://doi. org/10.1016/S0891-4222(99)00022-0 Goh, H.-L., Iwata, B. A., & DeLeon, I. G. (2000). Competition between noncontingent and contingent reinforcement schedules during response acquisition. Journal of Applied Behavior Analysis, 33, 195–205. https://doi.org/10.1901/jaba.2000.33-195. Hanley, G. P., Iwata, B. A., & McCord, B. E. (2003). Functional analysis of problem behavior: A review. Journal of Applied Behavior Analysis, 36, 147–185. https://doi. org/10.1901/jaba.2003.36-147. Herrnstein, R. J. (1970). On the law of effect. Journal of the Experimental Analysis of Behavior, 13, 243–266. https:// doi.org/10.1901/jeab.1970.13-243 Hunter, I., & Davison, M. (1985). Determination of a behavioral transfer function: White-noise analysis of session-to-session response-ratio dynamics on concurrent VI VI schedules. Journal of the Experimental Analysis of Behavior, 43, 43–59. https://doi.org/10.1901/jeab. 1985.43-43 Kahng, S., Iwata, B. A., Thompson, R. H., & Hanley, G. P. (2000). A method for identifying satiation versus extinction effects under noncontingent reinforcement schedules. Journal of Applied Behavior Analysis, 33, 419–431. https://doi.org/10.1901/jaba.2000. 33-419 Kelley, M. E., Shillingsburg, M. A., & Bowen, C. B. (2017). Time since reinforcer access produces gradations of motivation. Learning and Motivation, 57, 61–66. https://doi.org/10.1016/j.lmot.2016.10.003 Kuroda, T., Cançado, C. R. X., Lattal, K.A., Elcoro, M., Dickson, C. A., & Cook, J. E. (2013). Combinations of response–reinforcer relations in periodic and aperiodic schedules. Journal of the Experimental Analysis of Behavior, 99, 199–210. https://doi.org/10.1002/ jeab.13
Lalli, J. S., Casey, S. D., & Kates, K. (1997). Noncontingent reinforcement as treatment for severe problem behavior: some procedural variations. Journal of Applied Behavior Analysis, 30, 127–137. https://doi.org/10. 1901/jaba.1997.30-127 Laraway, S., Snycerski, S., Michael, J., & Poling, A. (2003). Motivating operations and terms to describe them: Some further refinements. Journal of Applied Behavior Analysis, 36, 407–414. https://doi.org/10.1901/jaba. 2003.36-407 Lattal, K. A., & Bryan, A. J. (1976). Effects of concurrent response-independent reinforcement on fixedinterval schedule performance. Journal of the Experimental Analysis of Behavior, 26, 495–504. https://doi. org/10.1901/jeab.1976.26-495 Lindberg, J. A., Iwata, B. A., Roscoe, E. M., Worsdell, A. S., & Hanley, G. P. (2003). Treatment efficacy of noncontingent reinforcement during brief and extended application. Journal of Applied Behavior Analysis, 36, 1–19. https://doi.org/10.1901/jaba.2003. 36-1 Mace, F. C., McComas, J. J., Mauro, B. C., Progar, P. R., Taylor, B., Ervin, R. and Zangrillo, A. N. (2010). DIifferential reinforcement of alternative behavior increases resistance to extinction: clinical demonstration, animal modeling, and clinical test of one solution. Journal of the Experimental Analysis of Behavior, 93, 349–367. https://doi.org/10.1901/jeab.2010.93-349 Marcus, B. A., & Vollmer, T. R. (1996). Combining noncontingent reinforcement and differential reinforcement schedules as treatment for aberrant behavior. Journal of Applied Behavior Analysis, 29, 43–51. https:// doi.org/10.1901/jaba.1996.29-43 Mazaleski, J.L., Iwata, B.A., Vollmer, T.R., Zarcone, J.R., & Smith, R.G. (1993). Analysis of the reinforcement and extinction components in DRO contingencies with self-injury. Journal of Applied Behavior Analysis, 26, 143–156. https://doi.org/10.1901/jaba.1993.26-143 McDowell, J. J. (1988). Matching theory in natural human environments. The Behavior Analyst, 11, 95–108. Nevin, J. A. (1992). Behavioral contrast and behavioral momentum. Journal of Experimental Psychology: Animal Behavior Processes, 18, 126–133. https://doi.org/10. 1037/0097-7403.18.2.126 Pinkston, J. W., Ginsburg, B. C., & Lamb, R. J. (2009). Examination of reinforcement magnitude on the pharmacological disruption of fixed-ratio performance. Experimental and Clinical Psychopharmacology, 17, 237–246. https://doi.org/10.1037/a0016609 Podlesnik, C. A., Jimenez-Gomez, C., Ward, R. D., & Shahan, T. A. (2009). Resistance to change and frequency of response-dependent stimuli uncorrelated with reinforcement. Journal of the Experimental Analysis of Behavior, 92, 199–214. https://doi.org/10.1901/ jeab.2009.92-199 Podlesnik, C. A., & Kelley, M. E. (2014). Resurgence: Response competition, stimulus control, and reinforcer control. Journal of the Experimental Analysis of Behavior, 102, 231–240. https://doi.org/10.1002/ jeab.102 Podlesnik, C. A., & Shahan, T. A. (2008). Response– reinforcer relations and resistance to change. Behavioural Processes, 77, 109–125. https://doi.org/10. 1016/j.beproc.2007.07.002
NONCONTINGENT REINFORCEMENT Reed, G. K., Piazza, C. C., Patel, M. R., Layer, S. A., Bachmeyer, M. H. Bethke, S. D., & Gutshall, K. A. (2004). On the relative contributions of noncontingent reinforcement and escape extinction in the treatment of food refusal. Journal of Applied Behavior Analysis, 37, 27–42. https://doi.org/10.1901/jaba. 2004.37-27 Rescorla, R. A., & Skucy, J. C. (1969). Effect of responseindependent reinforcers during extinction. Journal of Comparative and Physiological Psychology, 67, 381–389. https://doi.org/10.1037/h0026793 Roscoe, E. M., Iwata, B. A., & Zhou, L. (2013). Assessment and treatment of chronic hand mouthing. Journal of Applied Behavior Analysis, 46, 181–198. https://doi. org/10.1002/jaba.14
11
Sidman, M. (1960). Tactics of scientific research: Evaluating experimental data in psychology. New York, NY: Basic Books. Vollmer, T. R., & Iwata, B. A. (1991). Establishing operations and reinforcement effects. Journal of Applied Behavior Analysis, 24, 279–291. https://doi.org/10. 1901/jaba.1991.24-279 Wallace, M. D., Iwata, B. A., Hanley, G. P., Thompson, R. H., & Roscoe, E. M. (2012). Noncontingent reinforcement: A further examination of schedule effects during treatment. Journal of Applied Behavior Analysis, 45, 709–719. https://doi.org/10. 1901/jaba.2012.45-709 Received: September 26, 2016 Final Acceptance: February 28, 2017
