593529
research-article2015
AEIXXX10.1177/1534508415593529Assessment for Effective InterventionBurns et al.
Article
Effect of Modifying Intervention Set Size With Acquisition Rate Data While Practicing Single-Digit Multiplication Facts
Assessment for Effective Intervention 1–10 © Hammill Institute on Disabilities 2015 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/1534508415593529 aei.sagepub.com
Matthew K. Burns, PhD1, Anne F. Zaslofsky, PhD2, Kathrin E. Maki, MEd2, and Elena Kwong, MA2
Abstract Incremental rehearsal (IR) has consistently led to effective retention of newly learned material, including math facts. The number of new items taught during one intervention session, called the intervention set, could be used to individualize the intervention. The appropriate amount of information that a student can rehearse and later recall during one intervention session is called the acquisition rate (AR). The current study taught single-digit multiplication facts with factors of 6, 7, and 8 to 55 third- and fourth-grade students. Each student was randomly assigned to be taught two multiplication math facts, eight multiplication math facts, or a set size determined by each student’s AR. The average AR was 4.05 (SD = 0.71). Set size led to a significant and large effect on retention, percentage of multiplication math facts retained, and efficiency as computed by math facts retained per minute of intervention time. IR appeared to be an effective intervention when AR data were used to determine the intervention set size, and it was more efficient to do so. Keywords curriculum-based assessment, math Students need to be proficient in a wide variety of complex math skills, but many struggle with math problems that require high levels of analysis and application (Winick et al., 2008). Perhaps one reason why students struggle with complex math problems is that they lack the basic early math skills such as fluent computation (Kilpatrick, Swafford, & Finell, 2001; National Mathematics Advisory Panel, 2008). Moreover, students who can store math facts in memory and quickly retrieve them are likely to develop skills necessary for (a) solving a wide variety of complex problems, (b) interpreting abstract mathematical principles, and (c) successful independent living (Patton, Cronin, Bassett, & Koppel, 1997).
The Importance of Math Fact Fluency Computational fluency is the use of “efficient and accurate methods of computing” (National Council of Teachers of Mathematics, 2000, p. 32), which is enhanced by automatic recall of math facts (Gersten & Chard, 1999). A math fact is considered automatic when it is faster to solve the problem by recalling the answer than by performing the necessary algorithm (Logan, Taylor, & Etherton, 1996). The automatic processing theory of reading (Samuels, 1987) suggests that
students who do not need to devote cognitive resources to decoding the text can dedicate those resources to understanding. It is similar for math, in that students who can compute math facts with little cognitive energy can dedicate their resources to more advanced applications within the problem (Singer-Dudek & Greer, 2005). The speed with which students complete math facts (e.g., single-digit multiplication and division) could serve as an intervention goal for some students, especially given that students with math difficulties frequently struggle to quickly recall math facts (Geary, Hoard, Byrd-Craven, Nugent, & Numtee, 2007; Gersten, Jordan, & Flojo, 2005; Hanich, Jordan, Kaplan, & Dick, 2001). Practicing math facts by repeatedly presenting the facts and stating the corresponding answer has consistently led to increased retention and generalization (Burns, 2005; Codding, Archer, & Connell, 2010). Interventions that 1
University of Missouri, Columbia, USA University of Minnesota, Minneapolis, USA
2
Corresponding Author: Matthew K. Burns, University of Missouri, 109 Hill Hall, Columbia, MO 65211, USA. Email:
[email protected]
Downloaded from aei.sagepub.com at University of Missouri-Columbia on July 14, 2015
2
Assessment for Effective Intervention
include practicing math facts to increase speed of responding were more effective than teaching math facts as concepts (Kanive, Nelson, Burns, & Ysseldyke, 2014), and combining practice with strategy instruction was more effective than strategy instruction alone (Woodward, 2006). Meta-analytic research also found that practice in addition to modeling led to the largest effect size as compared with other components of math interventions (Codding, Burns, & Lukito, 2011). Thus, math interventions have frequently involved some aspect of practicing facts (Hulac, Dejong, & Benson, 2012; Zheng, Flynn, & Swanson, 2013), and repeated practice has been described as a principle for effective math intervention (Fuchs et al., 2008). Incremental Rehearsal (IR; Tucker, 1988) is an intervention strategy that has been successfully used to practice (i.e., rehearse) math facts with elementary-aged students who struggled with math (Burns, 2005; Codding et al., 2010), and meta-analytic research found large effects for math (φ = .94, confidence interval [CI] = [.80, 1.00]; Burns, Zaslofsky, Kanive, & Parker, 2012). IR rehearses each unknown fact with eight or nine known items while incrementally increasing the number of known items presented in between every presentation of the unknown fact being taught (Tucker, 1988). Previous meta-analytic research found that math fact rehearsal tasks should contain approximately 80% to 90% known items and 10% to 20% new items to represent an instructional match (Burns, 2004b), and IR incorporates 80% to 90% known items within the learning task.
Acquisition Rate (AR) and IR Although IR can be an effective intervention to rehearse facts, math interventions need to be tailored to the individual student to be successful (Fuchs & Fuchs, 2001). IR can be individualized by determining known and unknown items for each student and using the student’s known items to teach the unknown items. The number of new items practiced may also be a variable to modify to tailor the intervention to individual student needs. Students can become frustrated when too much information is covered at one time (Haegele & Burns, 2015). The amount of information that a student can rehearse and later recall during one intervention session is called the AR (Gravois & Gickling, 2002) and is a critical component of providing a match between intervention and student need (Burns & Parker, 2014). A student’s AR is measured by practicing unknown items until the student makes three errors while rehearsing one new item (Burns, 2001). An error is counted whenever the student does not present the correct response to the stimulus within 2 s. The number of new items successfully rehearsed at that point is considered the student’s AR. For example, if a student correctly rehearses four new items, but makes three errors on any item while rehearsing the fifth
item, the student’s AR is four items. The reliability of assessing AR, using a delayed-alternate form reliability estimate, was .76 among first-grade students and .91 for third and fifth graders (Burns, 2001). Moreover, AR data correlated highly with a standardized memory measure (corrected r = .70; Burns & Mosack, 2005) and followed predicted developmental patterns (fifth graders scored significantly higher than third graders, who scored significantly higher than first graders; Burns, 2004a), but all of the previous research focused on reading. For that reason, although research shows that using student AR data to tailor interventions results in improved student outcomes, research on considering AR when intervening with math skills is necessary. The concept of an AR is consistent with cognitive interference as defined by Ceraso’s (1967) seminal work that found that teaching too much information at one time resulted in difficulties acquiring new information and reduced retention of previously learned material. The reduced retention was the result of retroactive cognitive interference, which occurred when learning a new item interfered with retention of a previously learned item. For example, if a student is learning eight items, but can retain only four items (i.e., the AR), then attempting to teach the remaining four (Item Numbers 5-8) will not only lead to poor retention of the last four items but will also reduce the retention of the first four. Interventionists have recognized the importance of using smaller instructional sets (i.e., amount of new information taught or practiced during instruction or intervention) versus massed practice to increase retention (Cepeda, Pashler, Vul, Wixted, & Rohrer, 2006). However, the amount of new material taught during one lesson is often arbitrarily selected or based on estimates of what students can cognitively handle, which may result in wasted instructional time. For example, an interventionist may rehearse four math facts at one time, but the student can only retain two based on his or her AR. In this example, the student would not retain two of the four taught items, would likely forget some portion of the other two, and time spent teaching the additional two facts would not result in increased student knowledge. It would be more efficient for the interventionist to stop after teaching two facts and teach the additional two facts in a subsequent lesson. Previous research with reading determined that teaching students with sets that equated their individual AR led to better retention and more efficient instruction than using sets of two or eight items (Haegele & Burns, 2015). Teaching a number of new facts that exceed a student’s AR may result in wasted time, but not fully using the total AR may result in inefficiency. For example, if an interventionist teaches four math facts, but the student’s AR is six, then the interventionists may not have efficiently used the instructional time because he or she may have ended the
Downloaded from aei.sagepub.com at University of Missouri-Columbia on July 14, 2015
3
Burns et al. task too soon. Researchers are focusing more attention on the efficiency of instructional practices (Burns & SterlingTurner, 2010; Cates et al., 2003) because time is a valuable commodity in elementary and secondary schools. Instructional efficiency can be computed by dividing the number of items learned by the instructional time in minutes required to learn them (Skinner, Belfiore, & Watson, 1995). Previous efficiency research examined the amount of information acquired (e.g., Cates et al., 2003), but number of items retained is a higher skill, and efficiency estimates vary depending on whether they are computed from retention rather than acquisition (Burns & Sterling-Turner, 2010).
Purpose Students who struggle with math often struggle to retain and recall math facts (Geary et al., 2007), and IR can be an effective tool to remediate poor retention of facts (Burns et al., 2012; Codding et al., 2010). Determining intervention set size with AR data may help tailor interventions to student needs and increase efficiency, but previous AR research has focused on reading fluency. Therefore, we conducted this study to extend previous AR research to math. The following research questions guided the study: Research Question 1: What effect does set size have on retention of single-digit multiplication facts learned with IR? Research Question 2: What effect does determining the set size with AR have on the efficiency of IR for rehearsal of single-digit multiplication facts?
Method Participants and Setting The participants were 55 students in third or fourth grade who attended one of six classrooms from two elementary schools in the same district in Minnesota. Consent forms were sent home to all students in the third and fourth grades, and those who returned a signed consent form were included in the study. However, students receiving special education services were not included in the sample. Therefore, 44% of those eligible to participate returned the consent form and made up the sample for the study. A total of 29 (52.7%) of the students across the two schools were in third grade, and 26 (47.3%) were in fourth grade, and 29 (52.7%) of them were female. There was a relatively equal distribution of students selected from each classroom (range 7 to 11 students from one classroom). A majority of the students (n = 35, 63.6%) were Caucasian, 10 (18.2%) were African American, 7 (10.9%) were Asian American, and 4 (7.3%) were Hispanic.
One of the participating schools served 260 students in kindergarten through fifth grade. The student population of the school was 60% Caucasian, and 51% were eligible for the federal free or reduced price lunch program. Approximately 82.5% and 79.4% of the third- and fourthgrade students, respectively, scored in the proficient range on the 2013 state accountability test for math. The second school served 480 students in kindergarten through fifth grade, 59% of whom were Caucasian and 25% of whom were eligible for a free or reduced price lunch. A total of 79% of the third-grade students and 87.5% of the fourthgrade students met or exceeded standards on the 2013 state accountability test for math. All of the instructional and assessment sessions occurred at a small table in a quiet place (i.e., media center, small room, or hallway). The student participant and researcher sat across from one another at one corner of a desk or table.
Measures The Optional Local Purpose Assessment (OLPA; Minnesota Department of Education, 2013) was used to assess the math skills of each student. The OLPA is a group-administered standardized measure of math skills that was developed with item-response theory to predict how students will score on the state accountability test for Minnesota. The computer adaptive measure is administered online with a series of multiple-choice items. Items from the test equally represent four strands of (a) numbers and operations, (b) algebra, (c) geometry and measurement, and (d) data analysis and probability. The data are converted to grade-based standard scores that range from 300 to 399 for third grade and 400 to 499 for fourth grade. Scores of 350 and 450 represent proficiency in third and fourth grades, respectively. The mean OLPA math score was 350.90 (SD = 33.31) for the third-grade participants and 440.58 (SD = 34.15) for the fourth-grade students in the study, both of which approximated the 30th percentile. A total of 40% (n = 22) of the students scored in the Meets or Exceeds proficiency range on the OLPA, with one student (1.8%) exceeding the proficiency range. Therefore, a majority of the students’ (n = 33, 60%) OLPA scores fell in the Partially Meets or Does Not Meet proficiency range. OLPA data correlate with the state test of accountability for Minnesota at r = .84. Although no reliability data are reported, the standard error of measure ranges from 4 to 9 for third grade, and from 4 to approximately 11 for fourth grade. The OLPA was administered by the classroom teacher using the school’s computer lab as part of the school’s overall assessment program.
Materials The participants were taught single-digit multiplication math facts using flashcards. The facts on each flashcard
Downloaded from aei.sagepub.com at University of Missouri-Columbia on July 14, 2015
4
Assessment for Effective Intervention
consisted of Factors 6, 7, and 8, because previous research found that these factors were the most difficult for students to learn and required the most practice to reach mastery (Burns, Ysseldyke, Nelson, & Kanive, 2014). However, facts multiplied by 0, 1, and 2 (e.g., 6 × 0, 7 × 2, etc.) were not included. There were 21 total facts written on cards to serve as a potential set of unknown items. Each fact was written on a 3 by 5 index card with black ink in a portrait orientation. Eight potential known facts were identified by randomly selecting 8 single-digit multiplication facts from the 15 facts taken from the 0s, 1s, and 2s to 5s (e.g., 0 × 3, 2 × 4, 1 × 5). The 8 randomly selected known items were also written in black ink on a 3 by 5 index card using a portrait orientation.
Unknown Fact 1 Known Fact 1 Unknown Fact 1 Known Fact 1 Known Fact 2 Unknown Fact 1 Known Fact 1 Known Fact 2 Known Fact 3 Unknown Fact 1 Known Fact 1 Known Fact 2 Known Fact 3 Known Fact 4 Unknown Fact 1
Known Fact 1 Known Fact 2 Known Fact 3 Known Fact 4 Known Fact 5 Unknown Fact 1 Known Fact 1 Known Fact 2 Known Fact 3 Known Fact 4 Known Fact 5 Known Fact 6 Unknown Fact 1 Known Fact 1 Known Fact 2
Known Fact 3 Known Fact 4 Known Fact 5 Known Fact 6 Known Fact 7 Unknown Fact 1 Known Fact 1 Known Fact 2 Known Fact 3 Known Fact 4 Known Fact 5 Known Fact 6 Known Fact 7 Known Fact 8
Figure 1. Incremental Rehearsal With Math Facts.
Conditions Before beginning the study, each student was randomly assigned to one of the three conditions (sets of two, sets of eight, or sets determined by AR). The first group rehearsed two unknown single-digit multiplication facts, the second group rehearsed eight unknown facts, and the third group rehearsed the number of facts matched to their AR, which was determined during the intervention. All facts were taught in one intervention session. The numbers two and eight were selected because previous research found a mean AR of approximately 5 (SD = approximately 2) among third-grade students, and of 6 (SD = approximately 2) among fifth-grade students (Burns, 2004a). Thus, two fell more than two standard deviations below the lowest mean, and eight exceeded two standard deviations above the mean for fifth graders. Two and eight were also the conditions used for previous AR research (Haegele & Burns, 2015). The mean OLPA score was 390.74 (SD = 55.26) for the group of students with set sizes determined by AR, 396.67 (SD = 58.61) for those who rehearsed two facts, and 392.61 (SD = 57.80) for those who rehearsed eight facts. A oneway ANOVA resulted in a nonsignificant, F(2, 52) = 0.05, p = .95, and negligible, η2 = .002, effect. The three groups had equivalent math skills before beginning the intervention.
Intervention All interventions occurred during one session at each school. Unknown single-digit math facts were taught with IR at a ratio of one new fact to eight known facts. The sequence began by presenting the unknown fact to the student while verbally reading it to him or her and providing the correct answer (e.g., 6 × 8 = ? would be read as “six times eight equals forty-eight”). Next, the student was asked to restate the entire equation, including the answer that was not provided on the card. Finally, the student was asked one final time to complete the equation by restating the equation
Note. Unknown Fact 1 becomes Known Fact 1, and Known Fact 8 is removed at completion of one cycle.
and the answer to the math fact. Once the fact was correctly answered within 2 s of the prompt, it was rehearsed with the pattern shown in Figure 1. The student was asked to orally provide the answer to the math fact every time it was presented. The rehearsal pattern for the unknown fact described in Figure 1 was used for every unknown fact. Each time a new unknown fact was presented, the previous unknown fact was treated as the first known fact, the previous eighth known fact was removed from the deck when a different unknown fact was added to the set, and the process began over again. After the student practiced an unknown fact eight times, that item became a known fact and a new unknown item was rehearsed in the same manner outlined in Figure 1. AR. Each participant’s AR was assessed with procedures outlined by Burns (2001), which involved rehearsing unknown multiplication facts with IR at a ratio of one unknown fact and eight known facts, as shown in Figure 1. Any time a student did not provide the correct response to the multiplication fact written on the card, regardless if it was designated as “unknown” or “known,” it was immediately corrected and counted as an error. New unknown facts were added into the sequence until the student made three errors while practicing any of the new unknown facts. At this time, the number of unknown facts successfully completed was recorded as the AR. For example, if a student rehearsed the first four unknown facts while making few errors, but made three errors while completing the fifth fact, his AR would be five. Previous research with reading found that ARs can be reliably measured (r > .90) for third- and fifth-grade students (Burns, 2001) and were highly correlated (r = .70)
Downloaded from aei.sagepub.com at University of Missouri-Columbia on July 14, 2015
5
Burns et al. with a standardized norm-referenced measure of memory (Burns & Mosack, 2005). IR was used as the instructional approach because it was used in the aforementioned previous AR research. Retention. To test retention of the unknown facts that were taught, participants were shown the cards of the targeted multiplication facts 1 day after the intervention. If the student stated the correct answer within 2 s, then it was considered retained. If he or she stated it incorrectly, or did not remember it within 2 s, it was counted as not retained. The number of multiplication facts retained 1 day later was the primary dependent variable to address the first research question. However, because the number of facts taught varied by condition, the data were converted to a percentage by dividing the number retained by the total number rehearsed (two, eight, or the set size determined by the AR). Efficiency. Each intervention session was timed with a stopwatch to determine intervention efficiency. Timing began when the first unknown math fact was presented and stopped after the student’s response to the last known item. Efficiency was determined by converting the time required to rehearse the new facts to a decimal by dividing the number of s by 60 (e.g., 4 min and 15 s equals 4.25 s) and then dividing the number of facts that were retained 1 day later by the total time for each student. The number of multiplication facts retained per intervention minute served as the data for the analyses to address the second research question.
Procedure Each student accompanied one of the data collectors to a quiet place, the study was explained, and written assent was obtained. The unknown math facts were assessed by shuffling the 21 cards on which the facts were horizontally hand-written and presenting each to the student 1 at a time. Any facts that were not correctly answered within 2 s were identified as unknown items to use in the study. The known items were also tested in a similar fashion. The cards containing the eight known facts were shuffled and presented one at a time to each student. All facts correctly answered within 2 s of presentation were identified as known. Any fact not answered, not answered correctly, or answered correctly after 2 s was identified as an unknown. All students correctly answered all eight known facts before the study began. The cards containing the unknown items were then shuffled and were rehearsed in the order that they appeared. However, a fact was reordered if it fell within one of the previous facts (e.g., 6 × 8 would be reordered if it immediately followed 6 × 7, 6 × 9, or 7 × 8), and the reversed fact was not rehearsed. For example, 8 × 6 would be removed
from the deck of possible unknown facts if 6 × 8 appeared first and was already rehearsed. The facts were rehearsed with IR as described above. The number of facts rehearsed depended on the condition to which the student was randomly assigned. The intervention ended after two facts for one condition, eight facts for the second condition, and after reaching the student’s AR for the third condition. After completing the intervention, the students were asked to return to class. Each session was timed using a stopwatch to measure intervention efficiency. Timing began with the presentation of the first multiplication fact and ended after the presentation of the last multiplication fact. An interventionist returned to the school the next day and tested retention. The cards that contained the facts that were taught the day before were shuffled and presented one at a time to the student. Each fact correctly answered within 2 s was identified as retained. Thus, each student participated in one intervention session and one assessment session in which retention of the math facts was assessed. Data were collected within a 2-week period. Approximately 33% of the students participated in the intervention on the first day, and their retention was assessed on the second day. Another approximate 33% of the students received the intervention on the third day and assessed for retention on the fourth day, and all of the first four days occurred within the same week. The researchers completed the final intervention and assessment sessions with the remaining third of the students during the week that immediately followed the first two intervention and assessment sessions (i.e., first, second, third, and fourth days).
Fidelity and Interobserver Agreement The data were collected by two female school psychology graduate students and the first author. Each interventionist was trained in IR and how to assess an AR before beginning the study. Then, each observed the first author complete the study procedures with one of the participants before beginning on their own. All data collectors had to demonstrate at least 95% integrity before beginning data collection. The procedures for conducting IR and assessing the AR were observed by the first author for 20% of the data collection sessions using an eight-item implementation checklist. The total number of items correctly implemented was divided by the total number of items and multiplied by 100. Each observed intervention session was completed with 100% correct implementation. Interobserver agreement (IOA) for assessing the AR was also computed during the fidelity observations of 20% of the intervention sessions. The observer noted the point at which three errors were recorded while rehearsing one multiplication fact and the AR was reached. Instances in which the interventionist and second observer both judged the AR
Downloaded from aei.sagepub.com at University of Missouri-Columbia on July 14, 2015
6
Assessment for Effective Intervention
Table 1. Retention and Efficiency Data by Condition. Set size
Two (n = 18)
Variable
M
SD
M
SD
M
SD
F
η2
3.19 0.47 0.31
0.67 0.46 0.31
6.70 0.76 0.51
1.84 0.23 0.28
14.40 0.39 0.25
4.61 0.21 0.16
68.87* 8.29* 5.34*
.73 .25 .17
Intervention length Percentage retained Efficiency
Acquisition rate (n = 19)
Eight (n = 18)
Note. Intervention length is the number of minutes needed to complete the intervention, and efficiency is the number of facts retained divided by intervention length in minutes. *p < .017.
to be reached were identified as agreements. If the interventionist judged the AR to be reached and stopped the session before the second observer determined that AR had been reached, or if the interventionist continued the intervention beyond the point that the second observer judged the AR to be reached, then the session was recorded as a disagreement. The total number of agreements between the interventionist and second observer was divided by the total number of observed session (agreements plus disagreements) and multiplied by 100, which resulted in a total of 90% agreement. Likewise, IOA was computed for 20% of the retention sessions. The observer and interventionist recorded students’ answers to the retention items as correct or incorrect independently. Answers that were marked as correct or incorrect by both the interventionist and observer were counted as agreements. Answers that were marked as correct by the interventionist and incorrect by the observer, or vice versa, were counted as disagreements. The total number of agreements between the interventionist and second observer divided by the total number of observed sessions (agreements plus disagreements) and multiplied by 100 resulted in 100% agreement.
Analyses The effect of instructional set size was addressed by comparing the percent of facts retained for each condition. The data were analyzed with a one-way ANCOVA in which the condition was the independent variable, the percentage of facts retained was the dependent variable, and OLPA math score was the covariate. Before conducting the analyses to address the research question, a separate ANCOVA was conducted using the interaction between the independent variable (condition) and covariate (OLPA math score). The interaction effect was not significant, F(5, 50) = 1.23, p = .46, which suggested that the independent variable and covariate did not significantly interact. The effect of intervening at a student’s AR on intervention efficiency was addressed with a one-way ANOVA. The condition was
again used as the independent variable, and the efficiency (computed as the number of facts retained for each instructional minute) was the dependent variable. The number of minutes needed for each instructional condition was also compared with an ANOVA. A corrected alpha level was used to address the research questions. Three analyses were conducted. Therefore, an alpha level of .017 was used to determine significance. A partial η2 was also computed as an estimate of effect, in which a value of .14 or higher was considered large (Cohen, 1988).
Results The data collected for the study were first examined to determine the normality of the distribution. All but two of the estimates of kurtosis and skew were less than 1.00, and the remaining two were slightly more than 1.00, which suggested that the data were acceptably distributed to conduct the parametric analyses (Howell, 2002). Next, the OLPA scores across the two grades were compared with independent t tests. There were no significant differences between the third and fourth graders on length of intervention time, t(53) = .24, p = .81, d = .07; percentage of facts retained, t(53) = .20, p = .85, d = .03; or efficiency, t(53) = 1.01, p = .32, d = .29. Therefore, the data were combined for subsequent analyses.
Set Size and Retention The first research question inquired about the effect that the set size had on retention of single-digit multiplication facts. As shown in Table 1, the students retained approximately 76% of the math facts taught within the AR condition, but remembered less than 50% when taught 2 facts and just more than 33% when taught 8 facts. The result of the ANCOVA was significant, F(2, 52) = 8.29, p < .017, and the effect was large (η2 = .25), which indicated that there was a significant difference between the three conditions on the number of facts retained. On average, the students in the AR condition were taught 4.05 facts (SD = 0.71) and retained
Downloaded from aei.sagepub.com at University of Missouri-Columbia on July 14, 2015
7
Burns et al. 3.16 (SD = 1.21) of them. The students who were taught 2 facts retained an average of 0.94 facts (SD = 0.87), and those who were taught 8 facts retained an average of 3.11 (SD = 1.64). Post hoc analyses determined that the AR condition led to significantly higher retention than students who were taught 2 facts, t(35) = 2.55, p < .017, or 8 facts, t(35) = 5.22, p < .017. Interestingly, the difference in retention between students who were taught 2 or 8 facts was not significant, t(35) = 0.73, p = .47.
Intervention Efficiency As also shown in Table 1, condition had a significant effect on time required to complete the intervention. Students in the AR condition required approximately 7 min, those who rehearsed two facts required about 3 min, and those who rehearsed eight facts required approximately 14 min. Therefore, it required approximately twice as long to rehearse the number of facts as determined by the AR as it did to rehearse two facts, and twice as long to rehearse eight facts as it did to reach their AR. The number of facts retained per intervention minute is also outlined in Table 1. It was most efficient to rehearse facts until each student reached their AR (M = .51 facts retained per intervention minute, SD = .28), and approximately equal for rehearsing two (M = .31, SD = .31) or eight facts (M = .25, SD = .16). The effect that condition had on efficiency was significant, F(2, 52) = 5.34, p < .017, and large (η2 = .17). Post hoc analyses found that the AR condition led to significantly higher efficiency than students who rehearsed eight facts, t(35) = 3.50, p < .017. However, the difference in efficiency did not reach significance between the AR condition and the condition in which students rehearsed two facts, t(35) = 2.05, p = .048, or between students who rehearsed two or eight facts, t(35) = .83, p = .41.
Discussion The purpose of this study was to extend AR research by examining the effect that intervention set size had on retention of single-digit multiplication facts rehearsed with IR and on the efficiency using IR to rehearse single-digit multiplication facts. Students in the AR condition retained the highest percentage of facts compared with students who rehearsed two unknown facts and students who rehearsed eight unknown facts, and the AR condition was the most efficient condition. The comparison of set sizes led to large effects on percentage of items retained and efficiency but also on time needed to complete the intervention. The current study supported the importance of providing an appropriate level of challenge by measuring an AR (Gravois & Gickling, 2002) and was consistent with Ceraso’s (1967) theory of cognitive interference in which attempting to teach too much information at one time
resulted in difficulties acquiring and retaining new information. For example, students in the AR condition retained on average 3.15 facts (of the 4.05 mean), and students who were taught 8 facts retained 3.11 facts. Thus, the number of facts retained was almost identical between the two conditions, but students in the latter condition were taught twice as many facts. It was more effective and efficient to stop the intervention session after reaching the students’ AR (approximately four) than it was to attempt to teach the additional four facts. IR was an effective intervention because students retained 75% of the math facts in the AR condition, which is consistent with previous AR research (Burns, 2005; Burns et al., 2012; Codding et al., 2010). However, the effectiveness seemed to be at least partially dependent on how many items were rehearsed. The small percentage of items retained when rehearsing only two facts worked to counter the concept of an AR. It seems that if students retain approximately 75% of the math facts when taught in the AR condition, then they would retain close to 100% of facts when taught only two at a time. The relative ineffectiveness of using sets of two facts could be attributed to fewer opportunities to respond (OTR). OTR is defined as each interaction where a student responds to the presented instructional stimulus and immediately receives feedback from the teacher (Greenwood, Delquadri, & Hall, 1984). Previous research has consistently linked increases in OTR to increased retention (Burns, 2007; Hargis, Terhaar-Yonkers, Williams, & Reed, 1988) and has been suggested to be a potential causal mechanism for IR (Szadokierski & Burns, 2008). The unknown facts in the IR model were left in as known items after they were taught, and the students continued to rehearse them while practicing the subsequent unknown facts. For example, with 8 known items, a student would practice the first unknown fact 8 times, but then would practice it an additional 7 (total of 15) while rehearsing the second fact, 6 more (total of 21) while rehearsing the third fact, and so on. Students had fewer opportunities to rehearse the fact in the set made up of only 2 because the first fact was rehearsed a total of 15 times and the second fact, a total of 8 times. The effect of set size needs to be further researched to better isolate its effect from OTR, but teaching students until they reach their AR seems to provide the correct balance of OTR while not presenting too much information. Instructional efficiency was calculated based on instructional time (Cates et al., 2003), and there was a large effect on time and efficiency. Time is one of the most precious commodities in education, which makes the efficiency of using a student’s AR to determine set size a potentially useful finding. Interventionists should consider using instructional efficiency when determining which intervention to use because sometimes the most effective intervention is not the most efficient.
Downloaded from aei.sagepub.com at University of Missouri-Columbia on July 14, 2015
8
Assessment for Effective Intervention
Although the term acquisition rate has been consistently used in the literature to refer to the amount of information that a student can successfully rehearse and recall at a later time (Burns, 2001; Burns & Dean, 2005; Gravois & Gickling, 2002), it is not actually a rate. The term rate implies a comparison of a quantity with a unit of time (e.g., words read correctly per minute, words learned per instructional minute, miles per hour). Although the study included an efficiency metric that was reported as a rate, the AR itself was a measure of quantity with no comparison with time. It would be more accurate to call the number of facts retained per instructional minute an AR, but the term AR was used here in a manner that is consistent with previous research. Researchers and scholars should consider a more accurate way to reflect the construct being assessed and use that term in future research.
research should also consider conceptual understanding of multiplication within the context of AR.
Conclusion Published AR research remains in its infancy with most previous efforts focusing on psychometric properties of the data (Burns, 2001, 2004a; Burns & Mosack, 2005). The current data suggest direct instructional implications. AR data are directly linked to instruction, are easily interpreted by interventionists, and are reliably assessed. Interventions should be matched to student needs, and knowing how much information to teach for each student could be an important piece of that picture. Declaration of Conflicting Interests The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Limitations and Future Directions Although the current data are potentially interesting to researchers and practitioners, they should be examined within the context of their limitations. First, the students were all in the third and fourth grades, but rapid fact calculation and number naming are important skills for younger students as well. Future researchers could replicate the study with a more diverse group to enhance the external validity and practical applications. Second, the current study was only conducted with students who knew all eight known facts. Therefore, the extent to which the findings apply to students who know very few multiplication facts is unknown. Third, the instructional stimuli were unknown facts taken from the most difficult facts to acquire (Burns et al., 2014). Thus, the effect on easier facts or other aspects of math is unknown. Moreover, using math facts enhances the application of the data to instructional settings, but it could have limited the internal validity because students may have learned the math facts as part of math instruction during the teach–test interval. The interval between the facts being taught and tested was only 1 day, but the effect of math instruction on the results cannot be ruled out. Fourth, retention was tested 1 day after the math facts were taught; therefore, the extent to which students retained math facts over time is not known. Future research could examine the relationship between AR and retention over a longer period of time (e.g., 1 week, 1 month). Fifth, we did not examine the effect that classroom instruction had on student retention. Although each participating classroom was somewhat equally represented, the effect that the classroom had on the results was not addressed and could be an area of future research. Finally, the current study focused on teaching math facts and therefore did not account for student conceptual understanding of multiplication. Therefore, future
Funding The authors received no financial support for the research, authorship, and/or publication of this article.
References Burns, M. K. (2001). Measuring acquisition and retention rates with curriculum-based assessment. Journal of Psychoeducational Assessment, 19, 148–157. Burns, M. K. (2004a). Age as a predictor of acquisition rates as measured by curriculum-based assessment: Evidence of consistency with cognitive research. Assessment for Effective Intervention, 29, 31–38. doi:10.1177/073724770402900203 Burns, M. K. (2004b). Empirical analysis of drill ratio research: Refining the instructional level for drill tasks. Remedial and Special Education, 25, 167–175. doi:10.1177/074193250402 50030401 Burns, M. K. (2005). Using incremental rehearsal to practice multiplication facts with children identified as learning disabled in mathematics computation. Education & Treatment of Children, 28, 237–249. Burns, M. K. (2007). Comparison of drill ratio and opportunities to respond when rehearsing sight words with a child with mental retardation. School Psychology Quarterly, 22, 250–263. Burns, M. K., & Dean, V. J. (2005). Effect of acquisition rates on off-task behavior with children identified as learning disabled. Learning Disability Quarterly, 28, 273–281. Burns, M. K., & Mosack, J. (2005). Criterion-referenced validity of measuring acquisition rates with curriculum-based assessment. Journal of Psychoeducational Assessment, 25, 216–224. Burns, M. K., & Parker, D. C. (2014). Curriculum-based assessment for instructional design: Using data to design instruction and intervention. New York, NY: Guilford Press. Burns, M. K., & Sterling-Turner, H. (2010). Comparison of efficiency measures for academic interventions based on acquisition and maintenance of the skill. Psychology in the Schools, 47, 126–134.
Downloaded from aei.sagepub.com at University of Missouri-Columbia on July 14, 2015
9
Burns et al. Burns, M. K., Ysseldyke, J., Nelson, P., & Kanive, R. (2014). Number of repetitions required to retain single-digit multiplication math facts for elementary students. School Psychology Quarterly. Advance online publication. doi:10.1037/spq0000097 Burns, M. K., Zaslofsky, A. F., Kanive, R., & Parker, D. C. (2012). Meta-analysis of incremental rehearsal: Using phi coefficients to compare single-case and group designs. Journal of Behavioral Education, 21, 185–202. doi:10.1007/s10864012-9160-2 Cates, G. L., Skinner, C. H., Watson, T. S., Meadows, T. J., Weaver, A., & Jackson, B. (2003). Instructional effectiveness and instructional efficiency as considerations for data-based decision making: An evaluation of interspersing procedures. School Psychology Review, 32, 601–616. Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132, 354– 380. doi:10.1037/0033-2909.132.3.354 Ceraso, J. (1967). The interference theory of forgetting. Scientific American, 217, 117–124. doi:10.1038/scientificamerican1067-117 Codding, R. S., Archer, J., & Connell, J. (2010). A systematic replication and extension of using incremental rehearsal to improve multiplication skills: An investigation of generalization. Journal of Behavioral Education, 19, 93–105. doi:10.1007/s10864-010-9102-9 Codding, R. S., Burns, M. K., & Lukito, G. (2011). Meta-analysis of mathematic basic-factfluency interventions: A component analysis. Learning Disabilities Research & Practice, 26, 36–47. Cohen, J. (1988). Statistical power analyses for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum. Fuchs, L. S., & Fuchs, D. (2001). Principles for the prevention and intervention of mathematics difficulties. Learning Disabilities Research & Practice, 16, 85–95. Fuchs, L. S., Fuchs, D., Powell, S. R., Seethaler, P. M., Cirino, P. T., & Fletcher, J. M. (2008). Intensive intervention for students with mathematics disabilities: Seven principles of effective practice. Learning Disability Quarterly, 31, 79–92. Geary, D. C., Hoard, M. K., Byrd-Craven, J., Nugent, L., & Numtee, C. (2007). Cognitive mechanisms underlying achievement deficits in children with mathematical learning disability. Child Development, 78, 1343–1359. doi:10.1111/ j.1467-8624.2007.01069.x Gersten, R., & Chard, D. (1999). Number sense rethinking arithmetic instruction for students with mathematical disabilities. The Journal of Special Education, 33, 18–28. doi:10.1177/002246699903300102 Gersten, R., Jordan, N. C., & Flojo, J. R. (2005). Early identification and interventions for students with mathematics difficulties. Journal of Learning Disabilities, 38, 293–304. doi:10.11 77/00222194050380040301 Gravois, T. A., & Gickling, E. (2002). Best practices in curriculum-based assessment. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology IV (pp. 885–898). Bethesda, MD: National Association of School Psychologists. Greenwood, C. R., Delquadri, J., & Hall, R. V. (1984). Opportunity to respond and student academic performance. In W. Heward,
T. Heron, D. Hill & J. Trap-Porter (Eds.), Focus on behavior analysis in education (pp. 58–88). Columbus, OH: Charles E. Merrill. Haegele, K., & Burns, M. K. (2015). The effect of modifying instructional set size based on the acquisition rate among students identified with a learning disability. Journal of Behavioral Education, 24, 33–50. Hanich, L. B., Jordan, N. C., Kaplan, D., & Dick, J. (2001). Performance across different areas of mathematical cognition in children with learning difficulties. Journal of Educational Psychology, 93, 615–626. doi:10.1037//00220663.93.3.615 Hargis, C. H., Terhaar-Yonkers, M., Williams, P. C., & Reed, M. T. (1988). Repetition requirements for word recognition. Journal of Reading, 31, 320–327. Howell, D. C. (2002). Statistical methods for psychology (5th ed.). Pacific Grove, CA: Duxbury. Hulac, D. M., Dejong, K., & Benson, N. (2012). Can students run their own interventions? A self-administered math fluency intervention. Psychology in the Schools, 49, 526–538. Kanive, R., Nelson, P., Burns, M. K., & Ysseldyke, J. (2014). Comparison of computer-based practice and conceptual understanding interventions on math fact fluency. Journal of Educational Research, 107, 83–89. Kilpatrick, J., Swafford, J., & Finell, B. (2001). Adding it up: Helping children learn mathematics. Washington, DC: National Academy Press. Logan, G. D., Taylor, S. E., & Etherton, J. L. (1996). Attention in the acquisition and expression of automaticity. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 620–638. doi:10.1037//0278-7393.22.3.620 Minnesota Department of Education. (2013). Summary of Optional Local Purpose Assessment (OLPA) mathematics in grades 3–8. St. Paul, MN: Author. National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics (Vol. 1). Reston, VA: Author. National Mathematics Advisory Panel. (2008). Foundations for success: The final report of the National Mathematics Advisory Panel. Washington, DC: U.S. Department of Education. Patton, J. R., Cronin, M. E., Bassett, D. S., & Koppel, A. E. (1997). A life skills approach to mathematics instruction preparing students with learning disabilities for the real-life math demands of adulthood. Journal of Learning Disabilities, 30, 178–187. Samuels, S. J. (1987). Information processing abilities and reading. Journal of Learning Disabilities, 20, 18–22. doi:10.1177/002221948702000104 Singer-Dudek, J., & Greer, R. D. (2005). A long-term analysis of the relationship between fluency and the training and maintenance of complex math skills. The Psychological Record, 55, 361–376. Skinner, C. H., Belfiore, P. J., & Watson, T. S. (1995). Assessing the relative effects of interventions in students with mild disabilities: Assessing instructional time. Assessments in Rehabilitation and Exceptionality, 20, 207–220. doi:10.1177/ 073428290202000403
Downloaded from aei.sagepub.com at University of Missouri-Columbia on July 14, 2015
10
Assessment for Effective Intervention
Szadokierski, I., & Burns, M. K. (2008). Analogue evaluation of the effects of opportunities to respond and ratios of known items within drill rehearsal of Esperanto words. Journal of School Psychology, 46, 593–609. doi:10.1016/ j.jsp.2008.06.004 Tucker, J. A. (1988). Basic flashcard technique when vocabulary is the goal. Unpublished teaching materials, School of Education, University of Chattanooga, Chattanooga, TN. Winick, D. M., Avallone, A. P., Smith, C. E., Crovo, M., Martin, W., Olson, J., & Wilson, L. (2008). Mathematics framework
for the 2009 National Assessment of Educational Progress. Washington, DC: U.S. Department of Education, National Assessment Governing Board. Woodward, J. (2006). Developing automaticity in multiplication facts: Integrating strategy instruction timed practice drills. Learning Disability Quarterly, 29, 269–289. Zheng, X., Flynn, L. J., & Swanson, H. L. (2013). Experimental intervention studies on word problem-solving and math disabilities: A selective analysis of the literature. Learning Disability Quarterly, 36, 97–111.
Downloaded from aei.sagepub.com at University of Missouri-Columbia on July 14, 2015
