Journal of Special Education Technology
Using Electronic Performance Support Systems to Improve Academic Performance of Secondary Students with Disabilities Katherine J. Mitchem California University of Pennsylvania Gail Fitzgerald University of Missouri Kevin Miller Buffalo State College Candice Hollingsead Northern State University A key challenge in secondary education and transition is ensuring that students with disabilities are prepared to access and participate in postsecondary education. Electronic performance support systems offer potential for addressing needs of secondary students who are at risk for failure or who encounter challenges in school due to high-incidence disabilities. The purpose of this study was to investigate the effects of training and implementation of an electronic performance support systems on targeted IEP goals for ninth and twelfth grade students with disabilities. A series of multiple probe designs was used to examine the effectiveness of tool usage across four ninth grade students and four twelfth grade students as well as to investigate tool usage across settings (training, academic, and transition) for each student. This study demonstrated an improvement in target behaviors when the intervention was introduced in the training setting for ninth grade and twelfth grade students with high-incidence disabilities. In addition, each student showed some improvement in the target behavior when the intervention was implemented across settings. The authors discuss limitations along with implications for future research and practice.
A
ccording to findings from the National Longitudinal Transition Study-2 (Wagner, Newman, Cameto, Levine, & Marder, 2007), postsecondary education is a primary goal for 80% of secondary students with disabilities (Newman, Wagner, Cameto, & Knokey, 2009). Given that the majority of students with disabilities expect to finish high school with a regular diploma and access postsecondary education, secondary schools today must ensure that coursework, supports, and accommodations are provided to make these expectations a reality (Shaw, 2009). Ensuring that students with disabilities have “access to and full participation in postsecondary education” has been identified as one of
JSET 2013 Volume 28, Number 3
the key challenges in secondary education and transition (National Center on Secondary Education and Transition, 2003, p.1). How best to facilitate this requires careful and effective transition planning that includes identification of strategies that allow students not only to access but also to succeed in postsecondary education. The end of the 20th century saw a paradigm shift in beliefs and practices about how best to educate and support students with special needs in secondary schools (Gersten, 1998). Programming moved from remedial, pull-out classes to integrated models where students remain in general education environments and receive support and
1
Journal of Special Education Technology accommodations as needed. Those students with special needs who are included in high school general education settings are typically those with high-incidence learning disabilities and/or emotional/behavioral disorders as well as those at risk for school failure. They typically exhibit problems such as disorganization, poor reading comprehension and study skills, ineffective learning strategies, difficulty with classroom behavior and social interactions, impulsive behaviors, and failure to plan ahead and engage in self-control (Bryant, Bryant, & Raskind, 1998; Edyburn, 2000; Okolo, 2000). To be successful in integrated general education classrooms—with their heavy focus on mastery of content and independent, self-guided work habits—all students have increased needs for self-regulation, learning, and problemsolving strategies (Pierson, Carter, Lane, & Glaeser, 2008; Test, Mazzotti, Mustian, Fowler, Kortering, & Kohler, 2009). Many educators struggle to prepare students with disabilities to respond successfully to the heavy curriculum demands of middle and high school coupled with this need for self-regulation. At these levels, textbooks and instructional materials often consist primarily of expository text that is more difficult than narrative text for all students, especially those with learning disabilities, to comprehend (Deshler et al., 2001; Mastropieri, Scruggs, & Graetz, 2003; Williams, 2005). Because of the difficulty that students have in comprehending expository texts, effective instructional practices such as content enhancements must support them in learning from such texts. Content enhancements such as graphic and advance organizers, computer-assisted instruction, study guides, and cognitive strategy instruction are important because they enable teachers to present key ideas and their interrelationships and provide students with a scaffold for structuring information (Gajra, Jitendra, Sood, & Sacks, 2007). Recent federally-funded projects for students with mild/ moderate disabilities using the computer as a study tool (electronic studying, electronic note taking, learning study strategies) assert that technology tools, rather than remediation, provide bridges to support students in learning (Mitchem, Fitzgerald, Koury, Cepel, & Boonseng, 2009; Mitchem, Kight, Fitzgerald, & Koury, 2007). Technology offers assistance by providing direct instruction and scaffolding learning. Englert, Zhao, Dunsmore, Collings, and Wolbers (2007) suggest that technologies can be designed to offer scaffolds in the writing process and “prompt routines and processes in a timely way just as a tutor might
2
prompt students to employ particular writing procedures and actions” (p.11). Researchers have documented the effectiveness of computer-based interventions on both academic and social behaviors (Boon, Fore, Blankenship & Chalk, 2007), including computer-based cognitive organizers on social studies performance (Boon, Burke, Fore, & Spencer, 2006); computer-assisted instruction and reading comprehension (Stetter & Hughes, 2010), computer-based cognitive mapping on reading comprehension for students with emotional or behavioral disorders (Blankenship, Ayres, & Langone, 2005); and computer-assisted learning (CAI) on student knowledge of the Self-Determined Learning Model of Instruction and disruptive behavior (Mazzotti, Wood, Test, & Fowler, 2012). In response to reported potential barriers created by technology for students with disabilities (Parker & Banerjee, 2007), Banerjee (2010) has described the computer as a cognitive tool for implementing computer-based study strategies, but cautions that technology is only successful when students assume responsibility for learning, a conclusion supported by other researchers (AndersonInman & Horney, 1997; Anderson-Inman & Horney, 2007; Anderson-Inman, Knox-Quinn & Szymanski, 1999; Lewis, 2005). Supporting this conclusion, Hartley (2001) found high school students could learn strategies from hypermedia computer programs, but learning these strategies did not impact performance. In a research study on the use of hypermedia to assist secondary students’ learning, Hartley hypothesized that better outcomes might occur if instruction in learning strategies was integrated with opportunities to utilize the strategies in realistic settings. He concluded that the use of strategies by secondary students ultimately depends on the decision for usage. These studies lend support to an approach that utilizes computer-based performance support tools to facilitate strategy use, thus building self-responsibility for the student’s own learning and performance.
Electronic Performance Support Systems Electronic performance support systems—a relatively new field in technology—have potential for addressing the needs of secondary students who are at risk for failure or who encounter challenges in school due to mild disabilities (Fitzgerald, 2005). The goal of this system is to provide
JSET 2013 Volume 28, Number 3
Journal of Special Education Technology whatever supports are necessary to ensure performance and learning at the moment of need in a seamless activity (Gustafson, 2000; Laffey, 1995; Schaff, Bannan-Ritland, Behrmann, & Ok, 2005). Electronic performance support system tools can be designed that integrate the main components of such a system—information, user guidance, procedural tools, and feedback—with technological enhancements for effective use. Although system components are being included in some general education software, they are only starting to emerge in specialized software for students with mild cognitive and behavioral disabilities. Initial developments with electronic performance support systems in special education were limited to tools related to academic learning, primarily for literacy and story writing (Schaff et al., 2005), and hypertext reading supports (Higgins & Boone, 1990a; Higgins & Boone, 1990b; Higgins, Boone, & Lovitt, 2002). Such efforts led developers to conclude that, “Children and their reading facilitators benefit greatly from access to higherlevel strategies and visual, text, and motivational supports as well as engagement with reading content.” As noted by Schaff et al. (2005), “the philosophy and structure of
[electronic performance support system] programs hold great potential for those with special needs” (p. 505). The StrategyTools™ Support System used in this study is based on cognitive-behavioral approaches and addresses secondary level concerns for student self-regulation, strategic learning, and self-determination (Fitzgerald, 2005). With U.S. Department of Education funding, the StrategyTools Support System was developed and tested prior to use in this study (Fitzgerald & Koury, 2004 –2005; Mitchem et al., 2007; Mitchem et al., 2009). The conceptual framework for the StrategyTools Support System, as shown in Figure 1, is based on the innovative use of an electronic performance support system to assist students with learning disabilities and/or emotional behavioral disability by developing self-regulation and learning strategies using cognitive behavioral approaches within existing curriculum. This system is operationalized through computerized template tools. It scaffolds the student through cognitive processes of self-regulatory behavior, problem solving, and learning. Previous qualitative research on the StrategyTools system has explored its feasibility and acceptability as an assistive
Figure 1 Conceptual framework for the StrategyTools™ Support system.
Learning Strategies Online Discussion Groups
METHOD EPSS Software
CONTENT Orientation Module for Implementation
JSET 2013 Volume 28, Number 3
Transition Planning Resource Infobase for Teachers and Parents
MATERIALS
CognitiveBehavioral Self-Regulation
Strategy Coach Website
3
Journal of Special Education Technology technology with three high school students identified with behavior disorders and two special education teachers (Mitchem et al., 2007). In another quasi-experimental study utilizing a wait list control design in which students were divided into two groups and received staggered intervention, seven high school teachers and 35 students with mild learning and behavior disabilities used the StrategyTools over a six-month duration (Mitchem et al., 2009). Results indicated that students were successful in learning and using tools at expected levels on goal attainment scales, and their success exceeded expectations when they generalized tool usage to new goals or settings. There were no significant performance differences between treatment and replication groups at the conclusion of intervention. The current study is part of a larger Institute for Education Sciences Goal 2 project designed to evaluate the effects of StrategyTools on improving transition and secondary outcomes of high school students with disabilities across three sites in two large northeastern states (Fitzgerald, Mitchem, Koury, Miller, & Hollingsead, 2008 –2010). This study reports the outcomes for one site. The purpose of this study was to investigate the effects of training and implementation of the StrategyTools Support System on targeted IEP goals for ninth and twelfth grade students with disabilities. A series of multiple probe designs was used to examine the effectiveness of tool usage across four ninth grade students and four twelfth grade students as well as to investigate tool usage across settings (training, academic, and transition) for each student. Additionally, this study investigated the feasibility of using the StrategyTools system in a variety of general education high school classrooms and the acceptability of the tools to students. Primary research questions were: What are the effects of tool use on individual educational outcomes for ninth and twelfth grade students with disabilities in a training setting? What are the effects of tool usage on individual educational outcomes across training, academic, and generalization settings for each student?
Method The primary research design was a multiple probe baseline across four ninth grade students and four twelfth grade students. This investigated the effect of training in selection and use of electronic performance support system tools on individually selected educational outcomes for 4
each student across one academic school year. For each participant, there was a secondary multiple probe design across settings to investigate the effects of tool use beyond the training setting and across academic and transition settings.
Participants and Setting Three special education teachers at a rural high school volunteered to participate in the project. Each special education teacher identified two to four students for participation in the study based on the following criteria: (a) receiving special education services for a high-incidence disability, (b) reported as having academic and/or behavioral deficits; (c) in ninth or twelfth grade at the time of the study; and (d) observed deficits in self-regulation, organizational skills, and/or use of learning strategies documented in their IEPs and re-evaluation reports conducted at eighth grade, per district convention. The study targeted ninth grade students to provide intensive supports for the transition to high school and twelfth grade students to focus on planning and preparation for life after high school. Once student participants and a content class where training would take place were identified, two additional classes were selected for each student—an academic class and a transition-related class where tools could be used to support selected target behaviors. The special education teachers agreed to collaborate with the students’ general education teachers from these two classes. Table 1 provides demographic information on the student participants.
Special education teachers. Two female and one male
high school special education teachers participated in this study. All three teachers had Masters degrees; both female teachers had two to three years of teaching experience, while the male teacher had eight years of experience. These teachers volunteered to participate in the study and agreed to collaborate with two general education teachers for each student, who would assist them in integrating the use of tools in their content classrooms. General education teachers. A total of 11 general education
teachers (six male, five female) volunteered to participate in the study. All general education teachers had Masters degrees or Masters equivalents and more than three years of teaching experience; four taught English, four taught science, and three taught social studies.
JSET 2013 Volume 28, Number 3
Journal of Special Education Technology
Table 1 Student Participants Participant
Age (Yr, Mo)
Grade
Race
Disability
IEP Goal Areas
JR
17, 7
12
Caucasian
LD
Organization and note-taking skills
JA
18, 8
12
AfricanAmerican
MR
Organization and note-taking skills and self-advocacy
RJ
17, 10
12
Caucasian
LD
Note-taking skills and reading comprehension
EC
17, 11
12
Caucasian
LD
Organization skills
AS
14, 7
9
Caucasian
LD
Writing skills
VP
15, 9
9
Caucasian
LD
Organization and note-taking skills
RM
14, 4
9
Caucasian
LD
Writing skills
MW
15, 4
9
Caucasian
LD
Writing skills
Dependent Variables and Measures For each student, the researcher and special education teacher collaborated to identify a target behavior based on the following criteria: (a) identified deficit area as
documented in student’s IEP; (b) behavior/skill deficit that could be addressed across a training, academic, and transition setting; (c) behavior/skill deficit that could be addressed by one or more tools from the StrategyTools
Table 2 Dependent Measures for Each Participant Across Settings Participant
Training Setting
Academic Setting
Transition Setting
JR
% correct and complete note taking (physics)
% correct and complete note taking (government)
% correct note taking (English)
JA
% correct and complete note taking (English)
% note taking (government)
% note taking (physics)
RJ
% correct and complete note taking (English)
% note taking (math & science)
% correct and complete comprehension skills (homeroom–newspaper)
EC
% correct and complete note taking (economics)
% note taking (science)
% organization skills/planner (homeroom)
AS
% writing prompt score (science) % writing prompt score (history)
% writing prompt score (English)
VP
% writing prompt score (English)
% correct and complete note taking (history)
% correct vocabulary (science)
RM
% writing prompt score (English)
% correct/ and complete note taking (science)
% writing prompt score (history)
MW
% writing prompt score (science) % writing prompt score (history)
% writing prompt score (English)
Note: Percentage calculated from rubrics with a range of 16–20 possible points.
JSET 2013 Volume 28, Number 3
5
Journal of Special Education Technology system. Table 2 shows the dependent measures for each student and each setting. Rubrics were developed to score permanent work samples created by students; three graduate research assistants were trained to independently score student work using the rubric. Training continued until research assistants independently and consistently reached 100% agreement scoring a series of work samples from students who were not participating in the study. Two research assistants scored each assignment independently in each phase of the study. Rubrics. The Pennsylvania State Standardized Assessment
writing prompt rubric was adapted for use in this project. Specifically, participant writing probes were scored across four categories (focus, content and development, organization, and style) on a four-point scale. In collaboration with the special and general education coteaching team, a notes organization rubric listing 8–10 criteria was developed to score organization and completion of notes across a variety of high school subjects (English, government, history, science, and math). Each criterion was scored using the following scale: (0) incorrect and incomplete; (1) correct or complete; (2) correct and complete resulting in a total of 16–20 points possible. Reading comprehension and science quizzes. One twelfth
grade participant identified for himself in his transition planning meeting the goal of improving reading comprehension. In particular, he wanted to be able to read and comprehend newspaper or current event articles published on the Internet. The researcher and research assistant selected a current events article each week that was between 200 and 250 words and that yielded a Flesch-Kincaid readability level between Grades 10 and 12 using the Microsoft Word readability tool. A series of five questions was developed for each article that asked a who, what, where, when question and one inferential question about the implications of the article. Each question was scored 0,1, or 2, with 0 = incorrect and incomplete response, 1 = either correct or complete response, and 2 = correct and complete response. A research assistant and the science teacher created a series of weekly 10-item vocabulary matching quizzes (terms and definitions) that addressed the vocabulary words in each article. Each quiz contained eight new terms and two previously taught terms.
6
Data Collector Training and Interscorer Agreement Data collectors were trained to score all permanent products independently using assignments from students not participating in the study until they reached agreement using the rubric (organization of notes, writing prompt, and reading comprehension answers). Raters discussed any disagreements until they reached 100% agreement. Following interscorer training, two research assistants— one blind to the nature and condition of the study— scored each artifact using the rubrics and criteria described previously. Interscorer agreement was calculated on a point-by-point basis (each score for each criterion) across conditions and yielded the following mean scores: vocabulary matching—100%; comprehension questions—97% (range 88 –100%); organization of notes—100%; and writing prompts—97% (range 88 –100%).
Independent Variable The StrategyTools components serving as the independent variable included StrategyTools and Strategy Coach™ for students. These tools are organized into meaningful groups in the metaphor of rooms of a school, as shown in Figure 2. The StrategyTools program contains 54 tools that are content-free templates to support success in six areas (Figure 2). The tools can be used independently following instruction in the skills. The Quick View displays each tool when highlighted by the user and explains its purpose. The tools provide support in getting organized, learning new information, demonstrating learning, working on projects, solving personal problems, and planning for the future. To use the tools, students may enter their own information using their own words and then print the completed tool for personal use. The tools have special features that make them appropriate for older, more mature students, including suitable graphics, advanced saving and editing features, options for expanding the fields using scroll bars, and printed tools that look like forms instead of screen dumps. When exiting the tool construction mode, the tools are automatically saved in a database for each student. The tool database allows students to call up previously created tools to reuse, edit, or use dynamically in self-monitoring, as well as to document progress over time.
JSET 2013 Volume 28, Number 3
Journal of Special Education Technology
Figure 2 Tools are organized into meaningful groups in the metaphor of rooms of a school.
The special education teachers were taught to use the StrategyTools software and learned about the research requirements in a one-day training session at the university campus. In this training, after reviewing the conceptual framework behind the tools and the steps for teaching strategies, the teachers explored the software to identify potential tools to use with participating students. A second 90-minute follow-up training for special education and participating general education teachers was conducted before implementation began in the nontraining settings.
JSET 2013 Volume 28, Number 3
Procedure Baseline condition. No experimental procedures were in
effect. Students were assigned academic work as usual and attended the pull-out resource room (training setting) for support from the special education teacher a couple of times a week. This consisted of a check-in with the teacher to review class notes, assignments, the opportunity to complete a quiz or writing prompt in a less distracting setting, or to meet with the teacher to discuss any concerns. Copies of student assignments (notes, writing prompts, quizzes) were collected by the research assistants, who also collected treatment fidelity data in the training settings. The ninth grade students were assigned writing prompts 7
Journal of Special Education Technology
Figure 3 Note taking tool.
once a week in content area classes to demonstrate what they had learned each week. Students completed these writing prompts in the resource room for the training setting and in the general education classroom for the other two settings. During the 40-minute class period, students were permitted to use their notes and were encouraged to make an outline before beginning writing. StrategyTools condition training setting. Students imple-
mented StrategyTools with the help of the teacher, who met individually with them during the check-in time. Depending on the skill/behavior selected for improvement, the teacher and student opened StrategyTools, selected a
8
tool, and discussed how to enter information, save, and use it. For example, JR’s goal was to improve her notetaking skills in physics, economics, and English classes. She and her teacher selected the note-taking tool to guide her note taking in physics. During the introduction to StrategyTools, the teacher guided her as she transferred notes from physics class into the note-taking tool and discussed with JR which key information should be highlighted in the right hand column. Support from the teacher was gradually faded, and JR began to transfer her class notes to the tool independently, reviewing with the teacher at the end of the
JSET 2013 Volume 28, Number 3
Journal of Special Education Technology support period. For writing prompts, the teacher and student typically selected the Chunker Tool to create a visual map for the prompts. Students wrote the main topic in the center node and then brainstormed subtopics and details for the remaining cells with the teacher. Students printed this out and then used it as a guide to complete the writing prompt. All tools can be viewed at the Strategy Coach website: www.strategytools.org. StrategyTools Support System condition: Academic and
transition settings. As stated earlier, the general education teachers in the academic and transition settings also agreed to participate in the study and to use the tools with the target student in their classroom. After completing a 90-minute after-school training/exploration session with StrategyTools, all general education teachers elected to use the tools with the entire class rather than just the target students as they indicated that all students would benefit from learning skills in note taking, organizing information, and study skills. The teacher used either the SmartBoard, if available, or a printed copy on the overhead projector to introduce and model the use of the StrategyTools to the class. Students then completed their own tools using paper copies of tools or the StrategyTools computer program in the mobile computer lab, when available. The teacher provided feedback on correct and complete use of the tool during initial implementation. The researcher and research assistant worked with the teaching teams to ensure that tools were implemented in the academic (and then the transition) setting when and only when two criteria were met in the training (and subsequently academic) setting. To progress, students had to independently complete the
tool at 80% accuracy for two consecutive uses and student performance on the dependent measure had to demonstrate a visible improvement over baseline performance.
Fidelity of Implementation Data were collected on the introduction and implementation of the independent variable (StrategyTools Support System) throughout the study for each student across all settings through the use of a treatment fidelity checklist. To indicate the integrity of StrategyTools system implementation, fidelity of implementation procedures consisted of recording the presence or absence of observable teacher and student behaviors, a review of created documents, and a review of user records to document time and completion of tools. Based on checklist completion, opportunities to use tools, and correct completion of the tools, each student was assigned a rating of high fidelity (greater than 80% on checklist, use of tools at least twice a week, and correct completion of the tools), partial fidelity (2 of 3 criteria), or poor fidelity (1 or fewer criteria) for each setting. These ratings were then summarized for each StrategyTools condition to provide a rating for each student in each setting (training, academic, and transition). The results are provided in Table 3. EC received the only poor rating in the first setting because of absences and difficulty setting up times for the student and teacher to meet. In addition, of all the students, EC exhibited the least interest in organizing materials, completing assignments, attending school, and using the tools. Implementation in the third setting was delayed for RM and AS and, as a result, both students had limited opportunities to use the tools in this
Table 3 Fidelity of Implementation Data by Student Name
Grade
Outcome
Fidelity 1
Fidelity 2
Fidelity 3
JR
12
Successful
High
High
High
JA
12
Successful
High
High
High
RJ
12
Successful
High
High
Partial
EC
12
Successful
Poor
Partial
Partial
AS
9
Successful
Partial
High
N/A
MW
9
Successful
Partial
High
High
VP
9
Successful
High
Partial
Partial
RM
9
Successful
High
Partial
N/A
JSET 2013 Volume 28, Number 3
9
Journal of Special Education Technology
Results
setting. Since a minimum of two observations of teacher/ student behavior was required along with completion of at least two tools, fidelity data could not be calculated for these two students in the final setting.
Effects of StrategyTools on Correct Completion of Note Taking for Twelfth Graders
Social Validity
Figure 4 presents the percentage correct and complete note taking by twelfth grade students in the training setting. Mean scores in baseline varied across students from 49.4% (RJ) to 71.7% (JR). After implementation of the StrategyTools system, mean scores increased fairly quickly to above 80% for each student. While scores remained variable for the three of four students whose performance increased the most (RJ, JA, and EC), the student with the highest baseline level of performance, JR, improved her performance to a mean score of 96% (SD 4.01) with less variability in scores.
At the end of the school year, teachers indicated their agreement (very much, somewhat, not at all) on a 20-item questionnaire related to the acceptability of intervention goals, procedures, and satisfaction with electronic performance support system outcomes. Sample questions from the teacher questionnaire included, “How important is it for students to learn how to create organizational, memory, and learning strategy guides?” “How did you like supporting students in creating their own tools?” and, “How much did the [support system] software help students learn different academic subjects at school?” Students completed a similar 15-item questionnaire.
Figure 4 Percentage correct and complete note taking during baseline and StrategyTool use by four twelfth grade students with learning/behavioral disabilities.
Percentage Correct Complete Note Taking
Baseline
10
ϵϬ ϳϬ ϱϬ ϯϬ ϭϬ ϵϬ ϳϬ ϱϬ ϯϬ ϭϬ ϵϬ ϳϬ ϱϬ ϯϬ ϭϬ ϵϬ ϳϬ ϱϬ ϯϬ ϭϬ
STSS
JR ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
ϰϭ
JA ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
ϰϭ
RJ ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
ϰϭ
EC ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
ϰϭ
JSET 2013 Volume 28, Number 3
Journal of Special Education Technology from her class notes, she implemented the tools with physics content in the training setting and then in economics and English for the second and third settings. Mean baseline performance for JR was variable in all three settings but was highest in the training setting. Implementation of StrategyTools in each setting was associated with an immediate increase in level and reduction in variability of performance (see Figure 6).
Effects of StrategyTools on Writing Prompt Scores or Correct Completion of Notes for Ninth Graders Figure 5 presents the percentage correct for ninth graders on writing prompts or correct completion of notes during baseline and intervention in the training setting. Prior to intervention, baseline performance ranged from 46% to 67% with considerable variability in performance for each student. After implementation of StrategyTools, student performance levels increased immediately to a mean between 85% and 99% for all four students. Variability in performance decreased noticeably during the StrategyTools condition.
JA also identified some type of postsecondary education as a goal and used the tools in English, economics, and physics to help organize her notes. She noted that she wanted to work on self-advocacy skills and asked to use the tools to help her prepare for interviews, asking for information and help related to living independently. Data were collected on the percentage of steps completed correctly on a variety of role-play situations. Baseline performance across all four settings/situations was relatively low and stable, with performance increasing immediately each time StrategyTools was implemented. Performance in the training setting was somewhat variable after implementation of the tools, but mean performance was still higher
Effects of StrategyTools Across Settings Figures 6 –9 show the effects of tool usage on individual educational outcomes across training, academic, and transition settings for each twelfth grade student. Since JR identified attending college in her transition planning meeting and was having difficulty organizing and studying
Figure 5 Percentage correct scores on writing prompts or note taking during baseline and StrategyTool use by four ninth grade students with learning/behavioral disabilities.
Percentage Correct Writing Prompt Score
ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ
Baseline
STSS
AS ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
MW ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
RM ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
VP ϭ
ϯ
ϱ
ϳ
JSET 2013 Volume 28, Number 3
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
11
Journal of Special Education Technology
Figure 6 Percentage correct performance for JR across training, academic, transition settings, and during extended phase. ^ĐŝĞŶĐĞ EŽƚĞ dĂŬŝŶŐ ĐŽŶŽŵŝĐƐ EŽƚĞ dĂŬŝŶŐ
ϵϬ ϳϬ ϱϬ ϯϬ ϭϬ
ŶŐůŝƐŚ EŽƚĞ dĂŬŝŶŐ
Baseline ϵϬ ϳϬ ϱϬ ϯϬ ϭϬ
ϵϬ ϳϬ ϱϬ ϯϬ ϭϬ
džƚĞŶĚĞĚ WŚĂƐĞ
Percent Correct Performance for JR
ϵϬ ϳϬ ϱϬ ϯϬ ϭϬ
STSS
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
Figure 7 Percentage correct performance for JA across training, academic, and transition settings. Baseline
ĐŽŶŽŵŝĐ EŽƚĞ dĂŬŝŶŐ
ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ
WŚLJƐŝĐƐ EŽƚĞ dĂŬŝŶŐ
dƌĂŝŶŝŶŐ ŶŐůŝƐŚ EŽƚĞ dĂŬŝŶŐ
Percent Correct Performance for JA 12
ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ
WĞƌĐĞŶƚ ^ƚĞƉƐ ŽŵƉůĞƚĞĚ
ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ
STSS
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
ϰϭ
ϰϯ
ϰϱ
ϰϳ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
ϰϭ
ϰϯ
ϰϱ
ϰϳ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
ϰϭ
ϰϯ
ϰϱ
ϰϳ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
ϰϭ
ϰϯ
ϰϱ
ϰϳ
JSET 2013 Volume 28, Number 3
Journal of Special Education Technology during intervention (90.3%) than during baseline (52.1%). A similar pattern was seen across the other legs of the multiple baseline design with less variable performance after intervention (see Figure 7).
with completion of her planner for the third leg of the baseline. Mean performance was variable in baseline across all three legs (50.5%, 53.3%, and 11.6% respectively) and improved after StrategyTools implementation to 83.6%, 91.1%, and 54.7% (see Figure 9).
RJ identified graduating from high school and working in the construction industry after graduation as goals. Correct completion of notes in English and his math/science classes was selected for the first two settings and a weekly reading comprehension quiz based on a newspaper article was selected for his third setting, based on his expressed desire to able to read and understand the daily newspaper. Mean performance improved from baseline to intervention in both the training setting and the second setting; however, performance on reading comprehension quizzes was very variable during baseline with little improvement after StrategyTools was implemented (see Figure 8).
Figures 10 –13 show the effects of tool usage on individual educational outcomes across training, academic, and generalization settings for each ninth grade student. Since target IEP goals for both AS and MW included improving written language skills and reading comprehension, they both implemented the tools with science content in the training setting, and then in history and English for the second and third settings; percentage scores on weekly writing probes were used to assess performance. Mean scores for AS improved from 45% during baseline to 85.6% during intervention in the training setting. A similar increase from baseline to intervention was seen in history (49.2% to 84.9%) and in English (53.3 to 81%); however, only one data point was collected during the intervention phase in the third setting, limiting conclusions (see Figure 10).
EC identified “getting through” school as her goal and did not note any plans for future education or employment. Her teachers indicated that she had most difficulty completing and organizing her notes, studying for tests, and completing and turning in homework. Correct completion of notes in economics and science were selected along
Figure 8 Percentage correct performance for RJ across training, academic, and transition settings. Baseline
STSS
ŶŐůŝƐŚ EŽƚĞ dĂŬŝŶŐ
ϴϬ ϲϬ ϰϬ ϮϬ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
DĂƚŚͬ^ĐŝĞŶĐĞ EŽƚĞ dĂŬŝŶŐ
ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ ϭϬϬ ƵƌƌĞŶƚ ǀĞŶƚƐ ZĞĂĚŝŶŐ ŽŵƉƌĞŚĞŶƐŝŽŶ
Percent Correct Performance for RJ
ϭϬϬ
ϴϬ ϲϬ ϰϬ ϮϬ
JSET 2013 Volume 28, Number 3
13
Journal of Special Education Technology
Figure 9 Percentage correct performance for EC across training, academic, and transition settings.
ĐŽŶŽŵŝĐƐ EŽƚĞ dĂŬŝŶŐ
Baseline ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ Ϭ
^ĐŝĞŶĐĞ EŽƚĞ dĂŬŝŶŐ
Percent Correct Performance for EC
ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ Ϭ
^ƚƵĚLJ^ŬŝůůƐ WůĂŶŶĞƌ
ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ Ϭ
STSS
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
ϰϭ
ϰϯ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
ϰϭ
ϰϯ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
ϰϭ
ϰϯ
Figure 10 Percentage correct performance for AS across training, academic, and transition settings. Baseline
STSS
^ĐŝĞŶĐĞ tƌŝƚŝŶŐ WƌŽŵƉƚƐ
ϴϬ ϲϬ ϰϬ ϮϬ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
,ŝƐƚŽƌLJ tƌŝƚŝŶŐ WƌŽŵƉƚƐ
ϴϬ ϲϬ ϰϬ ϮϬ ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ
14
ϭ
ϭϬϬ
ŶŐůŝƐŚ tƌŝƚŝŶŐ WƌŽŵƉƚƐ
Percent Correct Performance for AS
ϭϬϬ
JSET 2013 Volume 28, Number 3
Journal of Special Education Technology Similar patterns of improvement were seen for MW, whose mean performance during baseline improved from 55.3% to 87.7% during intervention. A slight increasing trend was observed in the second (mean 63.9%) and third settings (65.6%) during baseline; however, mean performance during intervention in both the second (88.8%) and third (90.6%) settings was higher and more consistent than during baseline (see Figure 11). Target IEP goal areas for RM also included improving writing skills and completing classwork and homework that had been identified as interfering with her academic progress in high school. For RM, English, science, and history were selected as classes where tool usage might help, and percentage scores on weekly writing probes were used to assess performance. RM’s performance during baseline was variable in all three classes, but showed improvement and less variability in each intervention condition. For example, mean performance improved from 67%, 66.4%, and 51.1% during baseline in English, science, and history respectively, to 99.4%, 95%, and 81.5% during intervention conditions (see Figure 12).
VP’s target behaviors were a little different and related to his lack of organizational skills and difficulty taking good notes and completing classwork and homework. Classes targeted for VP included English for the training setting, and history and science for second and third settings. For English and history, VP used tools to help organize his notes, and in science he used the tools to organize information and study for quizzes. Mean performance (% correct and complete note taking) in English and history showed an increase from baseline levels of 56.8% and 47.8% respectively to 98.3% and 78.8% during intervention. In his third setting, science, VP completed Computer Based Assessment (CBA) science quizzes across baseline and intervention conditions. Mean performance during baseline increased from 41.2% to 93.3% during intervention (see Figure 13).
Social Validity Thirteen participating teachers completed the social validity rating forms. Overall, all teachers rated the goals of the intervention as very important and the importance of having support tools available to students as somewhat or very important. All teachers rated the acceptability of the
Figure 11 Percentage correct performance for MW across training, academic, and transition settings.
,ŝƐƚŽƌLJtƌŝƚŝŶŐ WƌŽŵƉƚ
^ĐŝĞŶĐĞtƌŝƚŝŶŐ WƌŽŵƉƚ
ŶŐůŝƐŚtƌŝƚŝŶŐ WƌŽŵƉƚ
Percent Correct Performance for MW
Baseline
STSS
ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ
JSET 2013 Volume 28, Number 3
15
Journal of Special Education Technology
Figure 12 Percentage correct performance for RM across training, academic, and transition settings.
Baseline
STSS
ŶŐůŝƐŚ tƌŝƚŝŶŐ WƌŽŵƉƚƐ
ϴϬ ϲϬ ϰϬ ϮϬ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
ϰϭ
ϰϯ
ϰϱ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
ϰϭ
ϰϯ
ϰϱ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϯϱ
ϯϳ
ϯϵ
ϰϭ
ϰϯ
ϰϱ
^ĐŝĞŶĐĞ tƌŝƚŝŶŐ WƌŽŵƉƚƐ
ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ ϭϬϬ ,ŝƐƚŽƌLJ tƌŝƚŝŶŐ WƌŽŵƉƚƐ
Percent Correct Performance for RM
ϭϬϬ
ϴϬ ϲϬ ϰϬ ϮϬ
Figure 13 Percentage correct performance for VP across training, academic, and transition settings.
16
ŶŐůŝƐŚ EŽƚĞ dĂŬŝŶŐ ,ŝƐƚŽƌLJ EŽƚĞ dĂŬŝŶŐ
Baseline ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ Ϭ ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ Ϭ
^ĐŝĞŶĐĞ YƵŝnj
Percent Correct Performance for VP
ϭϬϬ ϴϬ ϲϬ ϰϬ ϮϬ Ϭ
STSS
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
ϭ
ϯ
ϱ
ϳ
ϵ
ϭϭ
ϭϯ
ϭϱ
ϭϳ
ϭϵ
Ϯϭ
Ϯϯ
Ϯϱ
Ϯϳ
Ϯϵ
ϯϭ
ϯϯ
JSET 2013 Volume 28, Number 3
Journal of Special Education Technology tools and tool supports as somewhat to very acceptable, with only one teacher reporting that one aspect of the Strategy Coach website, “tips for using tools,” was not helpful. In terms of satisfaction with student outcomes, 12 of 13 teachers reported that they were somewhat or very satisfied with the outcomes. One teacher reported that the tools did not impact academic or social behavior; however, all teachers reported that they would use the tools again and that tools would be good to use with students in other classes. Students reported similar findings. All students reported that teachers should use the tools with other students and that they liked the different components of the tools including sample tools, the tool print outs, and practice activities on the Strategy Coach website. In outcomes, students responded that tools helped somewhat to very much in terms of being more successful in school and 75% reported that the tools also helped outside of school; two students reported that the tools did not help outside of school.
Discussion Overall, this study demonstrated an improvement in target behaviors when the intervention was introduced in the training setting for ninth and twelfth grade students with high-incidence disabilities. In addition, the multiple probe design across classes for each student showed some improvement in the target behavior for each student when the intervention was implemented across settings. The fact that improvements in performance were not seen until the tools were implemented in these additional classes suggests that these students did not generalize use of the tools without training and implementation in at least two to three additional settings. The majority of students and teachers reported that the tools were acceptable and useful and that they would use them again. A further indication of teachers’ satisfaction with the tools was their request at the end of the project period for schoolwide training in tool use. When asked for recommendations about using tools, three teachers suggested that these tools should be used schoolwide and that usage should be integrated by departments into content areas. Teachers commented that extensive use of the tools across content areas would assist students in transferring the skills supported by the tools. A summary of these comments along with copies of the single subject graphs were shared with the building JSET 2013 Volume 28, Number 3
principal, who was in favor of giving teachers more options to help make their students academically successful. Central office administrators also supported schoolwide use and asked the first author to provide a brief overview of StrategyTools to all social studies, English, science, and special education teachers at one of the inservice days at the opening of the following school year. The positive results seen in this study should be interpreted with caution. Characteristics of the school site and level of researcher support may have influenced the outcomes achieved. McKenna and Walpole (2010) noted in their review of lessons from Reading First that, “Schools are exceedingly complex systems with intricate combinations of individual characteristics that are easily traced to achievement differences” (p. 479). Differences at the individual level (student characteristics and teacher knowledge and dispositions) as well as the organizational level (resource availability and organizational climate), among others, have been identified as possible mediating variables on intervention effectiveness (Lochman, 2003; Ringeisen, Henderson, & Hoagwood, 2003). Thus, context matters. For example, the site reported in this article is a rural high school serving a majority of students who come from low socioeconomic backgrounds. Generally, rural school districts face daunting and stifling roadblocks that include a limited tax base for needed revenues, a need to deliver services over a wide geographic area, inadequate facilities, limited support services, high transportation costs, teacher shortages, and a lack of access to effective professional development (Helge, 1992; Howley, 1991; Knapczyk, Rodes, & Brush, 1994; Mitchem, Kossar, & Ludlow, 2006). This school site had to spend a significant amount of its budget transporting students from a fairly wide geographic area to school, thus leaving limited resources for professional development or material resources. In addition, many teachers at this school were unfamiliar with the idea of cognitive strategy instruction and graphic organizers and had very limited access to current technologies. The positive results seen at this site may be related in part to the fact that this grant-funded research study provided the school access to previously unavailable resources, materials, and support. Perhaps the teachers and students at this school simply had more room for improvement and the addition of resources and researcher support was sufficient to produce change. The researcher met weekly with teachers to discuss program goals, feedback on implementation and outcomes, and consultation and support to teachers.
17
Journal of Special Education Technology An additional factor that may have influenced results was the presence of a veteran ninth grade teacher at the school who had prior experience and success using StrategyTools. Future research should examine site-based variables as well as the effect of an on-site advocate on the consistent and enthusiastic implementation of an innovation/practice. Specific questions that have been identified by many others in the field include a focus on how and why, for whom, and under what conditions do the tools work? How much support do you have to provide to change behavior? And, how much support should you provide? According to John Easton, Institute for Education Sciences Director at the U.S. Department of Education, as a field, we should begin with school needs and use descriptive data to explicate the current situation, seek capacity building in schools, communicate and collaborate with schools, and design studies with practitioners.
Limitations This study attempted to control for threats to internal validity by using a series of multiple probe designs across ninth and twelfth grade students and then across classes for each individual student. The logistics of this design and reluctant participation of one general education teacher (the English teacher for AS) delayed implementation for a couple of students and provided only one data point in the final setting for one of the students. It is possible that maturation may have contributed to the improvement seen in some students’ performance. Other limitations of this research relate to the fact that the electronic performance support system was a package intervention in which teachers modified their instruction to incorporate the use of tools and were provided feedback by the researcher in terms of student outcomes. It is impossible to separate the impact of this and other aspects of the intervention from the support system itself on student performance. Overall, the willingness of the majority of the teachers to implement the tools with students, help students discriminate between essential and nonessential information, and provide corrective feedback appear to be related to successful outcomes for students. It also is not clear from this study whether use of the tools improved some teachers’ organization and presentation of notes rather than improved student note taking and organization of notes. For example, the general education
18
economics teacher was himself forced to review his notes and identify key points prior to introducing the notetaking tool and other tools such as the study guide. During baseline, it was challenging for the research assistants and researcher to identify key points from his presentation. Regardless, StrategyTools appears to have served as the catalyst for teacher behavior changes and resulting student gains.
Implications for Practice Most secondary schools today are including students with disabilities as much as possible in the general education classroom, with special education services provided by a consulting teacher, coteacher, or teaching assistant. StrategyTools work well in this situation, since they support all learners and teachers can easily use them with all students (Fitzgerald, Mitchem, Miller, & Hollingsead, 2010). The teacher can model the use of a tool while introducing new content and use a think aloud process to model, for example, how to identify the main topic and supporting details. Access to computers is helpful so that students can complete their own tools; however, tools also can be printed and completed in pencil if access to technology is problematic. Use of StrategyTools is an effective way to provide supplemental supports to students who may have difficulty taking and organizing notes, studying for tests, and identifying critical content or main ideas. The teacher can introduce the tools with students in small groups or one on one and show students how these tools can be utilized in general education classrooms across content areas. Providing lots of examples and corrective feedback on tool completion is critical during the acquisition phase so that students learn how to use the strategies correctly prior to independent use. We found that students needed to score at least 80% complete and correct on the tool rubric prior to independent use of the tools. Implementing this step is a critical component in identifying those students who may have misconceptions about the content or who need additional support in completing and using the tools.
Summary The use of the electronic performance support system in this study was found to have positive effects and be a positive experience for both students and teachers. Students used the tools successfully to organize and write about
JSET 2013 Volume 28, Number 3
Journal of Special Education Technology academic content across a number of different classrooms. Tools were used with individual students as well as classwide, and the majority of students and teachers indicated that they would use them again. Future research should investigate specific site-based variables that may be associated with the positive outcomes achieved in this study.
References Anderson-Inman, L. & Horney, M. (1997). Computer-based concept mapping: Enhancing literacy with tools for visual thinking. Journal of Adolescent and Adult Literacy, 40, 302–306. Anderson-Inman, L., & Horney, M. A. (2007). Supported eText: Assistive technology through text transformations. Reading Research Quarterly, 42, 153–160. Anderson-Inman, L., Knox-Quinn, C., & Szymanski, M. (1999). Computer-supported studying: Stories of successful transition to postsecondary education. Career Development for Exceptional Individuals, 22, 185–212. Banerjee, M. (2010). Technology trends and transition for students with disabilities. In S. F. Shaw, J. W. Madaus, & L. L. Dukes (Eds.), Preparing students with disabilities for college success: A practical guide for transition planning (pp. 115–136). Baltimore, MD: Brookes. Blankenship, T. L., Ayres, K. M., & Langone, J. (2005). Effects of computer-based cognitive mapping on reading comprehension for students with emotional behavior disorders. Journal of Special Education Technology, 20(2), 15–23. Boon, R. T., Burke, M. D., Fore, C., III, & Spencer, V. G. (2006). The impact of cognitive organizers and technology-based practices on student success in secondary social studies classrooms. Journal of Special Education Technology, 21(1), 5–11. Boon, R. T., Fore, C., III, Blankenship, T., & Chalk, J. (2007). Technology-based practices in social studies instruction for students with high-incidence disabilities: A review of the literature. Journal of Special Education Technology, 22(4), 41–56. Bryant, D. P., Bryant, B. R., & Raskind, M. H. (1998). Using assistive technology to enhance the skills of students with learning disabilities. Intervention in School and Clinic, 34, 53–58. Deshler, D., Schumaker, J., Lenz, K., Bulgren, J., Hock, M., Knight, J., et al. (2001). Ensuring content-area learning by secondary students with learning disabilities. Learning Disabilities Research & Practice, 16, 96–109. Edyburn, D. (2000). Assistive technology and students with mild disabilities. Focus on Exceptional Children, 32(9), 1–23. Englert, C. S., Zhao, Y., Dunsmore, K., Collings, N. Y., & Wolbers, K. (2007). Scaffolding the writing of students with disabilities through procedural facilitation: Using an Internet-based technology to improve performance. Learning Disabilities Quarterly, 30, 9–29.
JSET 2013 Volume 28, Number 3
Fitzgerald, G. (2005). Using technologies to meet the unique needs of students with emotional/ behavioral disorders: Findings and directions. In D. Edyburn, K. Higgins, & R. Boone (Eds.), The Handbook of special education technology research and practice (pp. 335–354). Whitefish Bay, WI: Knowledge by Design. Fitzgerald, G., & Koury, K. (2004–2005). The StrategyTools Support System. United States Department of Education. Project #H327A030044. Fitzgerald, G., Mitchem, K., Koury, K., Miller, K., & Hollingsead, C. (2008–2010). Electronic Performance Support System (EPSS) Tools for Students with Mild Disabilities. U. S. Department of Education, Institute of Education Sciences, Project #R324B070176. Fitzgerald, G., Mitchem, K., Miller, K., & Hollingsead, C. (2010). Electronic performance support tools for adolescents: An implementation manual for StrategyTools. Project materials funded by the U. S. Department of Education, Institute of Education Sciences, Project #R324B070176). Columbia, MO: University of Missouri. Gajria, M., Jitendra, A. K., Sood, S., & Sacks, G. (2007). Improving comprehension of expository text in students with LD: A research synthesis. Journal of Learning Disabilities, 40, 210–225. Gersten, R. (1998). Recent advances in instructional research for students with learning disabilities. Learning Disabilities Research & Practice, 13, 162–170. Gustafson, K. (2000). Designing technology-based performance support. Educational Technology, 40(1), 38–44. Hartley, K. (2001). Learning strategies and hypermedia instruction. Journal of Educational Multimedia and Hypermedia, 10, 285–395. Helge, D. (1992). Solving special education reform problems in rural areas. Preventing School Failure, 36(2), 11–16. Higgins, K., & Boone, R. (1990a). Hypertext: A new vehicle for computer use in reading instruction. Intervention in School and Clinic, 26, 26–31. Higgins, K., & Boone, R. (1990b). Hypertext computer study guides and the social studies achievement of students with learning disabilities, remedial students, and regular education students. Journal of Learning Disabilities, 23, 629–540. Higgins, K., Boone, R., & Lovitt, T. C. (2002). Adapting challenging textbooks to improve content area learning. In M. Shinn, H. Walker, & G. Stoner (Eds.), Interventions for academic and behavior problems II: Preventive and remedial approaches (pp. 755–790). Bethesda, MD: National Association of School Psychologists. Howley, C. (1991). The rural dilemma as part of the rural dilemma: Rural education and economics. In A. J. DeYoung (Ed.), Rural education: Issues and Practice. New York, NY: Garland. Knapczyk, D., Rodes, P., & Brush, T. (1994). Improving staff development in rural communities using distance education and communication technology. Rural Special Education Quarterly 13(2), 19–24. Laffey, J. (1995). Dynamism in electronic performance support systems. Performance Improvement Quarterly, 8, 31–46.
19
Journal of Special Education Technology Lewis, R. (2005). Classroom technology for students with learning disabilities. In D. Edyburn, K. Higgins, & R. Boone (Eds.), The handbook of special education technology research and practice, 325–334. Lochman, J. E. (2003). Commentary: School contextual influences on the dissemination of interventions. School Psychology Review, 32, 174–177. Mastropieri, M., Scruggs, T., & Graetz, J. (2003). Reading comprehension instruction for secondary students: Challenges for struggling students and teachers. Learning Disability Quarterly, 26, 103–116.
Ringeisen, H., Henderson, K., & Hoagwood, K. (2003). Context matters: Schools and the research-to-practice gap in children’s mental health. School Psychology Review, 32, 153–168. Schaff, J., Bannan-Ritland, B., Behrmann, M., & Ok, S. (2005). Electronic performance support systems. In D. Edyburn, K. Higgins, & R. Boone (Eds.), Handbook of special education technology research and practice (pp. 493–506). Whitefish Bay, WI: Knowledge by Design. Shaw, S. F. (2009). Transition to postsecondary education. Focus on Exceptional Children, 42(1), 1–16.
Mazzotti, V. L., Wood, C. L., Test, D. W., & Fowler, C. H. (2012). Effects of computer-assisted instruction on students’ knowledge of the self-determined learning model of instruction and disruptive behavior. Journal of Special Education, 45, 216–226.
Stetter, M., & Hughes, M. (2010). Computer-assisted instruction to enhance the reading comprehension of struggling readers: A review of the literature. Journal of Special Education Technology, 24(4), 1–16.
McKenna, M. C., & Walpole, S. (2010). Planning and evaluating change at scale: Lessons from Reading First. Educational Researcher, 39, 478–483.
Test, D. W., Mazzotti, V. L., Mustian, A. L., Fowler, C. H., Kortering, L., & Kohler, P. (2009). Evidence-based transition predictors for improving post school outcomes for students with disabilities. Career Development for Exceptional Individuals, 32,180–181.
McKenney, S., & van den Akker, J. (2005) Computer-based support for curriculum designers: A case of developmental research, Educational Technology Research and Development, 53(2), 41–66. Mitchem, K., Fitzgerald, G., Koury, K., Cepel, C., & Boonseng, T. (2009). Electronic performance support system tools (EPSS) to enhance success in school for secondary students with special needs. In Leo Tan Wee Hin & R. Subranamiam (Eds.), Handbook of Research on New Media Literacy at the K–12 Level: Issues and Challenge (pp. 529–550). Hershey, PA: Information Science Reference.
Wagner, M., Newman, L., Cameto, R., Levine, P., & Marder, C. (2007). Perceptions and expectations of youth with disabilities. A special topic report of findings from the National Longitudinal Transition Study-2 (NLTS2) (NCSER 2007-3006). Menlo Park, CA: SRI International. Williams, J. P. (2005). Instruction in reading comprehension for primary-grade students: A focus on text structure. Journal of Special Education, 39, 6–18.
Mitchem, K., Kight, J., Fitzgerald, G., & Koury, K. (2007). Electronic performance support systems: An assistive technology for secondary students with mild disabilities. Journal of Special Education Technology, 22(2), 1–14.
Xin, Y. P., & Jitendra, A. K. (1999). The effects of instruction in solving mathematical word problems for students with learning problems: A meta-analysis. The Journal of Special Education, 32, 207–222.
Mitchem, K. J., Kossar, K., & Ludlow, B. L. (2006). Finite resources, increasing demands: Rural children left behind? Educators speak out on issues facing rural special education. Rural Special Education Quarterly, 25(3), 13–23.
Author Notes:
National Center on Secondary Education and Transition. (2003). A national leadership summit on improving results for youth: State priorities and needs for assistance. Retrieved from http://www.ncset. org/summit03/NCSETSummit03findings.pdf
Newman, L., Wagner, M., Cameto, R., & Knokey, A.M. (2009). The post-high school outcomes of youth with disabilities up to 4 years after high school. A report from the National Longitudinal Transition Study-2 (NLTS2) (NCSER 2009-3017). Menlo Park, CA: SRI International.
Katherine J. Mitchem is a professor and Edith L. Trees endowed chair in the Early, Middle, and Special Education Department, California University of Pennsylvania. Gail Fitzgerald is a professor in the School of Information Science & Learning Technologies, University of Missouri. Kevin Miller is a professor in the Exceptional Education Department, Buffalo (NY) State College. Candice Hollingsead is an associate professor in the School of Education, Northern State University, Aberdeen, South Dakota. This research was funded in part by the U.S. Department of Education Institute for Education Sciences Project#: R324B070176.
Okolo, C. M. (2000). Technology for individuals with mild disabilities. In J. D. Lindsey (Ed.), Technology and exceptional individuals, (3rd ed. pp. 243–301). Austin, TX: Pro-Ed.
Correspondence concerning this article should be addressed to Kate Mitchem, 250 University Avenue, Box 25, California University of Pennsylvania, California, PA 15419. Email:
Parker, D. R., & Banerjee, M. (2007). Leveling the digital playing field: Assessing the learning technology needs of college bound students with LD and/or ADHD. Assessment for Effective Intervention, 33(1), 5–14.
[email protected]
Pierson, M. R., Carter, E. W., Lane, K. L., & Glaeser, B. C. (2008). Factors influencing the self-determination of transition-age youth with high-incidence disabilities. Career Development for Exceptional Individuals, 31, 115–125.
20
JSET 2013 Volume 28, Number 3
