Based Instructional Practices for Students with

Learning Disabilities Research & Practice, 30(2), 91–101  C 2015 The Division for Learning Disabilities of the Council for Exceptional Children

Using the 6S Pyramid to Identify Research-Based Instructional Practices for Students with Learning Disabilities Tanya Santangelo Arcadia University

Leslie C. Novosel and Bryan G. Cook University of Hawai´i at M¯anoa

Meredith Gapsis Arcadia University To optimize students’ learning outcomes, educators are increasingly expected to use instructional practices shown to be effective by credible research. To help make this possible, organizations and scholars are producing resources that summarize research related to various instructional practices. However, as the collection of resources grows in size and complexity, it can be difficult and frustrating for practitioners to locate and utilize this information. In this article, we describe the 6S Pyramid (DiCenso, Bayley, & Haynes, 2009), a framework for efficiently and accurately prioritizing different types of research evidence. We also identify and briefly summarize Internet resources corresponding with each level of the Pyramid that can be used to identify research-based practices for students with learning disabilities. To illustrate how the 6S Pyramid can be used in school and classroom settings, we begin and end the article with vignettes featuring a pair of co-teachers seeking to improve instruction for their students who struggle with reading.

MS. MARKLOFF AND MS. HUTCHINSON: ACT I Ms. Markloff is a second-year, general education teacher. This year, she is teaching a diverse, inclusive class of 24 fourth graders, six of whom have learning disabilities (LD), and struggle with reading. An additional four students in the class have not been identified with LD, but consistently score below grade-level expectations on the state reading assessment. During the language arts period, Ms. Hutchinson, a special education teacher, provides push-in support for the students who struggle with reading. The progress monitoring data Ms. Markloff and Ms. Hutchinson gathered during the first 2 months of school suggest that their core literacy instruction is effective for most of the class. However, the teachers are very concerned because several students, including those with LD, are not making adequate gains. As the teachers begin to brainstorm how to improve reading instruction for the students who are struggling, they recall a workshop they attended last year, where the presenter encouraged them to use the Internet as a tool to identify effective instructional practices. Ms. Markloff and Ms. Hutchinson cannot recall the specific Web sites the presenter recommended, so they reach out and ask several colleagues, as well as a couple of their former college professors, for suggestions. They set aside an hour and a Requests for reprints should be sent to Tanya Santangelo, Arcadia University. Electronic inquiries should be sent to [email protected].

half after school the following Friday to check out their leads. When Friday afternoon arrives, they compile the recommendations they received into a list that includes more than 10 Web sites, such as What Works Clearinghouse and National Center on Intensive Intervention (see Table 1). As Ms. Markloff and Ms. Hutchinson navigate the Web sites, they become increasingly frustrated and overwhelmed with the magnitude, variability, and complexity of information across the different websites they visit. Many sites have far more information than they expected—much of which they find to be confusing and not directly related to their students. They also discover a lack of consistency in the types of resources available. For example, some sites offer narrative summaries of the research related to a particular topic area, others rate the effectiveness of particular practices and programs using specific criteria, and one site evaluates particular elements of individual studies. When the teachers finish exploring the last Web site on their list, they realize nearly two hours has passed; yet they are still unsure of how to improve their instruction to help their students who struggle with reading. Disappointed and frustrated after their experience, Ms. Markloff and Ms. Hutchinson find themselves questioning whether Internet resources really DO offer useful information for teachers. We suspect that many teachers find themselves in situations similar to Ms. Markloff and Ms. Hutchinson. That is, they want to find the most effective practices for improving the outcomes of their struggling learners, and

92

SANTANGELO ET AL.: 6S PYRAMID

TABLE 1 Summary of Internet Resources Levels of Information Offereda Name of Internet Resource

Web Address

Best Evidence Encyclopedia Center on Instruction Current Practice Alerts National Center on Intensive Intervention National Dissemination Center for Children with Disabilities National Secondary Transition Technical Assistance Center What Works Clearinghouse

www.bestevidence.org www.centeroninstruction.org www.teachingld.org/alerts www.intensiveintervention.org www.parentcenterhub.org www.nsttac.org www.ies.ed.gov/ncee/wwc

a Refers

2

3

4

X

X X X

5

X X X

X X

X

to the levels of evidence in the 6S Pyramid (DiCenso et al., 2009), as shown in Figure 1.

they like the idea of using practices shown to be effective by credible research evidence. However, despite the growing number of resources available on the Internet related to instructional practices supported by scientific research (i.e., research-based practices (RBPs); further defined in the next section), information from the Internet can be misleading, overwhelming, and difficult to translate into practice. In this article, we describe a model for guiding special educators through the maze of Internet resources related to RBPs for students with LD. Specifically, after providing a brief overview of RBPs and their importance for special educators, we discuss the 6S Pyramid (DiCenso, Bayley, & Haynes, 2009) and how it can help practitioners efficiently and accurately prioritize RBPs for students with LD that can be found on a number of existing Internet resources. We conclude with a description of Ms. Markloff and Ms. Hutchinson using the 6S Pyramid and Internet resources described in this article to identify RBPs for their students who struggle with reading.

RBPs Although no instructional program or technique works for every student, some practices are generally more effective than others for improving student outcomes (Cook, Smith, & Tankersley, 2012; Hattie, 2009). Because scientific research has an unparalleled track record of determining which practices work best (Odom et al., 2005; Slavin, 2002, 2008), recent educational reforms and laws emphasize identifying and implementing practices shown by sound, scientific research to be effective (Yell & Rozalski, 2013). In this article, we use the term RBPs to refer generally to instructional strategies, techniques, and programs shown by credible scientific research to be effective. We distinguish RBPs from evidencebased practices which, according to Cook and Cook (2013), are a subset of RBPs with research bases that meet rigorous standards related to research design, quality, and quality (e.g., Gersten et al., 2005; Horner et al., 2005). Thus, in this article we use RBPs synonymously with “empirically validated practices” to refer to practices supported as effective by scientific research, but with research bases that do not necessarily meet the rigorous standards required for evidence-based practices.

Although RBPs can help improve outcomes for all learners, they are especially important for students with LD, who require highly effective instruction to reach their goals and potential (e.g., Swanson, Harris, & Graham, 2013). The research base on interventions for students with LD, however, is large, varied, and technical (Cook et al., 2012). Miech, Nave, and Mosteller (2005) referred to this as the 20,000 article problem because approximately 20,000 education-related articles are published annually and, clearly, educators do not have time to read and make sense of that much information. Moreover, even if time were not a barrier, many educators do not have the advanced training necessary to critically analyze the research base. In response to this problem, educational organizations and scholars have begun to appraise and synthesize research findings to help practitioners and other stakeholders readily identify RBPs (Odom et al., 2005; Slavin, 2002, 2008). However, like the research bases they intend to simplify, the growing array of resources has become sizeable, diverse, and complicated. As Ms. Markloff and Ms. Hutchinson discovered, rather than streamlining the process of determining what works, these resources may unintentionally confound the problem by adding another layer of information through which practitioners must sift.

IDENTIFYING INTERNET RESOURCES FOR RBPs We used the following process to locate and analyze the resources described in this article. First, we conducted a search of Internet resources that we and our colleagues identified as potentially including information about RBPs for students with LD to determine which ones offer free and publicly available information corresponding with different levels of the 6S Pyramid. Specifically, we focused on practices targeting academic (e.g., reading, math, writing, content area knowledge) and other college- and career-readiness outcomes (e.g., functional life and transition skills) for K-12 students. Thus, assessment (e.g., curriculum-based measurement) and behavioral interventions (e.g., schoolwide positive behavioral support) were not included. We also restricted our focus to resources that synthesize and appraise evidence from studies using research designs generally recognized as establishing cause and effect

LEARNING DISABILITIES RESEARCH

relationships: true random experiments (e.g., randomized control trials), quasi-experiments, and single-case studies (Cook, Tankersley, Cook, & Landrum, 2008; Odom et al., 2005). We originally planned to only include information derived from research conducted exclusively with students with LD. However, given the scarcity of such resources, we decided to also include information derived from research with samples that include students with LD along with other groups, such as students determined to be at risk of academic failure. In subsequent sections of this article, we note when the participants involved in research supporting a practice are not just students with LD. Through our initial search, we identified seven relevant Internet resources that provide information on RBPs for students with LD that correspond with one or more levels of the 6S Pyramid (see Table 1). Next, two authors independently examined all of the RBPrelated resources available at the seven web sites and documented critical characteristics of each relevant publication found (e.g., title, level/type of evidence, percentage of students with LD in the sample, targeted outcome area, primary findings). The data collected by the two authors were then compared and the few differences found were resolved through discussion (inter-rater reliability, calculated as total% agreement, >99 percent). Our review and analysis of the Web sites took place during the fall and spring of 2013 and was updated in September of 2014. THE 6S PYRAMID The 6S Pyramid (DiCenso et al., 2009; see Figure 1) is a framework for categorizing and prioritizing resources that appraise and synthesize research findings. The 6S Pyramid was developed in the field of medicine to help doctors and other professionals make decisions based on the best available research evidence (e.g., Robeson, Dobbins, DeCorby, & Tirilis, 2010). The Pyramid represents a hierarchy of evidence in which higher levels provide increasingly accurate and efficient types of information. Therefore, when searching for RBPs using the 6S Pyramid, practitioners should start at the highest possible level and only work down to lower levels as necessary. For some well-researched practices, the highest level of available evidence might be a summary of the research (Level 2). In contrast, the highest level of evidence for a practice that is not yet well-researched might be a single study (Level 6). In the following sections, we describe each level in the 6S Pyramid and summarize Internet resources currently available to educators. Because each level of the Pyramid builds on information from lower levels, we begin our description at the bottom and move up. Level 6: Studies The lowest level in the 6S Pyramid is studies. Although individual studies are the building blocks of research bases, a single study, by itself, offers relatively little value to practitioners looking to identify RBPs (DiCenso et al., 2009). No study is perfect or represents “the truth.” Indeed, many studies contain bias and may report misleading findings (Cook, 2014).

93

As such, examining the effectiveness of a practice should involve consideration of findings from multiple, high-quality, experimental studies. However, searching for, critically analyzing, and synthesizing the findings of multiple studies are incredibly time-consuming tasks and, thus, are not practical for busy educators to do on their own (Haynes, 2001). Given the availability of more efficient and accurate information at higher levels of the Pyramid, examining original studies is not the preferred method to determine the effectiveness of a practice and should only be used when higher-level resources are unavailable. Level 6 Resources Individual studies examining the impact of instructional practices for students with LD were not offered at any of the Web resources we examined. However, they are readily available in professional journals, such as Learning Disabilities Research & Practice. Google Scholar (www.scholar.google.com) and ERIC (www.eric.ed.gov) are two search engines educators can use to search for individual studies, and they are also available through many public and university-based libraries. Level 5: Synopses of Studies Synopses of studies are concise, accessible descriptions of individual studies written by experts (DiCenso et al., 2009). Synopses of studies are beneficial to practitioners because they summarize the critical elements of the original research into a much shorter format, typically using nontechnical language. Most synopses of studies also include an evaluative commentary that highlights important strengths and limitations of the study. Therefore, reading a synopsis of a study is more accurate and efficient than reading the original study. Nonetheless, study synopses are limited because they only provide evidence from one study. Confidence in the effectiveness of a practice is greatly increased when multiple studies are considered and the results are consistent (Moonesinghe, Khoury, & Janssens, 2007). Because of the important limitations associated with considering evidence one study at a time, all higher levels of the 6S Pyramid consider findings across multiple studies. Level 5 Resources We identified two Internet resources that currently offer Level 5 information related to instructional practices for students with LD: What Works Clearinghouse (WWC) and National Center on Intensive Intervention. The study synopses produced by the WWC are called “Single Study Reviews” and can be accessed by clicking the “Single Study Reviews” link located on WWC’s home page. Each Single Study Review includes a concise summary of the targeted study’s features and findings, as well as a rating of research quality (i.e., meets WWC evidence standards without reservations, meets WWC evidence standards with reservations, or does not meet WWC evidence standards). Although the WWC had produced more

94

SANTANGELO ET AL.: 6S PYRAMID

FIGURE 1 The 6S Pyramid (DiCenso, Bayley, & Haynes, 2009).

than 80 Single Study Reviews at the time of our review, only one was a synopses of a study that included students with LD (i.e., Same-Language-Subtitling [SLS]: Using Subtitled Music Video for Reading Growth). Because WWC releases several new Single Study Reviews each month, we anticipate additional LD-related Level 5 resources will be available from WWC in the future. The study synopses produced by National Center on Intensive Intervention are organized into a “Tools Chart” that can be accessed using the “Tools Charts” dropdown menu at the top of the home page or directly at the URL: www.intensiveintervention.org/chart/instructionalintervention-tools. Each synopsis includes: (1) a summary of the intervention (e.g., targeted students, cost); (2) an evaluation and description of key elements related to research quality (e.g., design, fidelity); (3) details related to implementation (e.g., group size, duration, training); (4) a description of the findings (e.g., effect size for all participants as well as targeted subgroups, such as students with LD); and (5) related research (i.e., number of additional studies evaluating the practice and whether they have been reviewed by WWC). At the time of our review, 59 study synopses were available from National Center on Intensive Intervention. Consistent with the organization’s mission, all of the synopsized studies focused on students with academic difficulties and 19 specifically included students with LD in the sample (16 studies targeted literacy and 3 focused on math). Practitioners should prioritize practices that are supported as effective by multiple, high-quality studies for a target population (e.g., elementary students with LD). For example, in the “Tools Chart,” we found three studies evaluating Stepping Stones to Literacy, all of which involved kindergarten participants, received positive

ratings for study quality, and showed positive effects on reading and prereading outcomes. Thus, educators can have considerable confidence in the effectiveness of this practice for this population. Level 4: Syntheses Syntheses draw conclusions about the effectiveness of a practice by systematically considering findings across multiple experimental studies. A research synthesis can, then, be thought of as a “study of studies.” Because it is unclear whether the results from any individual study are accurate or generalizable until they are (or are not) confirmed by other studies, replication is one of the foundational tenets of scientific inquiry (Jasny, Chin, Chong, & Vignieri, 2011). Since syntheses consider the findings of multiple studies, they represent a higher level of evidence than individual studies. Reading a synthesis is also more efficient than reading all of the individual studies included in the review. Because of their advantages, syntheses are being conducted with increasing frequency to determine what works in education, including for students with LD (e.g., Swanson et al., 2013). A common type of synthesis used to identify RBPs is a meta-analysis. In a meta-analysis, researchers establish criteria for what studies will be included in their review, and then calculate the average impact (i.e., effect size) of the practice across the studies reviewed (Cooper, Hedges, & Valentine, 2009; see Banda & Therrien, 2008, for a practitioner-friendly introduction to meta-analysis). An evidence-based review is another type of synthesis used to identify RBPs (Cook et al., 2012). Similar to meta-analyses, evidence-based reviews analyze multiple studies, but the latter (1) consider only studies

LEARNING DISABILITIES RESEARCH

that meet quality standards and (2) use predetermined standards to categorize the effectiveness of the practice based on the findings from high-quality studies. Despite their strengths, research syntheses are also subject to limitations that can impact the accuracy and efficiency of their results. For example, published syntheses can be lengthy and, without an understanding of advanced statistical methods, the details and nuances of the findings can be challenging to interpret. Also, just like with individual studies, low quality methods can negatively impact the accuracy and utility of the findings (Cooper et al., 2009). Moreover, most practitioners have neither the time nor expertise needed to critically examine and evaluate a synthesis. Additionally, because syntheses conducted by different organizations and researchers do not necessarily use the same methods (e.g., different studies are included), they can produce conflicting results about the effectiveness of a practice (Briggs, 2008; Cook & Cook, 2013; Slavin, 2008). Finally, syntheses have not been conducted on all practices, and they can quickly fall out of date as new research is completed. Level 4 Resources We identified five Web sites that currently offer syntheses. Meta-analyses are available at the Center on Instruction and evidence-based reviews can be found at WWC, Best Evidence Encyclopedia, National Secondary Transition Technical Assistance Center, and Current Practice Alerts. The Center on Instruction has produced two meta-analyses that include research with samples containing some or all students with LD: Interventions for Adolescent Struggling Readers: A Meta-Analysis with Implications for Practice; and Mathematics Instruction for Students with Learning Disabilities or Difficulty Learning Mathematics: A Synthesis of the Intervention Research. They focus on grades 4–12 and 1–12, respectively. These syntheses can be found by clicking the “Special Education” link on the Center on Instruction’s home page and then selecting “Research: Meta-analyses and Summaries.” Although Center on Instruction is not producing new publications, all their resources are archived and remain available. The evidence-based reviews offered by WWC, called “Intervention Reports,” are produced by a team of experts and include: (1) an overview of the targeted practice, (2) a detailed description of the methods and results for each included study, and (3) a list of studies excluded from the review along with the reasons why. WWC syntheses categorize practices—based on the number, quality, and effects of reviewed studies—as having either positive, potentially positive, mixed, no discernible, potentially negative, or negative effects (Effectiveness Rating). An Improvement Index (which summarizes the impact of the intervention in percentile points) and an Extent of Evidence Rating (either small or medium to large) are also provided. WWC Intervention Reports can be accessed by clicking the “Intervention Reports” link on the home page. At the time of our review, WWC had published 17 syntheses targeting students with LD (i.e., included research with samples comprised of at least 50 percent of students with LD). As shown in Table 2,

95

TABLE 2 What Works Clearinghouse Ratings for Practices with Reviewed Research Studies Involving Students with Learning Disabilities Rating

Practice and Grade Level by Outcome Area

General Reading Achievement No Discernible Effects Alphabetics Potentially Positive Effects No Discernible Effects Reading Fluency Potentially Positive Effects No Discernible Effects Reading Comprehension Potentially Positive Effects No Discernible Effects Mathematics Potentially Positive Effects No Discernible Effects Writing Potentially Positive Effects

Project Read Phonology (K-4) Repeated Reading (5–12) Lindamood Phoneme Sequencing (4) Reading Mastery (2–4) Repeated Reading (5–12) Lindamood Phoneme Sequencing (4) Peer-Assisted Learning Strategies (2–6) Read Naturally (4–6) Repeated Reading (5–12) Peer-Assisted Learning Strategies (2–6) Repeated Reading (5–12) Lindamood Phoneme Sequencing (4) Reading Mastery (2–4) Lindamood Phoneme Sequencing (4) Peer-Assisted Learning Strategies (2–6)

Read Naturally (4–6) Spelling Mastery (2–4) Potentially Negative Effects Lindamood Phoneme Sequencing (4) No Studies Meet Evidence Standards Alphabetic Phonics, Barton Reading & Spelling System, Dyslexia Training Program, Fundations, Herman Method, Read 180, Reciprocal Teaching, Unbranded Orton–Gillingham-based Interventions, Voyager Reading Programs, Wilson Reading System.

seven of these syntheses include at least one study that met WWC evidence standards and, thus, these practices received an Effectiveness Rating. Whereas WWC syntheses focus on specific practices, the evidence-based reviews produced by Best Evidence Encyclopedia analyze research related to broader topic areas, such as elementary mathematics instruction. The rating scale used by the Best Evidence Encyclopedia to describe the effectiveness of a practice has six categories: strong evidence of effectiveness, moderate evidence of effectiveness, limited evidence of effectiveness—strong evidence of modest effects, limited evidence of effectiveness—weak evidence with notable effects, insufficient evidence of effectiveness, and no qualifying studies. Each “Best Evidence Synthesis” completed by the Best Evidence Encyclopedia results in two publications: a “Full Report,” which comprehensively describes the synthesis methods and findings, and an “Educator’s Summary,” which is a short and easy-to-read summary designed to help practitioners understand and utilize the synthesis results. Syntheses can be accessed via the topic-specific links on the left-hand side of the home page. At the time of our review, the Best Evidence Encyclopedia offered one synthesis that included research with (K-5) students with LD, titled Effective Programs for Struggling Readers: A Best Evidence Synthesis.

96

SANTANGELO ET AL.: 6S PYRAMID

TABLE 3 National Secondary Transition Technical Assistance Center Ratings for Practices with Reviewed Research Studies Involving Students with Learning Disabilities Level of Evidence

Practice and Grade/Age Level by Outcome Area

Academic Skills Strong

Functional Life and Transition Skills Moderate

Potential

The evidence-based reviews published by National Secondary Transition Technical Assistance Center, called “Practice Descriptions,” are written specifically for practitioners. Each includes a brief summary of the practice, a short description of the research included in the review, a list of additional references, and recommendations for learning more about implementation (including links to sample lesson plans). The level of evidence supporting a practice is rated as strong, moderate, potential, or low. Although National Secondary Transition Technical Assistance Center’s synthesis process is generally consistent with the way other organizations (e.g., WWC and Best Evidence Encyclopedia) conduct evidence-based reviews, it is unique in that, sometimes, an existing synthesis (e.g., a meta-analysis published in a peer-reviewed journal) is used to develop a Practice Description (rather than independently locating and analyzing the individual studies). At the time of our review, 14 Practice Descriptions included research with students with LD (see Table 3). They can be accessed from the “Evidence-based Practices” tab at the top of their home page or directly from http://nsttac.org/content/evidence-basedpractices-secondary-transition. The Council for Exceptional Children’s Division for Learning Disabilities and Division for Research collaboratively produce a series of “Current Practice Alerts”. Like National Secondary Transition Technical Assistance Center’s Practice Descriptions, Current Practice Alerts are written specifically for educators and include a concise description of the targeted practice and research. Current Practice Alerts rate the effectiveness of practices reviewed as either Go for It (evidence suggests a high likelihood of effectiveness) or Use Caution (either insufficient evidence to determine effectiveness or evidence indicating small, no, or negative effects). Current Practice Alerts differ from the systematic syntheses produced by WWC, Best Evidence Encyclopedia, and National Secondary Transition Technical Assistance Center because a uniform set of predetermined standards is not used

Mnemonics (13–17) Peer Assistance (13–17) Self-Management (13–16) Technology (12–22) Visual Displays (13–16) Self-Advocacy strategy (to teach student involvement in the IEP meeting, 12–18) Self-Determined Learning Model (to teach goal attainment, 14–19) Self-Directed IEP (to teach student involvement in the IEP meeting, 12–21) Simulations (to teach social skills, 12–21) Training Modules (to promote parent involvement in the transition process, 15+) Whose Future is it Anyway? (to teach knowledge of transition planning, 12–16) Whose Future is it Anyway? (to increase self-determination skills, 12–16) Computer-Assisted Instruction (to teach participation in the IEP process, 12–18) Mnemonics (to teach completing a job application, 15–16)

to categorize the effectiveness of each practice reviewed. Rather, authors, who are experts in the field, make a decision based on their own interpretation of the research (which is then reviewed and verified by the Alerts Editorial Committee). As shown in Table 4, 16 Current Practice Alerts that include research with students with LD were available at the time of our review. In addition to the Web resources described above, research syntheses examining the effectiveness of instructional practices for students with LD are frequently published in professional journals. As previously described with Level 6 resources, Google Scholar, ERIC, and other search engines can be used to locate relevant syntheses published in journals. For example, a teacher who wants to improve the written language skills of his/her students with LD would find several syntheses by conducting a search using the key words “learning disabilities,” “meta-analysis,” and “writing instruction.” Level 3: Synopses of Syntheses Synopses of syntheses are brief and accessible reviews of existing syntheses. They include a summary of the synthesis’ methods, findings, and implications and, in most cases, an expert evaluation of the research quality (DiCenso et al., 2009). Synopses of syntheses preserve the strengths of syntheses (e.g., determining effectiveness across multiple studies), while addressing some of their limitations (e.g., significant time and expertise required to critically analyze methods and interpret results). Nonetheless, it is important to recognize, like Level 5 and Level 4 evidence, Level 3 synopses are only as good as the original research on which they are based (Windish, 2013). Additionally, they are relatively rare in education and, as with syntheses, they become outdated as more research is conducted.

LEARNING DISABILITIES RESEARCH

97

TABLE 4 Current Practice Alerts Ratings for Reviewed Practices Practice and Grade Levela by Outcome Area

Rating Multiple Academic Domains and Content Areas Go For It

Use Caution

Reading Go For It

Use Caution Writing Go For It a Most

• Direct Instruction (K-12) • Class Wide Peer Tutoring (K-12) • Cognitive Strategy Instruction (K-12) • Graphic Organizers (K-12) • Mnemonics (K-12) • Self-Determined Learning Model of Instruction (not specified) • Vocabulary Instruction (K-12) • Cooperative Learning (K-12) • Co-teaching (K-12) • Learning Styles (not specified) • Fluency Instruction (not specified) • Phonics Instruction (beginning readers and older students with reading difficulties) • Phonological Awareness Acquisition and Intervention (beginning readers) • Reading Comprehension Strategy Instruction (not specified) • Reading Recovery (not specified) • Self-Regulated Strategy Development for Writing (upper-elementary through middle school)

Current Practice Alerts indicate a relevant age range, rather than the specific grades represented in the research used to determine the rating.

Level 3 Resources Center on Instruction and National Dissemination Center for Children with Disabilities are the two Internet resources we identified that currently offer synopses of syntheses related to students with LD. Center on Instruction’s synopses are written for practitioners and include an easy-to-understand summary of the synthesis methods and findings, as well as a section devoted to implications for practice. They do not include an explicit evaluation of the synthesis methods; however, the syntheses were all chosen to be synopsized because of their quality. Center on Instruction syntheses can be located by clicking the “Special Education” link on the lefthand side of the home page, and then selecting “Research: Meta-analyses and Summaries” or by typing “Synopsis” into the “Search Terms” box at the top of the page. Center on Instruction offers one synopsis of a synthesis examining ways to improve expository text comprehension for students with LD, and two others related to writing instruction for all students, including those with LD. National Dissemination Center for Children with Disabilities offers a number of synopses of syntheses, which they call “Research Summaries.” Each synopsis uses the same, practitioner-friendly format and offers a summary of the synthesis methods, findings, and implications. As with Center on Instruction, National Dissemination Center for Children with Disabilities synopses do not include an evaluative component, but syntheses are selected because of their generally high quality. Eighteen of National Dissemination Center for Children with Disabilities’ synopses are of syntheses that included research with samples comprised of students with LD. They represent a variety of outcome areas (e.g., reading, math, writing) and span grades K-12. Because of

discontinued funding, National Dissemination Center for Children with Disabilities’ original Web site is no longer active; however, all the publications are archived and remain available through The Center for Parent Information and Resources (www.parentcenterhub.org/resources). Level 2: Summaries Summaries are publications typically produced by a group of experts, who seek to integrate all available research evidence (e.g., original research and syntheses) related to a particular topic area; therefore, they provide highly reliable recommendations (DiCenso et al., 2009; Haynes, 2006). To the degree that summaries reflect an entire research base, they represent the most accurate form of evidence that can be used to identify RBPs. Additionally, reading a single summary is much more efficient than reading the large body of studies and syntheses on which the recommendations in the summary are based. In education, Level 2 summaries are currently the highest level of evidence available and, thus, this is the level where educators should begin their search for information, examining lower levels only when relevant Level 2 summaries cannot be found. Although summaries do not exist for all academic areas, they are available for many key skill areas, such as reading, writing, and mathematics. Despite their many strengths, it is important to recognize that, like syntheses, summaries are only as valid as the studies and the process used to create them (Windish, 2013). Finally, summaries are not available for all populations (e.g., students with LD) in all topic areas. And, as with syntheses, summaries need to be reviewed and revised periodically so they do not become outdated.

98

SANTANGELO ET AL.: 6S PYRAMID

Level 2 Resources

Considerations and Limitations

“Practice Guides” are the only example of summaries we identified related to LD. Each Practice Guide is written by a panel of experts, which includes both researchers and practitioners, and is focused on a specific topic (e.g., reading comprehension) and (in most cases) specific student population (e.g., elementary-aged students; English language learners). Based on the panel’s extensive and rigorous review of the available, high quality research, a set of recommendations for school/classroom practice is offered. Each recommendation is rated as being supported by a strong, moderate, or minimal level of evidence. Although Practice Guides are comprehensive, they are written so that practitioners can understand the panel’s methods and findings and can use them to guide school/classroom practice. Practices Guides can be accessed by clicking the “Practice Guides” link on WWC’s home page. At the time of our review, none of the 17 available Practice Guides focused specifically on students with LD. However, nine included research involving at least some students with LD: Assisting Struggling Students with Mathematics; Dropout Prevention; Developing Effective Fractions Instruction for Kindergarten Through 8th Grade; Effective Literacy and English Instruction for English Learners in the Elementary Grades; Improving Adolescent Literacy: Effective Classroom and Intervention Practices; Improving Mathematical Problem Solving in Grades 4 Through 8; Organizing Instruction and Study to Improve Student Learning; Reducing Behavior Problems in the Elementary School Classroom; and Teaching Elementary School Students to be Effective Writers.

When using the 6S Pyramid as a guide for identifying effective instructional practices, it is important that special educators bear in mind that different resources use different procedures and criteria, the levels of the 6S Pyramid may overlap, RBP-related terminology is used inconsistently, and implementation of RBPs needs to be supported. Consideration of these issues will inform and optimize special educators’ use of the 6S Pyramid. Educators should not assume that RBPs at a given level of the 6S Pyramid are equivalent across Web sites/organizations. For example, at Level 4, some syntheses use stringent criteria applied to a comprehensive search of the literature as the basis for their conclusions, whereas others use no systematic criteria or criteria that are less rigorous. Similarly, synopses vary in terms of their clarity for practitioners, the depth of information provided, and the presence and quality of an evaluation. In fact, resources at the same level may occasionally provide conflicting recommendations (e.g., one synthesis indicates a practice is effective, but another reports it is not). In such cases, the higher quality resources should carry more weight. If practitioners encounter conflicting results from resources at the same level of the 6S Pyramid, we recommend they consult with individuals with advanced research training and expertise to determine which is more credible. Organizations providing resources related to RBPs have not, to our knowledge, designed their products to align directly with one or more levels of the 6S Pyramid. Rather, the 6S Pyramid is a heuristic framework that we have applied to existing resources in order to better understand and apply those resources. Although we believe that the different levels of the 6S Pyramid make useful distinctions between different types of resources, it is important to recognize that it may be difficult to neatly classify some resources into one of the six levels. For example, an exhaustive meta-analysis that is conducted by a group of scholars and includes an expanded discussion of practical recommendations might reasonably be considered either a Level 2 (summary) or Level 4 (synthesis). Despite this potential difficulty, it is important to remember that the guiding principle of the 6S Pyramid is for practitioners to prioritize resources that incorporate entire research bases, regardless of how we or others may categorize them. Terminology related to RBPs is used inconsistently in the field of special education (Cook & Cook, 2013). For example, National Secondary Transition Technical Assistance Center refers to a practice as evidence-based (with potential evidence) when supported as effective by a single study. In contrast, other synthesis authors (e.g., Baker, Chard, Ketterlin-Geller, Apichatabutra, & Doabler, 2009; Chard, Ketterlin-Geller, Baker, Doabler, & Apichatabutra, 2009) only categorized practices as evidence-based for students with LD when supported by multiple, high-quality studies. Additionally, sources use different terms and criteria to classify the effectiveness of practices. For instance, the WWC refers to practices supported by the highest level of evidence as having “positive effects,” whereas National Secondary Transition Technical Assistance Center uses

Level 1: Systems Representing the highest tier of the 6S Pyramid, systems are computerized decision support tools that summarize all relevant research evidence about a particular practice or topic (Cullum, Ciliska, Haynes, & Marks, 2008; DiCenso et al., 2009). Systems provide recommendations—based primarily on Level 2 summaries—in response to practitioner input (Haynes, 2001, 2006). For example, in medicine, clinicians can enter patient characteristics and symptoms, and the electronic system will provide research-based recommendations aligned to the patient’s specific needs and characteristics. In medicine, enthusiasm for systems is growing (e.g., Hlyva et al., 2010; Osheroff et al., 2007). Although systems do not yet exist in education, we envision them being developed and utilized in the future. Once this happens, an educator could enter information about a learner’s characteristics (e.g., grade, age, disability information) and target area (e.g., beginning reading skills, math problem solving) via a pulldown menu on a Web site, and the system would generate a list of recommended practices based on the best available research evidence. Importantly, the goal of systems is not to replace professional judgment, but rather to provide individualized, authoritative, and timely research-based recommendations to inform and support practitioners’ decision making (Hlyva et al., 2010).

LEARNING DISABILITIES RESEARCH

“evidence-based practices with strong evidence” and the Current Practice Alerts use “Go For It.” Because these labels correspond with different criteria, it is important to understand what a particular term designates. Although we recommend using the 6S Pyramid to identify effective practices, it is important to recognize identifying RBPs is but the first (and perhaps easiest) step in implementing effective instruction and improving learner outcomes. As Fixsen, Blase, Horner, and Sugai (2009) noted, identifying an RBP “is one thing, implementation of that practice is another thing altogether” (p. 5). Implementing RBPs successfully over time can be very difficult in schools and needs to be supported by ongoing training and a supportive school culture (see Cook & Odom, 2013; Fixsen, Blase, Metz, & Van Dyke, 2013). Moreover, some recommended practices/programs may entail costs and training not available to some educators. Additionally, special educators will need to balance implementing effective practices as designed (i.e., with fidelity) with adapting them in ways to optimize their positive impact on students with LD. We suggest interested educators read Johnson and McMaster’s (2013) excellent chapter for guidelines on adapting RBPs by implementing critical elements of RBPs as designed, while tailoring other aspects of interventions to meet the unique characteristics and needs of groups and individuals. Finally, it is important to recognize that even practices with significant research support will not work for every learner; there will be “nonresponders” to every practice. As such, even when using practices at the highest levels of the 6S Pyramid, special educators should systematically monitor student progress to evaluate the effects of the practice on individual learners. Notwithstanding these limitations and considerations, the 6S Pyramid can be a useful tool for educators to efficiently identify and prioritize RBPs for students with LD using existing Internet resources. We conclude with the second act of our vignette, in which Ms. Markloff and Ms. Hutchinson demonstrate how the 6S Pyramid can be applied in practice. Ms. Markloff and Ms. Hutchinson: Act II After reading about the 6S Pyramid, Ms. Markloff and Ms. Hutchinson are excited to conduct a second search for RBPs; this time, they will use their understanding of hierarchical levels of evidence to search strategically and efficiently. The teachers plan to start searching at the top of the Pyramid and work down only as necessary. Because there are no resources at Level 1 in education, they begin their search browsing through the only Level 2 resources available for students with LD, the Practice Guides, and decide that Improving Adolescent Literacy: Improving Classroom and Intervention Practices is the most relevant because it focuses on students in 4th–12th grade and includes some research with students with LD. In the guide, they find five recommendations. The first four relate to core reading instruction for all students: provide explicit vocabulary instruction, provide direct and explicit comprehension instruction, provide opportunities for extended discussion of text meaning and interpretation, and increase

99

student motivation and engagement in literacy learning. After discussing these recommendations, Ms. Markloff and Ms. Hutchinson conclude that they already incorporate these practices as part of their instructional routine, though the descriptions and examples in the Practice Guide do provide them with several new and important ideas for optimizing their instruction. The teachers recognize they do not implement the fifth recommendation (make available intensive and individualized interventions for struggling readers that can be provided by trained specialists), so they read this section of the guide very carefully and take notes on the recommendations they would like to implement. These include: using assessments to precisely identify each student’s strengths and weaknesses, aligning instruction and intervention with students’ individual needs, establishing learning goals, utilizing learning aids, such as graphic organizers, and explicitly teaching reading strategies. Ms. Markloff and Ms. Hutchinson decide to search for additional resources to help them deepen and extend what they learned by reading the Practice Guide. Having exhausted the relevant Level 2 resources, the teachers search next for Level 3 resources. To their surprise—between National Dissemination Center for Children with Disabilities and Center on Instruction—they locate more than 10 synopses of syntheses related to reading instruction for students with LD. While reading these publications, they expand and refine their notes. For example, reading National Dissemination Center for Children with Disabilities’ synopsis of How Reading Outcomes of Students with Disabilities are Related to Instructional Grouping Formats: A Meta-Analytic Review convinces them to use small group instruction more frequently, because a solid body of research indicates it improves reading outcomes for students with disabilities, including those with LD. They also decide to implement peermediated programs, such as peer tutoring and reciprocal teaching, because they are supported as effective in the synopsis. From National Dissemination Center for Children with Disabilities’ synopses of Fluency and Comprehension Gains as a Result of Repeated Reading and A Synthesis of Research on Effective Interventions for Building Reading Fluency with Elementary Students with Learning Disabilities, they learn about the positive outcomes associated with repeated reading and decide to also incorporate repeated reading into their instructional routine. Center on Instruction’s synopsis of Improving Comprehension of Expository Text in Students with Learning Disabilities: A Research Synthesis and several similar synopses produced by National Dissemination Center for Children with Disabilities reinforce what they learned from the Practice Guide about the effectiveness of content/text enhancements and cognitive strategy instruction, and also provide additional suggestions for implementation. At this point, Ms. Markloff and Ms. Hutchinson review all of their notes and decide that they have identified a sufficient number of RBPs to implement based on what they learned from the Level 2 and Level 3 resources. Rather than try to implement everything at once, they select two RBPs with which they feel comfortable and that align well with the needs of their students: graphic organizers and repeated reading. They each take one practice to research and set a goal of finding or making a checklist of the critical elements of each

100

SANTANGELO ET AL.: 6S PYRAMID

practice by the end of next week that they will use to guide their use of the practices. REFERENCES Baker, S. K., Chard, D. J., Ketterlin-Geller, L. R., Apichatabutra, C., & Doabler, C. (2009). The basis of evidence for Self-Regulated Strategy Development for students with or at risk for learning disabilities. Exceptional Children, 75, 303–318. Banda, D. R., & Therrien, W. J. (2008). A teacher’s guide to meta-analysis. Teaching Exceptional Children, 41(2), 66–71. Briggs, D. C. (2008). Synthesizing causal inferences. Educational Researcher, 37(1), 15–22. doi:10.3102/0013189×08314286 Chard, D. J., Ketterlin-Geller, L. R., Baker, S. K., Doabler, C., & Apichatabutra, C. (2009). Repeated reading interventions for students with learning disabilities: Status of the evidence. Exceptional Children, 75, 263–281. Cook, B. G. (2014). A call for examining replication and bias in special education research. Remedial and Special Education, 35, 233–246. doi:10.1177/0741932514528995 Cook, B. G., & Cook, S. C. (2013). Unraveling evidence-based practices in special education. Journal of Special Education, 47, 71–82. doi:10.1177/0022466911420877 Cook, B. G., & Odom, S. L. (2013). Evidence-based practices and implementation science in special education. Exceptional Children, 79, 135–144. Cook, B. G., Smith, G. J., & Tankersley, M. (2012). Evidence-based practices in education. In K. R. Harris, S. Graham, & T. Urdan (Eds.), APA educational psychology handbook (Vol. 1, pp. 495–528). Washington, D.C.: American Psychological Association. Cook, B. G., Tankersley, M., Cook, L., & Landrum, T. J. (2008). Evidence-based practices in special education: Some practical considerations. Intervention in School & Clinic, 44, 69–75. doi:10.1177/ 1053451208321452 Cooper, H., Hedges, L. V., & Valentine, J. C. (2009). The handbook of research syntheses and meta-analysis (2nd ed.). New York, NY: Russell Sage. Cullum, N., Ciliska, D., Haynes, R. B., & Marks, S. (Eds.) (2008). Evidencebased nursing: An introduction. Oxford, UK: Blackwell. DiCenso, A., Bayley, L., & Haynes, R. B. (2009). Accessing pre-appraised evidence: Fine-tuning the 5S model into a 6S model. Evidence Based Nursing, 12, 99–101. doi:10.1136/ebn.12.4.99-b Fixsen, D. L., Blase, K. A., Horner, R., & Sugai, G. (2009). Concept paper: Developing the capacity for scaling up the effective use of evidence-based programs in state departments of education. Retrieved from http://www.uconnucedd.org/lend/readings/2011/pdfs/Session %2025%20-%20Mar%2025,%202011/Concept_Paper_SISEP_0409_ WEB.pdf Fixsen, D., Blase, K., Metz, A., & Van Dyke, M. (2013). Statewide implementation of evidence-based programs. Exceptional Children, 79, 213–230. Gersten, R., Fuchs, L., Compton, D., Coyne, M., Greenwood, C., & Innocenti, M. S. (2005). Quality indicators for group experimental and quasi-experimental research in special education. Exceptional Children, 71, 149–164.

Hattie, J. A. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York, NY: Routledge. Haynes, R. B. (2001). Of studies, syntheses, synopses, and systems: The ‘‘4S’’ evolution of services for finding current best evidence. Evidence Based Medicine, 6, 36–38. doi:10.1136/ebm.6.2.36 Haynes, R. B. (2006). Of studies, syntheses, synopses, summaries, and systems: The “5S” evolution of information services for evidencebased healthcare decisions. Evidence- Based Medicine, 11, 162–164. Hlyva, O., Cotoi, C., Hobson, N., Parrish, R., Jedraszewski, D., Wilczynski, N., et al. (2010). MacPLUS Federated Search: A new era tool for evidence-informed clinical decision making. McMaster University Medical Journal, 7, 57–60. Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179. Jasny, B. R., Chin, G., Chong, L., & Vignieri, S. (2011). Again, and again, and again . . . Science, 334(6060), 1225. doi:10.1126/ science.334.6060.1225 Johnson, L. D., & McMaster, K. L. (2013). Adapting research-based practices with fidelity: Flexibility by design. In B. G. Cook, M. Tankersley, & T. J. Landrum (Eds.), Advances in learning and behavioral disabilities (Vol. 26, pp. 65–91). Bingley, UK: Emerald. Miech, E. J., Nave, B., & Mosteller, F. (2005). The 20,000 article problem: How a structured abstract can help practitioners sort out educational research. Phi Delta Kappan, 86, 396–400. Moonesinghe, R., Khoury, M. J., & Janssens, A. C. J. (2007). Most published research findings are false—But a little replication goes a long way. PLoS Medicine, 4(2), e28. doi:10.1371/journal.pmed.0040028 Odom, S. L., Brantlinger, E., Gersten, R., Horner, R. H., Thompson, B., & Harris, K. R. (2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71, 137–148. Osheroff, J. A., Teich, J. M., Middleton, B., Steen, E. B., Wright, A., & Detmer, D. E. (2007). A roadmap for national action on clinical decision support. Journal of the American Medical Informatics Association, 14, 141–145. doi:10.1197/jamia.M2334 Robeson, P., Dobbins, M., DeCorby, K., & Tirilis, D. (2010). Facilitating access to pre-processed research evidence in public health. BMC Public Health, 10, 95. doi:10.1186/1471-2458-10-95 Slavin, R. E. (2002). Evidence-based education policies: Transforming educational practice and research. Educational Researcher, 31(7), 15–21. doi:10.3102/0013189×031007015 Slavin, R. E. (2008). Perspectives on evidence-based research in education: What works? Issues in synthesizing educational program evaluations. Educational Researcher, 37(1), 5–14. doi:10.3102/ 0013189×08314117 Swanson, H. L., Harris, K. R., & Graham, S. (2013). Handbook of learning disabilities (2nd ed.). New York, NY: Guilford. Windish, D. (2013). Searching for the right evidence: How to answer your clinical questions using the 6S hierarchy. Evidence-Based Medicine, 18(3), 93–97. doi:10.1136/eb-2012-100995 Yell, M. L., & Rozalski, M. (2013). The peer-reviewed requirement of the IDEA: An examination of law and policy. In B. Cook, M. Tankersley, & T. J. Landrum (Eds.), Evidence-based practices. Advances in learning and behavioral disabilities (Vol. 26, pp. 149–172). Bingley, UK: Emerald.

About the Authors Tanya Santangelo is an Associate Professor in the Department of Leadership for Educational Equity and Excellence in the School of Education at Arcadia University. She received her Ph.D. in Special Education from the University of Maryland in 2005. Her two primary areas of research/interest are: developing, validating, and disseminating effective practices for teaching and assessing writing; and increasing educators’ knowledge and use of evidence-based practices. Leslie C. Novosel is an Assistant Professor in the Department of Special Education at the University of Hawaiʻi at M¯anoa. She received her Ph.D. in Special Education from the University of Kansas in 2011. Her research interests include evidencebased practices, school-wide models of response to intervention, and literacy interventions for adolescents with learning disabilities.

LEARNING DISABILITIES RESEARCH

101

Bryan G. Cook is a Professor in the Department of Special Education at the University of Hawaiʻi at M¯anoa. He received his Ph.D. in Special Education from the University of California at Santa Barbara in 1997. His research interests include evidencebased practices, meta-research in special education, and physical activity among children and youth with high-incidence disabilities. Meredith Gapsis is a special education teacher who works in an inclusive high school English classroom. She is currently completing her doctoral degree in special education at Arcadia University. Her dissertation focuses on teachers’ use of Web-based resources to identify and implement evidence-based practices.