Scaffolding Critical Reasoning About History and Social ... - CiteSeerX

AAH GRAPHICS, INC. / (540) 933-6210 / FAX 933-6523 / 09-30-2002 / 11:57

Scaffolding Critical Reasoning About History and Social Issues in Multimedia-Supported Learning Environments John W. Saye Thomas Brush

This article advances a continuing line of research that investigates the potential of hypermedia resources and scaffolding for supporting problem-based social studies and developing critical reasoning. Our line of inquiry consists of a series of generative design experiments that informs problem-based curriculum development. Our findings suggest that expert guidance may be embedded into the learning environment to give students conceptual and strategic road maps that assist them in understanding the process of disciplined inquiry. However, our results also emphasize the difficulties in managing the cognitive challenges posed by ill-structured social problems and suggest limits to the embedded support that can be provided for complex thinking. Complex conceptual tasks may require spontaneous support that can only be provided by a skilled teacher. We suggest that embedded scaffolds may be used to support teachers by reducing the amount of spontaneous scaffolding they must do in an ill-structured environment and discuss other steps that might be taken to encourage problem-based inquiry.

Educators have persistently advocated the investigation of ill-structured problems as a way to engage students with a variety of content and develop better decision makers and problem solvers (Jonassen, 1997). However, implementing inquiry-oriented activities in a classroom environment remains difficult. Numerous challenges complicate efforts to develop thoughtful problem solvers. These include obstacles originating within organizational structures of the learning environment (Cuban, 1984; Linn, 1995; Onosko, 1991), teachers (Doyle & Ponder, 1977–78; Saye, 1998), and learners (Brush & Saye, 2000, Land, 2000). This article advances a continuing line of research that investigates how embedding hypermedia resources and scaffolding in a multimedia learning environment might mitigate some of the obstacles to critical reasoning about ill-structured problems. Previous findings in our line of inquiry have suggested that multimedia supported learning environments might assist with two learner obstacles to problem-based study: failure to deeply engage the content and failure to recognize alternative perspectives about the problem. This report focuses on scaffolding support that might assist students and teachers with a third barrier: meeting the heavy cognitive demands of critical reasoning about ill-structured problems.

OVERVIEW OF THE PROBLEM

Problem-based learning activities provide students with opportunities to examine complex problems using a wide variety of resources, ETR&D, Vol. 50, No. 3, 2002, pp. 77–96 ISSN 1042–1629

77


78 develop their own strategies for addressing these problems, and present and negotiate solutions to these problems in a collaborative manner (Hannafin, Hill, & Land, 1997). There are numerous examples of successful implementations of problem-based learning in a variety of content areas, including mathematics (Cognition and Technology Group at Vanderbilt— CTGV, 1992, 1993), science (Linn, Shear, Bell, & Slotta, 1999; Loh et al., 2001), and literature (Jacobsen & Spiro, 1994). However, problembased learning activities are particularly well suited to social studies and history because most social educators emphasize civic competence as a primary goal. Civic competency may be defined as “the ability to make informed and reasoned decisions for the public good as citizens of a culturally diverse, democratic society in an interdependent world” (NCSS, 1994). Critical reasoning about social problems is a central component in the ability to make competent civic decisions (Parker, Mueller, & Wendling, 1989; Saye & Brush, 1999).

Supporting Critical Reasoning About Ill-Structured Social Problems

Although developing the ability to reason critically is important in any content domain, social problems require a different kind of reasoning than well-structured problems of logic (Perkins, Allen, & Hafner, 1983). Social problems are illstructured, multilogical, and controversial. Conflicting perspectives and logics must be considered and various problem solutions are possible. Rather than formal problem solving, reasoning through a social issue is more like model building. A significant element in building a model of a social situation is dialectical reasoning: the exploration of competing logics or points of view (Parker et al., 1989; Perkins et al., 1983). Newmann (1991) noted five competencies that are peculiar to higher-order critical reasoning in social studies. (a) Thinkers must have empathy, an ability to view the world from the perspective of another. They must be able (b) to apply abstract concepts to specific situations and (c) to infer beyond limited data to draw con-

ETR&D, Vol. 50, No. 3

clusions. They must be able (d) to engage in critical discourse aimed at clarifying understanding about an issue and (e) to apply evaluative criteria to develop defensible decisions about a social problem. Many of these competencies must be used in concert by the thinker to engage in persuasive reasoning—taking an informed stand that provides an elaborated, convincing defense for a position (Newmann, 1990). However, persuasive reasoning is not enough to achieve the critical reasoning required for civic competence. Persuasive reasoning may be only sophisticated reasoning, a skillful justification of one’s position without real openness to entertaining opposing viewpoints. Competence regarding civic issues demands active empathy as well as persuasiveness. One must be able to entertain genuinely a set of beliefs and values which are not one’s own (Ashby & Lee, 1987). Critical reasoners move beyond sophisticated reasoning to dialectical reasoning that genuinely recognizes alternative perspectives regarding ill-structured social problems (Parker et al., 1989). Kuhn (1999) posited that critical thinking evolves developmentally. Thinkers move from assumptions that reality is directly knowable and knowledge is certain and received from authoritative sources to beliefs that reality is not directly knowable and knowledge is uncertain and constructed by the knower. However, more advanced epistemological understandings exist on two levels. One may hold the multiplist view that since knowledge is subjective, individuals’ assertions cannot be judged by others. They are opinions and “everyone is entitled to his/her opinion.” Thus, thinkers may have empathy without using that understanding to make reasoned decisions about social controversy. However, at the most advanced developmental level, thinkers assume the evaluative view that assertions may be compared and judged by the quality of evidence and reasoning used to support them. Such thinkers may incorporate empathy into true critical reasoning that genuinely considers and rigorously weighs alternative points of view to make a decision about a social issue. Kuhn’s concept focuses two challenges for developing competent civic reasoners: (a) How can we move learners to view knowledge as ten-


SCAFFOLDING CRITICAL REASONING

tative yet subject to defensible evaluative standards? (b) How might we help learners use such standards to engage in disciplined inquiry?

Obstacles to the Development of Critical Reasoning about Social Problems

Critical reasoning about problems requires a substantial knowledge base. Previous work in problem-based instruction has noted difficulties in encouraging learners to persist with exploration of the topic to develop deep knowledge (Newmann, 1991; Onosko, 1991; Rossi, 1995) and entertain competing perspectives (Newmann, 1991; Parker et al.,1989). Theorists have claimed that more authentic learning experiences may produce greater student engagement with content (Choi & Hannafin, 1995; Newmann, Wehlage, & Lamborn, 1992). Some researchers have suggested that rich, authentic contexts that can be facilitated by multimedia learning environments encourage students to become engaged with the content, explore more deeply, and develop more complex views of issues (CTGV, 1992; Dwyer, 1994; Kinzie & Sullivan, 1989). Findings from the initial study in our present line of inquiry support such claims. When compared to peers who encountered the topic in a more traditional expository classroom setting, students who studied the civil rights movement using a problem-based multimediaenhanced learning environment demonstrated greater engagement with the content and more empathetic understandings of historical dilemmas (Saye & Brush, 1999). Although engagement and empathy present imposing challenges for developing critical reasoning, perhaps the most formidable learner challenge is the lack of domain-specific knowledge that experts use for effective problem solving (Rossi, 1995; Voss, Greene, Post, & Penner, 1983; Wineburg, 1991b). Cognitive flexibility theorists refer to the three dimensions that experts perceive in a problem landscape (Chi, Glaser, & Farr, 1988; Spiro & Jehng, 1990). Novices tend to focus only on the two-dimensional surface features of a case. Experts incorporate an abstract third dimension, broader

79 conceptual structures that help them organize and analyze data in order to reason through a problem. Furthermore, unlike novices, experts employ metacognitive strategies that allow them to manage their thinking and apply effectively the knowledge that they possess (Palincsar & Brown, 1984; VanSickle & Hoge, 1991). Particularly germane to the present study are differences in the ways that novices and experts think historically. One’s ability to think historically begins with the way one perceives history. Novices tend to view history as a straightforward recounting of events and regard historical texts as authoritative narratives. Operating in the early developmental levels of critical thinking (Kuhn, 1999), they are confused by conflicting accounts (Wineburg, 1991b). Historians view texts as human creations or speech acts (Wineburg, 1991a). Historians look beyond the visible text for sub texts that allow them to make judgments about evidence and events. They seek to reconstruct the social context in which the document was created, “to summon the world” surrounding the document (Holt, 1990). When novices view texts as authoritative accounts, they do not anticipate the possibility of subtexts. A critical aspect of historical thinking is closed to them. Researchers have suggested that we may develop more expert historical thinking by engaging students in authoring historical narratives. Greene (1994) argued that developing a sense of authorship pushes students beyond acquiring knowledge to learning about the nature of facts, evidence, and interpretation. Other theorists have claimed that confronting students with conflicting historical narratives can cause students to think more deeply about claims and incite students to develop their own interpretations of evidence (e.g., Wineburg & Wilson, 1991). If they are to become competent thinkers and decision makers, novices must learn to ground their inquiry about social problems in disciplinary standards used by experts in the field (Scheurman & Newmann, 1998). Guidance by a skilled teacher is crucial to this process, but the sensitive task of guiding thinking while allowing room for discovery has not been explored in


80 enough depth to inform practice adequately (Brown, 1992). Situated cognition theorists have advocated a process of cognitive apprenticeship that involves modeling problem solving, supporting students’ developing understandings through scaffolding and coaching, and gradually decreasing guidance as student thinking becomes more expert (Brown, Collins, & Duguid, 1989). However, studies examining social studies teachers attempting guided discovery report that such practice is draining, both cognitively and physically (Rossi, 1995, Rossi & Pace, 1998). Teachers and learners are likely to need support in handling the demands of rigorous thinking. Some theorists have claimed that tools and resources available in hypermedia environments may be used to help scaffold more disciplined inquiry into ill-structured problems (Hannafin, Land, & Oliver, 1999; Jacobsen, Maouiri, Mishra, & Kolar, 1996). Hannafin and associates (1999) identified four types of scaffolding that may be incorporated into multimedia learning environments to support novices in thinking more expertly: (a) conceptual (guidance about what knowledge to consider), (b) metacognitive (guidance about how to think during learning), (c) procedural (guidance about how to utilize available resources and tools), and (d) strategic (guidance about alternative approaches that might assist decision making). Research investigating specifically the use of hypermedia scaffolding to support students’ historical authorship suggests that such support may improve conceptual representations of knowledge and analytical rigor (Hynd, Hubbard, Holschuh, Reinking, & Jacobsen, 2000; Perfetti, Britt, Van Dyke, & Gabrys, 1999; Spoehr & Spoehr, 1994). Little research has been done in technologyrich, student-centered social studies classrooms (Berson, 1996; Ehman, Glenn, Johnson, & White, 1998; Saye, 1997). To test ideas about the potential of technology for supporting disciplined inquiry, we collaborated with a veteran social studies teacher to develop and experiment with an integrated hypermedia learning environment, Decision Point! (DP), and a problem-based unit plan for utilizing the DP resources.


FRAMEWORK FOR THE DECISION POINT! LINE OF INQUIRY

Our line of inquiry consists of a series of generative design experiments that informs problemcentered curriculum development. We hope to develop a version of problem-based learning that is workable for the majority of social studies teachers who are deeply grounded in an expository tradition (e.g., Kagan, 1993; Shaver, Davis, & Helburn, 1979). Therefore, we have chosen to explore the potential of multimedia environments for facilitating problem-based inquiry in a typical rather than an ideal setting. We recruited an expository-oriented teacher with little experience using technology or studentcentered inquiry in her instruction. The same teacher taught all of the classes included in our field studies. Each experiment examined a different intact class of general level 11th grade U.S. history as it studied a unit on the civil rights movement. In the first year, we investigated two classes: one that used DP and one that did not. In the next year we focused on a single DP-based class. In this section, we describe the design of the DP environment, summarize results of the first year of field studies in the DP line inquiry (DP1), and present the purpose of the current study (DP2).

Design of the DP Environment

Our collaborating teacher selected the content focus for our curriculum development project: the African American civil rights movement. DP includes two basic components: (a) an interactive database of multimedia content resources related to the civil rights movement, and (b) scaffolding tools to support collecting, analyzing, and evaluating historical evidence and presenting conclusions. The database is organized conceptually into three strands that represent the principal change strategies employed by the movement: (a) legal challenges, (b) nonviolent protest, and (c) Black Power. Within each strand are seven to eight events associated with that strategy. Each event features an introductory essay, a timeline, and a number of associated documents. Featured documents



Figure 1

81

The Decision Point environment.

include primary and secondary text, images, audio, and audiovisual media (Figure 1.). We collaborated with the teacher to develop the unit problem, a collaborative learning strategy, and culminating problem-solving activities. Although we discussed possibilities for how the features of the DP multimedia environment might be used, the teacher decided the length of the unit, the structure of each day, the composition and management of collaborative groups, the level of teacher direction, and the determination of unit grades. Our unit scenario placed students in the roles of civil rights leaders immediately following the assassination of M.L. King, Jr. in 1968. Student teams answered the unit problem: What strategies should be pursued in 1968 to continue the struggle for a more just, equal society? Each team investigated one of the three change strategies by using the DP database and tools to explore civil rights movement events from 1954–1968. Representatives from each research team then joined new

decision making teams who shared their former groups’ research findings and used historical evidence to support arguments about the best course of action. Each group constructed a multimedia presentation to persuade the audience of the wisdom of its solution. Finally, each student composed an essay on an issue related to the unit problem (Saye & Brush, 1999). Ill-structured social problems such as our unit problem involve multiple complex issues. Without guidance, novices are unlikely to develop the expertise necessary for independent problem solving (Land & Hannafin, 1996). In our curriculum development, we conceptualized two types of thinking guidance: (a) hard scaffolds and (b) soft scaffolds. Hard scaffolds are static supports that can be anticipated and planned in advance based on typical student difficulties with a task. We experimented with embedding conceptual, metacognitive, and strategic hard scaffolds in the DP learning environment in ways that might


82 support teachers and learners in handling the cognitive demands of disciplined problem solving. For example, the structure of the database provided conceptual scaffolds to help students organize and connect evidence within the broader dimensions of the problem landscape. Events were associated with particular change strategies, and primary documents were associated with particular events. Within events, we grouped evidence by document type with the expectation that such categorization would help students consider the source of the evidence. Contextual data retrieval charts associated with each event provided more specific strategic scaffolds by helping students to ask more expert questions of historical evidence. A presentation construction tool provided procedural and strategic support to help students use evidence to author more reasoned problem narratives. In contrast to hard scaffolds, soft scaffolds are dynamic and situational. Soft scaffolding requires teachers to continuously diagnose the understandings of learners and provide timely support based on student responses. For example, if students fail to discern differences in the messages of two civil rights figures, a teacher might help them think more deeply about the texts by asking questions such as: “What does Lewis mean when he says _____? Why do you think he uses the word _____? Do you find similar words in King’s speech? Do you notice any difference in his tone and King’s?” Once students discover that differences exist, the teacher might refer them to other documents that could help them understand the origins of those differences. Some diagnosis of student difficulties occurs as teachers observe students at work. However, more comprehensive diagnosis may be achieved by building features into the unit that deliberately create opportunities for the teacher to probe and support understandings while they are under construction. In our DP planning, we discussed the possibilities of using features such as interim work products and regular teacher meetings with work groups, but reserved for the teacher the ultimate decisions about the soft scaffolding to be employed.


Summary of Findings from the Year-1 Study (DP1)

In the first year field tests (DP1), we compared two cases: our collaborating teacher taught one class using the DP environment within a problem-based unit (DP1), and a second class using her preferred expository style and normal resources. The teacher attempted little soft scaffolding. Guided discovery, modeling, and definition of group roles and processes were minimal. Despite these missing support structures, students in the DP1 group wrote postunit essays that were significantly stronger statistically than the non-DP group in both factual knowledge and persuasive and dialectical reasoning. Although aggregate DP1 essay scores were superior to those of their peers, the overall performance of the DP1 class did not demonstrate a high degree of critical reasoning. DP1 group performances on collaborative multimedia presentations of the unit problem were superficial and lacked evidence of critical reasoning. However, analysis of observations, surveys, and interviews suggested that DP1 students were more engaged with the content and developed more empathetic views of historical dilemmas than their non-DP peers (Saye & Brush, 1999).

Purpose of the Current Study

Findings from DP1 suggested that multimedia learning environments such as DP had considerable potential for encouraging engagement and empathy. Findings for improving disciplined inquiry were much more mixed. We centered our work in this second phase of our research (DP2) on discovering what kinds of scaffolding might support the inquiry process. We focused on improving collaborative group performances as measured by culminating multimedia presentations. Not only were group performances the area of greatest weakness in DP1, several issues emerging from the research literature recommended the culminating presentation as the focus for this phase of our inquiry: First, we sought to help students view knowledge as tentative, a prerequisite to developing critical


83


reasoning. The presentation required students to author a narrative. Theorists have suggested that such authoring encourages students to recognize the tentativity of knowledge claims (e.g., Greene, 1994). Second, we wished to improve student use of standards of disciplined inquiry to evaluate evidence and construct better reasoned narratives. As a group reasoning task, the presentation should encourage the process of collective rationality, which would allow students to reason at higher levels (Shulman & Carey, 1984). Since this did not appear to be the case in DP1, we wished to experiment with new scaffolds in addition to those used in DP1 to see if they might address identified group difficulties. We present a detailed description of these scaffolding changes in the Method section.

METHOD

Study Design

Our research can best be described as a design experiment. Design experiments view innovative teaching as an experiment occurring in the “multiply confounded” world of real classrooms rather than controlled environments (Brown, 1992). From this perspective, innovative educational environments may be simultaneously designed, taught, and studied. The results from our earlier DP1 intervention suggested design modifications that might make the DP learning environment more effective. In order to evaluate the impact of these modifications, we employed both qualitative and quantitative methods to study our new case: the DP2 class. Some evaluation questions may be addressed deductively. Others are best left open to inductive analyses based on direct observations (Patton, 1987). Given the theoretical premises and the iterative design of the DP environment, we could describe in advance some desirable outcomes and develop evaluations to see if they seemed to occur. Qualitative methods allowed us to explore possible explanations for any observed effects.

Setting and Participants for DP2

Our field study took place in the year following DP1 in the same setting and with the same teacher. We focused on one DP class and compared its group performances to those of the Year 1 DP class. The study high school was located in a small southeastern city that is home to a land-grant university. The school enrollment averaged 1,250 students (68% White, 28% Black, and 4% Asian). The median family income in the school district was $35,876. Three years before the present study, the social studies department moved into new classrooms equipped with sets of five networked computers. The following year the school moved to 97-min block classes. The principal encouraged teachers to incorporate technology into lessons and to include at least three different activities in each of the new, lengthier periods. Our study teacher was an 18-year veteran with a Master of Arts in Teaching in history. She had worked with preservice teachers for several years in the laboratory-experiences program directed by one of the researchers. Observations of her classroom over that time suggested that teacher-directed lecture, recitation, and individual seatwork were her most commonly used instructional strategies. Although she described herself as “teacher-oriented” and “not a hands-on kind of teacher,” she had begun experimenting with more student-centered instruction, largely because the administration had urged teachers to “really work on cooperative learning.” However, before our work in DP1, neither she nor her students had past experience or training in the principles of cooperative learning. She was skeptical about the effectiveness of student-centered learning and reported uncertainty about how to integrate technology into her teaching. She viewed our partnership as “a learning time” that might help her to adapt her practice to the new tools, time frame, and instructional methods with which she was presented. Her experiences in DP1 raised her appraisal of the potential of in-depth learning and caused her to reconsider the amount of guidance necessary for effective student-centered learning. The teacher’s evolving perspective on prob-


84


lem-centered learning and how her perspective affects her pedagogical decisions are important to a full understanding of our longterm curriculum development efforts. However, exploration of these complex issues requires a deep analysis that goes beyond the boundaries of this paper. This report focuses on students and refers only tangentially to the teacher’s perspective. As was true of the students in the DP1 study, the 18 students who took part in the DP2 study formed an intact class of 11th grade U.S. history. The class was required of all students who were not enrolled in honors history.

Changes to the Learning Environment for the DP2 Study

For the DP2 iteration, we attempted to improve scaffolding support for student inquiry in both the data-gathering and decision-making phases of the unit (Table 1). DP1 performances suggested that despite introductory essays and other conceptual organizers, students had great difficulty with an important facet of historical inquiry: placing evidence within a larger context (Wineburg, 1991b). In DP1, students had encountered database documents through menus that grouped items associated with a particular

Table 1

event by item type, such as “newspaper accounts.” In DP2, we made hyperlinks directly from terms in introductory essays to related documents in the database. Each highlighted term connected the user to a pertinent primary document, video clip, or interactive essay. We hypothesized that linking data to specific discussions in the introductory essays would allow students to encounter evidence within the sort of framing context that a more expert researcher might possess. Student groups used the DP presentation tool to produce multimedia slideshows supporting their solutions to the unit problem. We hypothesized that explicit hard scaffolding could assist groups in using evidence to make more disciplined arguments in these presentations. We developed and modeled such a scaffold: a storyboarding process for considering alternative solutions and planning the scope and,sequence of the final product. The storyboard provided a six-page template that corresponded to the six slides that groups were expected to include in their presentations. Each page guided students through a step in the process of constructing a persuasive, dialectical argument. Page prompts directed students to describe the problem, present two possible actions to address the problem, examine possible positive and

Summary of DP2 design modifications based on DP1 findings. DP1 Finding

DP2 Design Modification

Students had difficulty situating evidence within a larger conceptual context

Hard Scaffold: Hyperlink terms in introductory essays to related primary documents Soft Scaffold: Proposed regular teacher meetings with cooperative groups during data gathering

Group presentations lacked depth and failed to demonstrate critical reasoning

Hard Scaffold: Storyboard template and model storyboard demonstrating persuasive and dialectical reasoning Hard Scaffold: Teacher assessment rubric assigns greater weight to process and group outcomes Soft Scaffold: Proposed regular teacher meetings with cooperative groups during decision making Soft Scaffold: Proposed final debriefing of the problem following group presentations

Teacher had difficulty with soft caffolding sand with facilitating collaborative learning.

Hard Scaffold: Use first drafts of storyboard to preview student thinking prior to soft scaffolding


85


Figure 2

Multimedia Presentation. Screen shows one slide from work in progress on a culminating presentation. Contextural tool bars visible at top and bottom of the screen help students to build an interactive presentation incorporating multimedia data from the program.

negative consequences of each action, and defend their chosen action as the best solution to the problem. Groups were instructed to present best case and worst case arguments for pursing each of the two options and consider carefully and respond to counterarguments in defending their conclusion. At each point in the presentation, students were instructed to use historical evidence to support their claims (Figure 2). Storyboard pages featured three sections for students to complete: a space for the text that would show on the screen, a space listing the multimedia links that students would use to support that portion of the presentation, and a space for the dialogue that would be presented while the slide was displayed. In the dialogue section, students provided annotations to show the database sources for their claims. We gave students a completed storyboard that modeled

the use of persuasive and dialectical reasoning to arrive at a problem solution, and we demonstrated the presentation that we had constructed based on that storyboard (Figure 3). During our DP1 debriefing sessions and DP2 planning sessions with the teacher, we discussed possibilities for how the features of the DP multimedia environment might be used more effectively and answered questions raised by the teacher. We also discussed ideas for improving student inquiry through soft scaffolding. Three major ideas emerged regarding adjustments the teacher might make to support student efforts. We agreed that the teacher would hold regular meetings with the cooperative groups to monitor and probe their thinking as they collected and synthesized evidence. We discussed how a teacher-led comparison of the findings reported in the group presentations


86 Figure 3


Presentation Storyboard. Excerpt from one page in the model storyboard given to students.

might help students grapple more comprehensively with the complexity of the problem. Finally, we agreed that the teacher would require groups to submit rough drafts of storyboards for feedback before each group constructed its final presentation. Although the storyboard was a hard scaffold for students, we also conceptualized the rough draft process as a teacher support that might facilitate her soft scaffolding of student thinking. Our partner teacher made changes of her own that suggested shifts in her pedagogical

thinking. She placed greater weight on the assessment of process and group outcomes than in DP1. She spent time prior to the DP2 unit establishing cooperative norms because she discovered “they don’t have some of the skills they need to work in groups. I just assumed juniors would be able to do this. I guess I never expected before last year to use them (teambuilding activities).” Although she focused more on group work, student role assignments remained sketchy and demanded little positive interdependence.



Data Collection and Analysis for DP2

We collected four types of data. (a) Outcomes data on culminating group presentations provided a complex measure of student reasoning. (b) Data from student interviews, (c) student paths through the database, and (d) classroom observations expanded our understanding of DP2 group presentations outcomes. Group presentations. As in DP1, a seven-standard rubric was used to assess the quality of group presentations of their resolutions of the unit problem. Our scoring rubric assessed competency in meeting seven standards. Standards A and B assessed abilities to locate necessary information and construct a synthesis that connected data in meaningful patterns. Standards C and D gauged reasoning. Standard C measured whether students were able to reason dialectically about the problem solution. Standard D focused on persuasive reasoning by requiring students to support their chosen position with evidence. Standard E measured the clarity with which groups presented evidence and arguments in a connected narrative. Standard F was more procedural, assessing how well groups used the DP resources to locate, access, and manage data. Standard G assessed the effectiveness of the presentation more holistically. Each rubric standard featured a detailed set of descriptors for scoring at four levels of performance: exemplary (4), competent (3), marginal (2), and poor (1). The researchers scored presentations independently. Overall interscorer agreement was 93%. Discussion regarding scoring discrepancies brought agreement for final presentation rankings to 100%. DP2 group presentation scores were compared to those in DP1 to gauge differences in the overall performances of the two classes. Interviews and observations. We used a purposive selection strategy that sought diversity in gender, race, ethnicity, and ability and motivation in the subject area (as perceived by the teacher) to select six students from the case study class for 45-min postunit interviews. Each of the four presentation groups was represented

87 among the interview respondents. Interviews solicited student perspectives on the DP experience and how the resources and scaffolding supports affected their learning. Interviews were audiotaped and transcribed. We kept field notes of classroom observations of each day of the unit and corroborated those with the teacher’s observations and impressions. Paths through the database. “Log files” (Barab, Young, & Wang, 1999; Barab, Bowdish, & Lawless, 1997) were created as students navigated through the DP database. Each time a document was accessed within the database, a notation was added to the log file. Analysis of these files allowed us to determine which documents each group viewed, the order in which they viewed them, and the length of time that students spent with each document. We triangulated analyses of the latter three types of data to explore possible explanations for student presentation outcomes. We entered the DP2 phase of our inquiry with hypotheses about how certain interventions might improve outcomes. We used a form of analytic induction to test our hypotheses by examining the data to see if it fit our tentative explanations. Data from interviews and observations were coded and grouped into conceptual categories that addressed possible factors affecting student performance. This analysis included a purposeful search for confirming and disconfirming evidence regarding our initial hypotheses and for other evidence that could not subsumed under those hypotheses. Records of student data paths provided a third data stream for testing our conclusions. The results of these analyses were used to refine and expand our hypotheses (Goetz & LeCompte, 1984). The work we have done thus far is exploratory and has a number of limitations. Our findings are not generalizable beyond the study population. Furthermore, the classes under study cannot be compared in a standard statistical sense. However, we believe that a sustained line of inquiry in one setting offers useful insight for problem-based curriculum development. We cannot claim that a specific intervention led to any observed effects. However, we can triangulate data from multiple sources and


88


methodologies and offer a series of data-based speculations regarding how the changes in the DP learning experience may be affecting outcomes. Hypotheses generated from these studies may be tested with other subjects in other settings to broaden this line of inquiry.

FINDINGS AND DISCUSSION

In the DP2 implementation, the teacher continued to struggle with soft scaffolding support. She reviewed storyboards and provided brief feedback to groups. She reported that the storyboards helped her with feedback because they “gave me something to come back to with them.” However, her feedback addressed issues of factual error more than student reasoning. During data collection she held brief small group meetings to clarify students’ procedural understanding of unit tasks. However, she did not use the meetings to monitor and probe thinking as students attempted to synthesize data. For instance, in a meeting with one group she asked: “What are you doing?” “What do you not understand about what you are doing?” and closed with the admonition to “Be sure you can back up what you say with evidence.” As she circulated through the room while groups worked, the teacher provided guidance for student thinking only when a student asked directly for assistance. She did not hold debriefings to synthesize unit work after presentations were concluded.

Performances on Group Presentations

Although soft scaffolding remained meager, DP2 group presentations were more competent that those made in DP1. DP2 mean scores were higher for each of the seven rubric standards than in DP1, and the lowest cumulative mean score for a DP2 group was very close to that of the highest scoring DP1 group. Only one of four DP2 groups achieved a 3 (competent) ranking in its cumulative mean score. However, three DP2 groups demonstrated competence in using historical evidence to construct a coherent narrative. Two groups made persuasive arguments

for their case, and two groups presented competent dialectical views of the problem (Table 2). We focus our discussion in this report on those standards most closely related to using evidence to make persuasive and dialectical arguments, Standards A–E. Gathering, interpreting, and synthesizing evidence. As in DP1, DP2 groups did not demonstrate deep knowledge of a wide range of information about the civil rights movement. However, DP2 performances were better than those of the DP1 groups. For Standard A (locating information about an event) we used a low threshold for a competent (3) score: Groups had to present accurate information with no major omissions or misstatements. DP1 groups had not met this lenient standard, but three of four DP2 groups

Table 2

Standard

Group presentation assessment rubric scores Group 1

DP1: 1997–1998 A 2 B 2 C 1 D 1 E 1 F 2 G 1 Group Mean 1.43

DP2: 1998–1999 A 2 B 2 C 1 D 2 E 2 F 3 G 2 Group Mean 2.00

Group 2

Group Group 3 4

Item Mean

2 2 2 2 2 3 2

2 1 1 1 1 2 1

2 2 1 1 1 2 1

2 1.75 1.25 1.25 1.25 2.25 1.25

2.14

1.29

1.43

1.57

3 3 3 2 3 3 2

3 4 4 4 3 3 4

3 2 2 4 3 3 3

2.75 2.75 2.50 3.00 3.00 3.00 2.75

2.71

3.71

2.86

2.82

Note: Rating descriptors: 4 = exemplary; 3 = competent; 2 = marginal; 1 = poor.



demonstrated at least this minimally competent command of pertinent information. Two DP2 groups demonstrated adequate interpretation and synthesis of the information they had collected (Standard B). The creativity of the synthesis separated Group 3’s exemplary score from the competent ranking for Group 2. Reasoning with evidence. Two of four groups demonstrated some dialectical understanding of multiple viewpoints (Standard C). Two groups made exemplary persuasive arguments (Standard D). Only Group 3 demonstrated competence in both dialectical and persuasive reasoning. To assess dialectical reasoning, we focused first on whether students could demonstrate empathetic reasoning, a genuine understanding for opposing views about the problem. To score a 2, groups had to present two genuine options. A competent (3) score required demonstration of empathy. Groups had to present the basic arguments for each option in a balanced, unbiased manner. We considered competent performances to demonstrate an incipient form of dialectical reasoning because they interrogated each position by examining pro and con arguments for each alternative. However, those interrogations were internal to the presentation of each option rather than part of a group’s problem resolution. At the exemplary (4) level, groups were expected not only to present balanced arguments for accepting and rejecting both positions but also to address alternative perspectives when they defended their decision as the best solution to the problem. Exemplary scores for Standard C demanded persuasive reasoning as well because each argument had to be convincing. The highest scoring DP2 team, Group 3, offered two genuine options for action and presented convincing arguments for both choices. They showed an understanding of good and bad consequences that might result from either action and then made a persuasive argument that their choice was the most likely to succeed. One option was the creation of a hybrid group, the Black Crusaders for Freedom, who would pursue nonviolent actions for integrating society (but would not passively submit to violence) while also building independent Black Power bases. They used examples of the Student

89 Non-violent Coordinating Committee (SNCC) working with both the integrationist Southern Christian Leadership Conference (SCLC) and the separatist Black Panthers to show that groups with different emphases could work together for common goals. They presented a video of M.L. King (SCLC) and Stokely Carmichael (SNCC) disagreeing publicly about strategy to support the argument that such coalitions are difficult to maintain. However, like all other groups, the depth of their views was limited. For instance, they did not address the fact that SNCC was able to work with the Panthers only after they had abandoned the goals of SCLC. The group chose the Crusaders strategy over their second option, the creation of a new, integrated Equalitarian Party dedicated to a single issue: enacting laws and policies that would insure equal treatment of all people. They pointed to instances where political pressure had been brought upon political leaders to pass laws for more social justice, such as the 1965 Voting Rights Act. However, they argued that a new party was unlikely to gain enough power because politicians would protect their own power and put self-interest before equality. They used evidence such as the Democratic Party’s failure to seat the Mississippi Freedom Democratic Party at its 1964 Convention to support their argument. In defending their choice, they argued that such self-interest made it necessary for Blacks to depend upon themselves to gain equality. Group 2 made a dialectical argument, but did not demonstrate persuasive reasoning in defending its choice. Group 2 students presented two genuine options, but their arguments were thin. Their chosen position argued that organizing all Christians for another march on Washington would lead to national unity and equality. In making their case, they occasionally misinterpreted evidence or used it out of context. For example, they showed speeches by King and John Lewis at the Washington march to demonstrate the power of such rallies to unify people behind a cause. They treated the speeches as complementary messages, failing to differentiate the anger and frustration of Lewis’s message from the optimism of King’s. Evidence


90 provided to support their chosen option actually reflected less understanding of the historical evidence and its implications than did the supporting argument that they provided for the option that they rejected: encouraging nationwide riots to pressure Congress into creating a Blackmajority state. Group 4 made a persuasive argument, but failed to demonstrate dialectical reasoning. This group favored a more moderate form of Black Power that might attract support from advocates of nonviolence. They supported their argument with strong, nuanced evidence that the threat of Black retaliation for violence might reduce violence by Whites, decrease Black disillusionment, and make nonviolent methods more pervasive and effective. However, they did not present a genuine second option. Instead, they simply recounted a historical event, Mississippi Freedom, Summer, without proposing a specific action or drawing implications from the past event about what path to take in 1968. Developing a clear narrative. The most consistent improvement over DP1 performances was evident in the success of three of four DP2 groups to produce coherent, connected narrative storylines (Standard E). Compared to DP1 presentations, DP2 group presentations were more creative and better conceptualized and executed. Although not yet expert, their presentations were more authentic and less “school-like” than their predecessors. In the strongest, most authentic DP2 presentation, group members framed their presentation as an actual press conference in which all members assumed the roles of interested parties.

Possible Influences of Scaffolds on Performances

Analysis of data from observations, interviews, and student paths through the database suggest that both the storyboard scaffolding process and the hyperlinked contextual essays may have affected the quality of presentation outcomes. We had hypothesized that the storyboard would help students use evidence more effectively to make more well-reasoned arguments. In con-


trast to DP1 where groups spent no more than 13 min making a decision and planning presentations, DP2 groups worked with the storyboard planning scaffold for a minimum of 40 min before logging on to computers to begin assembling media for their on-line presentations. Most respondents found that the storyboard scaffolding was helpful in conceptualizing presentations. Student 6 explained, “When you presented and [demonstrated] how to go through it, that helped a lot, cause [sic] that helped us see how to do it.” Student 3 rated the presentation as her favorite aspect of the project: That was the best presentation thing I have ever done . . . we had to write it down and actually show what we learned . . . it was an easier process [than previous presentations] . . . The handouts really went good with it because it let you know, and it give [sic] you insight. Just like: step one in the process . . . At first, I didn’t understand, but as y’all gave us more handouts and talked to us more, I understood more clearly.

However, Student 4 found the presentation the least enjoyable part of the unit: “Actually putting it [the presentation] into the computer was easier; . . . trying to find what stuff we wanted in there was the hardest.” When asked how the process could be made easier for him, he suggested more soft scaffolding while students were in the process of developing their storyboards. “She [the teacher] could look at our work draft, and then decide whether we put too much or too little in . . . give you, like, little hints right there . . . that would help it go along smoothly.” Observations of the class at work support this student’s assertion that more timely soft scaffolding might improve the quality of student constructions. Some DP2 groups demonstrated greater thoughtfulness in synthesizing evidence than was evident for DP1 groups but continued to struggle to develop realistic problem solutions that reflected the complexity of social reality. For instance, one group carefully considered the prospects of working through legal channels to bring about change: Student 1: Well, Lowndes County Freedom Organization didn’t work because Whites fixed the elections. Student 2: I’m not sure about Little Rock either. I think



it is a con [in the process of weighing pros & cons for using that strategy]. What happened there shows using the legal system didn’t work.

However, when this group completed their discussion, they failed to factor complicating evidence into their decision about the problem. They concluded that the best solution was “to expose kids earlier to tolerance in their education.” The potential and the shortcomings in reasoning suggested by these observations illustrate the centrality of soft scaffolding for developing critical reasoning. If the teacher had held Socratic small group meetings during the decision-making phase, she might have helped this group refine its decisions as they were being formed with such questions as: “How will you expose them to tolerance? Will you force it with new laws? What happened the last time something like that was tried?” Analysis of student paths through the database revealed that DP2 students visited many more of the introductory essays (80%) than the DP1 group had done (40%). The DP2 class used the contextual links embedded in the essays to explore the database almost 40% of the time as opposed to a linear exploration of the documents in the order they were listed in event menus (the only method available to DP1 students). We hypothesized that encountering the documents within the framing context of the essays would allow those students who used that strategy to see more relationships between documents and the surrounding flow of events. Student interviews support this proposition. Student 1 used an example from her favorite media, music, to illustrate the deeper understanding she gained from contextualized documents: “A lot of the songs I knew, but when I heard it and actually learned about it, it became clearer. Just like . . . James Brown, that became much clearer when I actually studied it and could put it with something.” Student 6 explained, “If there was something I had a question about [while reading an essay], I would look at a link.” This student admitted that her group did not access :most documents from the links in the essays, but concluded that the links “did, actually [help the documents make more sense], and we probably should have done it that way, but we thought it was faster to do it the

91 other way [using the menu of documents listed by type].” Observers noted numerous other occasions when groups failed to use available scaffolding support. Researchers have noted this phenomenon in other studies as well (Land, 2000; Oliver, 1999). Such behaviors make it difficult to assess accurately the potential effectiveness of hard scaffolding. Soft scaffolding by the teacher might reduce these omissions by questioning students about understandings generated from use of embedded scaffolds. Finally, we had hypothesized that a combination of scaffolds might help students confront competing logics and the tentativity of knowledge claims and that these encounters might encourage more authentic historical narratives. Historical narratives are constructed by reconfiguring historical documents into a plot (Holt, 1990). In the expository tradition, students receive a completed product: an expert narrative. The construction process remains invisible, as does the possibility for alternative narratives. We hoped that scaffolding could introduce students to the process of historical thinking: the dialogue between evidence and plot. With these insights into how history was constructed, we hoped students might feel greater license for construction and more ownership of the evidence supporting their own narratives. In constructing the DP database, we had carefully included alternative narratives. However, we could not be sure if students would encounter them in contexts that would cause them to wrestle with their competing logics. In DP2, we included hyperlinks in the interactive essays that purposefully confronted students with competing perspectives on events. As they constructed their presentation narratives, the storyboard scaffold required students to juxtapose evidence supporting and opposing proposed actions. Our model presentation demonstrated how this might be done. DP2 groups did demonstrate a greater ability than their predecessors to author original, authentic presentations that integrated evidence into clear, connected storylines. Several students remarked on their encounters with alternative narratives in the DP2 unit. Their reactions support claims that interaction with diverse ac-


92 counts may push students to consider the tentative, interpretive nature of historical claims (e.g. Greene, 1994; Wineburg & Wilson, 1991). Student 5 was surprised at how his perspective was altered by his encounters with primary documents: Like, if I look at some of the chapters that we [studied] and I saw some of the pictures, then maybe I wouldn’t have thought differently about a certain thing, you know, cause [sic] if . . . she [the teacher] was just talking about the Black Panther Party, I would have thought they were just like all about violence . . . I wouldn’t have thought that, “dang, they just bear all the weight!” But you know, with me seeing the picture of the actual people and what they went through and the Breakfast Program and everything, I, you know, totally changed my view about what they were all about. I know even though they were about violence, but there was some good aspects to what they were, their cause, or whatever.

Student 6 found that diverse views enriched her understanding: You saw what the press saw and what everyone else in the world saw about it at the time, you know, and see what you think about it. Even though you can’t always, you know, tell if it’s true . . . you just kind of got to see that story from all different sides . . . cause usually we just go by the side that’s in the book . . . . It’s more interesting, and I think it’s a little bit easier [to understand history], I guess, cause [sic] you can see it from all different sides.

Closely related to their experience with richer encounters, respondents expressed a sense of engagement with the content that suggested a stronger sense of ownership for the narratives they constructed. You really felt like you were there . . . in that time when you could really see in actuality like what happened instead of just hearing it from someone else. (Student 6) I don’t enjoy history that much because you just, like, you listen to the teachers and they just lecture you, and you have no idea. Like, in math you’re always doing all the problems and in science you’re always doing labs and stuff. In history you just never interact with it or anything, and doing this [the: DP unit], you got to do it yourself and look at it. (Student 2) [At the end of the unit] I really understood, . . . like, in my presentation, I had that down pat, you know, I didn’t have to look at the note cards . . . I knew it just by working on that project for those few days . . . I felt


so good, knowing we had a presentation, and getting up and speaking and telling about it. I felt relaxed using this. (Student 3)

CONCLUSIONS AND IMPLICATIONS

This report describes a continuing line of inquiry in one site. Its findings cannot be generalized. We offer support for several hypotheses regarding how learners and teachers engaged in problem-based inquiry might be supported, and discuss their implications for future design work. These propositions should be tested and elaborated with further research in other settings. First, our work supports other studies that suggest that expert guidance may be embedded into the environment to give students conceptual and strategic roadmaps that assist them in understanding the process of disciplined inquiry (e.g., Hannafin et al., 1999; Jacobsen et al., 1996; Perfetti et al., 1999). We hypothesize that scaffolded encounters with conflicting accounts encouraged students to reconceptualize the nature of historical narrative. Further, we hypothesize that explicit modeling of the storyboard gave students insight into the process of using evidence to construct a historical narrative. Other researchers have used “model explanations” to scaffold student thinking in various disciplines. For example, Sandoval and Reiser (1997), developed ExplanationConstructor to support student development of explanations for various scientific phenomena. A central component of their tool is a series of “explanation templates,” similar in concept to the storyboard templates used by students in this study. Despite additional scaffolding in DP?, students continued to have difficulty acquiring deep, broad views of the civil rights knowledge base. These difficulties may be related to the breadth of the problem presented to the students in the unit. We hypothesize that the problem landscape for the DP unit is too expansive for students to gain mastery in the time they have been given. In most classrooms, resources such as those available in DP might be used more effectively if initial problems explored by students were smaller and more bounded. Students par-



ticipating in both DP1 and DP2 had virtually no experience participating in problem-based curriculum units in their social studies classes. Jonassen (1997, p. 80) has stated that “problemsolving skill develops from experience in solving problems.” If this is the case, then engaging students with more bounded ill-structured problems in order to “build up” to the expansive social issues explored in the DP unit may be a more effective strategy.

93 than does support for strategic thinking. Providing spontaneous scaffolding responses demands tremendous concentration, energy, and mental agility. This daunting task may require teachers who possess abilities, dispositions, and beliefs that are atypical of the teaching population (Brown, 1992; Lortie, 1975, Saye, 1998, Shaver, 1996). Much work needs to be done investigating whether and how a critical mass of our teachers might be assisted in engaging in effective scaffolding.

For example, a series of historical dilemmas might be constructed that present interior monologues for figures who have difficult decisions to make (O’Reilly, 1998). For the civil rights movement, such a dilemma might be the school board chairman in Little Rock in 1957 having to decide whether to close the schools for the year. Hyperlinks embedded in each dialogue might direct students to documents in the DP database that present evidence for differing points of view that the figure should consider in coming to a decision. This design might help with the challenge of contextual and abstract thinking because the hyperlinked documents can connect students to the three-dimensional conceptual landscape of the dilemma that is difficult for nonexperts to envision (Spiro & Jehng, 1990). Finally, more bounded problems like this example may facilitate inquiry for teachers by reducing the amount of spontaneous scaffolding they must do in an ill-structured environment. Instead, they would have a more defined knowledge base to master and a more limited set of problems for which to plan.

Some hard scaffolds for students may serve as intermediate structures that support teachers in the task of soft scaffolding by creating time for reflection before their response is required. In this way, we scaffold teacher thinking by providing a thinking space between the student’s initial response and the teacher’s supporting response. We believe our storyboarding and presentation tools have potential as such devices. Loh et al. (2001) have developed a similar set of tools collectively entitled Progress Portfolio to allow science teachers to review student presentations in order to “guide students to focus on specific aspects of their investigative process, encouraging students to reflect on the rationale for plans taken, the fate of different solution strategies, or how the students’ ideas have evolved during the investigation.” (p. 291). Although interactions such as these may not meet all student needs for soft scaffolding support, they may be more workable for many teachers who find spontaneous scaffolding difficult or problematic.

Clearly there are limits to gains that may be achieved through hard scaffolds. Ill-structured social problems invite diverse, multilogical reasoning paths that cannot be anticipated and supported through hard scaffolding alone. Our greatest hard-scaffolding successes have been with strategic scaffolds that guide students through approaches they might employ to make more effective decisions. We have had less success in scaffolding high-level conceptual tasks such as synthesizing the evidence assembled by research teams and finding patterns in the data. We can improve hard scaffolding in this area, but student conceptual support needs are complex. Guiding conceptual thinking likely requires much more spontaneous soft scaffolding

More is necessary if teachers are to scaffold effectively. Recognizing that teachers must perceive practices as legitimate and necessary before they will adopt them, we have been reluctant to prescribe specific classroom behaviors to our teacher. We believe that changes in teaching behaviors are likely to be most effective if they are developed by teachers as solutions to felt problems. However, our work leads us to conclude that if problem-based learning is to flourish, teachers must be provided more active support in envisioning such practices and gaining expertise. We need to encourage expositoryoriented teachers to challenge their assumptions about knowledge. Beyond establishing a legitimate rationale for problem-based learning,


94 teachers need models of these practices. Technology may be particularly suited for providing support here. Digital video of teachers implementing different elements of problem-based practice along with teacher reflections on the challenges and solutions they have discovered might be embedded in an on-line teacher’s guide to using DP or similar learning environments (e.g., Barab, MaKinster, Moore, & Cunningham, 2001; Savenye, Brush, Middleton, Blocher, et al., 2002). In this study we sought to discover whether hard scaffolds built into multimedia-supported learning environments might lessen the cognitive burdens of disciplined inquiry so that teachers and learners might engage in more rigorous reasoning about social problems. Our findings leave us optimistic that embedded supports can provide such assistance. However, there are clearly limits to the gains that can be achieved by hard scaffolding improvements. Hannafin and colleagues (1999) emphasize that a learning environment is unlikely to be effective unless its core foundations and values are consistent psychologically, pedagogically, technologically, culturally, and pragmatically. Ultimately, it is the master teacher who must align all elements of the learning environment into a cohesive whole. Few practitioners have made such an alignment in favor of problembased learning or active student inquiry. To gauge realistically the prospects for problembased inquiry in social studies and other content areas, we need deep investigations of how such alignment might be encouraged and supported among teachers.

John W. Saye [[email protected]] is an associate professor in the Department of Curriculum and Teaching at Auburn University. Thomas Brush is an associate professor at Indiana University. Correspondence concerning this article should be addressed to John W. Saye, Associate Professor, Auburn University, 5040 Haley Center, Auburn University, AL 36849-5212; (334)844-6891; fax: (334)844-6789. Support for this research has been provided by the Corporation for Public Broadcasting, Auburn University College of Education, and Auburn City Schools.


REFERENCES Ashby, R., & Lee, P. (1987). Children’s concepts of empathy and understanding in history. In C. Portal (Ed.), The history curriculum for teachers (pp. 62–88). London: Falmer Press. Barba, S., Bowdish, B., & Lawless, K. (1997). Hypermedia navigation: Profiles of hypermedia users. Educational Technology Research and Development, 45(3), 23–41. Barab, S., MaKinster, J., Moore, J., & Cunningham, D. (2001). Designing and building an on-line community: The struggle to support sociability in the Internet learning forum. Educational Technology Research and Development, 49(4), 71–96. Barab, S., Young, M., & Wang, J. (1999). The effects of navigational and generative activities in hypertext learning on problem solving and comprehension. International Journal of Instructional Media 26(3), 282– 309. Berson, M.J. (1996). Effectiveness of computer technology in the social studies: A review of the literature. Journal of Research on Computing in Education, 28(4), 486–499. Brown, A.L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of Learning Sciences 2(2), 141–178. Brown, J.S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32–42. Brush, T., & Saye, J. (2000). Implementation and evaluation of a student-centered learning unit: A case study. Educational Technology Research and Development, 48(3), 79–100. Chi, M.T.H., Glaser, R., & Farr, M.J. (1988). The nature of expertise. Hillsdale, NJ: Lawrence Erlbaum. Choi, J., & Hannafin, M. (1995). Situated cognition and learning environments: Roles, structures, and implications for design. Educational Technology Research and Development, 43(2), 53–69. Cognition and Technology Group at Vanderbilt. (1992). The Jasper experiment: An exploration of issues in learning and instructional design. Educational Technology Research and Development, 40(1), 65–80. Cognition and Technology Group at Vanderbilt. (1993). Anchored instruction and situated cognition revisited. Educational Technology, 33(3), 52–70. Cuban, L. (1984). How teachers taught. New York: Longmans. Doyle, W., & Ponder, G. (1977–78). The practicality ethic in teacher decision-making. Interchange, 8(3), 1– 12. Dwyer, D. (1994). Apple classrooms of tomorrow: What we’ve learned. Educational Leadership, 51(7), 4– 10. Ehman, L.H., Glenn, A.D., Johnson, V., & White, C.S. (1998). Using computer databases in student problem solving. In J.A. Braun, P. Fernlund, & C.S. White (Eds.), Technology tools in the social studies curriculum



(pp. 164–187). Wilsonville, OR: Frankilin, Beedle. Goetz, J., & LeCompte, M. (1984). Ethnography and qualitative design in educational research. San Diego: Academic Press. Greene, S. (1994). Students as authors in the study of history. In G. Leinhardt, J. Beck, & C. Stainton (Eds.), Teaching and learning in history (pp. 137–170). Hannafin, M., Hill, J., & Land, S. (1997). Studentcentered learning and interactive multimedia: Status, issues, and implication. Contemporary Education, 68(2), 94–99. Hannafin, M., Land, S., & Oliver, K. (1999). Open learning environments: Foundations, methods, and models. In C. Reigeluth (Ed.), Instructional design theories and models (Vol. II). Mahway, NJ: Erlbaum. Holt, T. (1990). Thinking historically: Narrative, imagination, and understanding. New York: College Entrance Examination Board. Hynd, C., Hubbard, B., Holschuh, J., Reinking, D., Jacobson, M. (2000). Reading like a historian: Critical reading of multiple texts in a hypertext environment. Paper presented at the American Educational Research Association, New Orleans. Jacobsen, M.J., Maouiri, C., Mishra, P., & Kolar, C. (1996). Learning with hypertext learning environments: Theory, design and research. Journal of Educational Multimedia and Hypermedia, 5(3/4), 239–281. Jacobson, M., & Spiro, R. (1994). A framework for the contextual analysis of technology-based learning environments. Journal of Computing in Higher Education, 5(2), 3–32. Jonassen, D. (1997). Instructional design models for well-structured and ill-structured problem-solving learning outcomes. Educational Technology Research and Development, 45(1), 65–94. Kagan, D.M. (1993). Laura and Jim and what they taught me about the gap between educational theory and practice. Albany, NY: SUNY Press. Kinzie, M., & Sullivan, H. (1989). Continuing motivation, learner control, and CAI. Educational Technology Research and Development, 37(2), 5–14. Kuhn, D. (1999). A developmental model of critical thinking. Educational Researcher, 28(2), 16–26, 46. Land, S.M. (2000). Cognitive requirements for learning in open-ended learning environments. Educational Technology Research and Development, 48(3), 61–78. Land, S.M., & Hannafin, M. (1996). Patterns of understanding with open-ended learning environments: A qualitative study. Educational Technology Research and Development, 45(2), 47–73. Linn, M. (1995). Designing computer learning environments for engineering and computer science: The scaffolded knowledge integration framework. Journal of Science Education and Technology, 4(2), 103–126. Linn, M., Shear, L., Bell, P., & Slotta, J.D. (1999). Organizing principles for science education partnerships: Case studies of students’ learning about ‘rats in space’ and ‘deformed frogs.’ Educational Technology Research and Development, 47(2), 61–84. Loh, B., Reiser, B., Radinsky, J., Edelson, D., Gomez, L.,

95 & Marshall, S. (2001). Developing reflective inquiry practices: A case study of software, the teacher, and students. In K. Crowley, C. Schunn, & T. Okada (Eds.), Designing for science: Implications from everyday, classroom, and professional settings. Mahwah, NJ: Lawrence Erlbaum. Lortie, D. (1975). School-teacher: A sociological study. Chicago: Univ. of Chicago Press. NCSS. (1994). Expectations of excellence: Curriculum standards for the social studies. Washington: National Council for the Social Studies. Newmann, F.M. (1990). A test of higher-order thinking in social studies: Persuasive writing on constitutional issues using the NAEP approach. Social Education, 54(6), 369. Newmann, F.M. (1991). Higher order thinking in the teaching of social studies: Connections between theory and practice. In J. Voss, D. Perkins, & J. Segal (Eds.), Informal reasoning and education (pp. 381–400). Hillsdale, NJ: Lawrence Erlbaum. Newmann, F.M., Wehlage, G.G., & Lamborn, S.D. (1992). The significance and sources of student engagement. In F. Newmann (Ed.), Student engagement and achievement in American secondary schools (pp. 11–39). New York: Teachers College Press. Oliver, K. (1999). Computer-based tools in support of Internet-based problem solving. Paper presented at the annual meeting of the Association for Educational Communications and Technology. St. Louis, MO. Onosko, J. (1991). Barriers to the promotion of higher order thinking in social studies. Theory and Research in Social Education, 19(4), 341–366. O’Reilly, K. (1998). What would you do? Social Education, 62(1), 48. Palincsar, A. & Brown, A. (1984). Reciprocal teaching and comprehension-fostering and monitoring activities. Cognition & Instruction, 1(2), 117–175. Parker, W.C., Mueller, M., & Wendling, L. (1989). Critical reasoning on civic issues. Theory and Research in Social Education, 17(1), 7–32. Patton, M.Q. (1987). How to use qualitative methods in evaluation. Newbury Park, CA: Sage. Perfetti, C.A., Britt, M.A., Van Dyke, J., & Gabrys, G. (1999). The sourcer’s apprentice: A program of research and development in history instruction. Paper presented at the American Educational Research Association, Montreal. Perkins, D.N., Allen, R., & Hafner, J. (1983). Difficulties in everyday reasoning. In W. Maxwell (Ed.), Thinking: The expanding frontier (pp. 177–189). Philadelphia: Franklin Institute. Rossi, J.A. (1995). In-depth study in an issues-centered social studies classroom. Theory and Research in Social Education, 23(2), 87–120. Rossi, J.A., & Pace, C.M. (1998). Issues-centered instruction with low achieving high school students: The dilemmas of two teachers. Theory and Research in Social Education, 26(3), 380–409. Sandoval, W., & Reiser, B. (1997, March). Evolving explanations in high school biology. Paper presented at


96 the annual meeting of the American Educational Research Association, Chicago, IL. Savenye, W., Brush, T., Middleton, J., Blocher, M., and others. (2002, April). Improving teaching with technology: The “best practices” digital video project. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Saye, J.W. (1997). Technology and educational empowerment: Students’ perspectives. Educational Technology Research and Development, 45(2), 5–26. Saye, J.W. (1998). Technology in the classroom: The role of dispositions in teacher gatekeeping. Journal of Curriculum & Supervision, 13(3), 210–234. Saye, J.W. & Brush, T. (1999). Student engagement with social issues in a multimedia-supported learning environment. Theory and Research in Social Education 27(4), 472–504. Scheurman, G., & Newmann, F.M. (1998). Authentic intellectual work in social studies: putting performance before pedagogy. Social Education, 62(1), 21– 35. Shaver, J.P. (1996). The prospects for issues-centered education. In R.W. Evans & D.W. Saxe (Eds.), Handbook on teaching social issues (pp. 380–386). Washington: National Council for the Social Studies. Shaver, J.P., Davis, O.L., & Helburn, S.W. (1979). The status of social studies education: Impressions from three NSF studies. Social Education, 4(3), 150–153. Shulman, L.S. & Carey, N.B. (1984). Psychology and the limitations of individual rationality: Implica-


tions for the study of reasoning and civility. Review of Educational Research, 54, 501–524. Spiro, R.J., & Jehng, J.C. (1990). Cognitive flexibility and hy[ertext: Theory and technology for the nonlinear and multidimensional traversal of complex subject matter. In D. Nix & R. Spiro (Eds.), Cognition, education, and multimedia (pp. 163–205). Hillsdale, NJ: Erlbaum. Spoehr, K., & Spoehr, L. (1994). Learning to think historically. Educational Psychologist, 29(2), 71–77. VanSickle, R.L., & Hoge, J.D. (1991). Higher cognitive thinking skills in social studies: Concepts and critiques. Theory and Research in Social Education, 19(2), 152–172. Voss, J.F., Greene, T.R., Post, T.A., & Penner, B.C. (1983). Problem solving skill in the social sciences. The psychology of learning and motivation: Advances in research and theory. New York: Academic. Wineburg, S.S. (1991a). On the reading of historical texts: Notes on the breach between the school and academy. American Educational Research Journal, 28(3), 495–519. Wineburg, S.S. (1991b). Historical problem solving: A study of cognitive processes used in the evaluation of documentary and pictorial evidence. Journal of Educational Psychology, 83(1), 73–87. Wineburg, S.S., & Wilson, S.M. (1991). Subject-matter knowledge in the teaching of history. Advances in Research on Teaching, Vol. 2, 305–347.