J. EDUCATIONAL COMPUTING RESEARCH, Vol. 31(3) 215-245, 2004
CAN STUDENTS COLLABORATIVELY USE HYPERMEDIA TO LEARN SCIENCE? THE DYNAMICS OF SELFAND OTHER-REGULATORY PROCESSES IN AN ECOLOGY CLASSROOM*
ROGER AZEVEDO FIELDING I. WINTERS DANIEL C. MOOS University of Maryland
ABSTRACT
This classroom study examined the role of low-achieving students’ selfregulated learning (SRL) behaviors and their teacher’s scaffolding of SRL while using a Web-based water quality simulation environment to learn about ecological systems. Forty-nine 11th and 12th grade students learned about ecology and the effects of land use on water quality by collaboratively using the RiverWebSM water quality simulation (WQS) during a two-week curriculum on environmental science. The students’ emerging understanding was assessed using pretest and posttest scores. Students’ self-regulatory behaviors and teacher’s scaffolding of SRL were assessed through an analysis of their discourse during several collaborative problem-solving episodes. Findings indicate that students learned significantly more about ecology after working collaboratively with the WQS. However, these learning gains were quite small and were related to the self-regulatory behaviors observed in the dyads and their teacher’s scaffolding and instruction. Analyses of video data indicate that a large amount of time was spent by the dyads and teacher in using only a few strategies, while very little time was spent on planning, monitoring, and handling task difficulties and demands. Further analysis revealed that both the dyads and teacher were using low-level
*This research was supported by funding from the National Science Foundation (REC#0133346) awarded to the first author. 215 Ó 2004, Baywood Publishing Co., Inc.
216
/
AZEVEDO, WINTERS AND MOOS
strategies (e.g., following procedural tasks, evaluating the content, searching, and selecting informational sources in the WQS) to learn about the topic. Our results provide a valuable initial characterization of the complexity of selfand other-regulated learning in a complex, dynamic, technology-enhanced, student-centered science classroom. We discuss how the results will be used to inform the design of computers as MetaCognitive tools designed to foster students’ learning of conceptually challenging science topics.
INTRODUCTION Educational researchers have successfully used student-centered methods to enhance students’ understanding of science with computer-based learning environments (CBLEs) (e.g., Azevedo, Verona, & Cromley, 2001; Biswas, Schwartz, Bransford, & CTGV, 2001; Clark & Linn, 2003; Hickey, Kindfield, Horowitz, & Christie, 2003; Kozma, Chin, Russell, & Marx, 2000; Lajoie & Azevedo, 2000; Lajoie, Guerrera, Munsie, & Lavigne, 2001; Reiser et al., 2001; Slotta & Linn, 2000; Suthers & Hundhausen, 2003; Vye et al., 1998; White, Shimoda, & Frederiksen, 2000). These studies, however, also provide evidence that students have difficulties regulating certain aspects of their learning when they use CBLEs to learn about complex and challenging science topics such as the circulatory system, Newtonian physics, and genetics (e.g., Hickey et al., 2003; Jacobson & Archodidou, 2000). More specifically, research shows that students have difficulties regulating several aspects of their learning, including their cognitive resources (e.g., activating prior knowledge), motivation and affect (e.g., task value, self-efficacy, interest in the topic), behavior (e.g., engaging in help-seeking behavior), and the instructional context (e.g., negotiating task structure and dealing with dynamically changing structure of the instructional context). While the majority of these studies have examined specific aspects of students’ self-regulated learning (SRL) such as planning, metacognitive monitoring, and reflection in isolation (e.g., Lajoie et al., 2001), most have not examined the complex nature of the dynamics between the phases of SRL and how this complexity may affect how students regulate their learning of science topics in student-centered classrooms using CBLEs. It is therefore critical that we understand how students working collaboratively regulate their own learning as well as how teachers facilitate students’ SRL by using different scaffolding and instructional approaches in student-centered, technology-rich science classrooms. To that end, we have adopted and extended existing models of SRL (see Boekaerts, Pintrich, & Zeidner, 2000; Zimmerman & Schunk, 2001 for extensive reviews) and have used them as a lens to examine the complex interactions between students’ self-regulatory behavior and teachers’ scaffolding of their students’ SRL when using CBLEs collaboratively in the science classroom for extended periods of time.
SRL AND COLLABORATIVE LEARNING WITH A CBLE /
217
Theoretical Framework: Self-Regulated Learning By adopting SRL as framework, we make certain theoretical and empiricallybased assumptions about learning. Self-regulated learners are active learners who efficiently manage their own learning in many different ways (Boekaerts et al., 2000; Butler, 1998; Paris & Paris, 2001; Perry, 2002; Winne, 2001; Winne & Hadwin, 1998; Winne & Perry, 2000; Zimmerman & Schunk, 2001). Students are self-regulating to the degree that they are cognitively, motivationally, and behaviorally active participants in their learning process (Zimmerman, 2001). These students generate the thoughts, feelings, and actions necessary to attain their learning goals. Models of SRL describe a recursive cycle of cognitive activities central to learning and knowledge construction activities (e.g., Pintrich, 2000; Schunk, 2001; Winne, 2001; Winne & Hadwin, 1998; Zimmerman, 2001). Several models of SRL posit that SRL is an active, constructive process whereby learners set goals for their learning and then attempt to monitor, regulate, and control their cognition, motivation, and behavior in the service of those goals (see Pintrich, 2000; Zimmerman, 2001). SRL is guided and constrained by both personal characteristics and the contextual features of the environment (Pintrich, 2000). Thus, these models offer a comprehensive framework with which to examine how students learn and how they adapt during learning with hypermedia. Recent research has begun to examine the role of students’ ability to regulate several aspects of their cognition, motivation, and behavior during learning of complex science topics with hypermedia in laboratory experiments and in tutoring situations (e.g., Azevedo, Cromley, & Seibert, 2004; Azevedo, Guthrie, & Seibert, 2004). Thus far, this research has demonstrated that students have difficulties benefiting from hypermedia environments because they fail to engage in key mechanisms related to regulating their learning with hypermedia. Self-Regulated Learning with CBLEs In CBLEs such as Web-based hypermedia learning environments, students are given access to a wide range of information represented as text, graphics, animation, audio, and video, which is structured in a non-linear fashion (Jonassen, 1996). Learning in such an environment requires a learner to regulate his or her learning; that is, to make decisions about what to learn, how to learn it, how much to learn, how much time to spend on it, how to access other instructional materials, determine whether he or she understands the material, when to abandon and modify plans and strategies, and to increase effort (Shapiro & Niederhauser, 2004; Williams, 1996; Winne & Hadwin, 1998). Specifically, students need to analyze the learning situation, set meaningful learning goals, determine which strategies to use, assess whether the strategies are effective in meeting the learning goal, evaluate their emerging understanding of the topic, and determine whether the learning strategy is effective for a given learning goal. They need to monitor their understanding and modify their plans, goals, strategies, and
218
/
AZEVEDO, WINTERS AND MOOS
effort in relation to contextual conditions (e.g., cognitive, motivational, and task conditions) (Azevedo, Cromley, & Seibert, 2004). Further, depending on the learning task, they may need to reflect on the learning episode and modify their existing understanding of the topic (Winne, 2001; Winne & Hadwin, 1998). Therefore, despite the potential for fostering learning, hypermedia environments are proving to be ineffective unless learners can regulate their learning. Being able to regulate one’s own learning is therefore extremely important in enhancing one’s ability to learn about complex topics with hypermedia. Researchers have recently begun to identify several problems associated with learners’ inability to regulate their learning with hypermedia (Azevedo, Cromley, Winters, Xu, & Iny, 2003). The next step in this line of research is to extend these results by focusing on whether students can collaboratively use Web-based hypermedia learning environment to learn about the effects of land use on water quality indicators in a student-centered classroom by focusing on the complex interactions between students’ SRL. Self-Regulated Learning with CBLEs in the Classroom: Complex Contextual Interactions The importance of the classroom context for students’ self-regulated has been well-established (e.g., Butler, 1998; Meyer & Turner, 2002; Paris & Paris, 2001; Perry, 2002; Perry & VandeKamp, Mercer, & Nordby, 2002). Current research has focused on the importance of the classroom as a complex context within which to study students’ SRL and the role of teachers (as external self-regulating agents) who foster students’ understanding by using scaffolding or any other instructional methods to support their learning (e.g., Rasku-Puttonen, Etelapelto, Arvaja, & Hakkinen, 2003). However, what has not been investigated is the complex dynamics between students’ SRL while working collaboratively and using a CBLE to learn about a challenging science topic, and their teacher’s scaffolding of their SRL in the classroom. Meyer and Turner (2002) emphasized that in a classroom context there is a shared responsibility among teachers and students for establishing and maintaining relationships as they coordinate multiple goals through available opportunities and scaffolding (McCaslin & Hickey, 2001). Scaffolding is an instructional process in which the teacher supports students cognitively, motivationally, and emotionally in learning while helping further develop their autonomy (Meyer & Turner, 2002). During instructional scaffolding, the teacher supports the students’ self-regulation, as needed, in three ways: a) helping students build competence through increased understanding; b) engaging students in learning while supporting their socioemotional needs; and c) helping students build and exercise autonomy as learners. The interactions between a teacher and a class full of students with varying needs and abilities to regulate their learning while working collaboratively with a CBLE to learn about a challenging science topic
SRL AND COLLABORATIVE LEARNING WITH A CBLE /
219
pose theoretical and methodological challenges which need to be addressed in order to study whether students working collaboratively can use CBLE to learn about challenging topics. In this study, we address the question of whether academically low-achieving students, who have failed several science classes in high school, can regulate their learning of conceptually challenging science topics such as the effects of land use on water quality in a student-centered technology-enhanced science classroom. Our study contributes to a large body of literature on fostering students’ learning of conceptually challenging science topics with CBLEs in student-centered classrooms. It examines low-achieving students’ SRL and the teacher’s scaffolding of students’ SRL in a classroom setting. It converges product data (i.e., learning gains) and process data (i.e., time and frequency of use of several SRL variables used by students working collaboratively, as well as teacher scaffolding and instructional moves). This classroom study was conducted to investigate the following research questions: 1) Can low-achieving high school students effectively use a water-quality simulation1 (WQS) collaboratively to learn about water quality indicators and land use? 2) Do the students and teacher spend the same amount of time using various regulatory behaviors to facilitate students’ conceptual understanding? and 3) Do the students and teacher differ in the frequency of use of these self-regulatory behaviors to facilitate students’ conceptual understanding? METHOD Participants Forty-nine 11th and 12th grade high school students (22 girls and 27 boys) from four ecology science classes volunteered to participate in this study. Ages ranged from 16 to 18 years. The students came from diverse socioeconomic and racial backgrounds. Forty-nine percent (n = 24) were African American, 41% (n = 20) were Hispanic, 6% (n = 3) were Caucasian, and 4% (n = 2) were Asian. All of these students were placed in an environmental science class after failing several required science classes in high school. The students had limited exposure to environmental science, and therefore low prior knowledge about it. In addition, we observed that students lacked both general learning skills (e.g., comprehension monitoring) and specific science-related skills (e.g., how to round numbers). Data were collected in April and May 2003, one month prior to the end of the school year. 1
The original designers and developers of the RiverWeb WQS labeled it as a simulation-based environment, however, we consider it to be a Web-based hypermedia learning environment since it provides students access to multiple representations of scientific information in multiple formats including text, and diagrams, graphics in a non-linear fashion.
220
/
AZEVEDO, WINTERS AND MOOS
Research Design We used a one-group pretest-posttest design (49 students). Video and audio data were collected while the students worked in groups of two using the RiverWeb Water Quality Simulator (WQS) to learn about ecological systems. We collected process data on dyads of students from two of four classes taught by two environmental science teachers. Instructional Context In a typical high school environmental science course, a unit is devoted to water quality and the impact of land use on the pollutants that find their way into streams, rivers, and eventually, the ocean. In order for students to understand the differential impact of agricultural, commercial, urban, suburban, manufacturing, and foresting/lumbering land uses on water quality, students need to have some kind of experience with each. An ideal project for students in such an environmental science course would be to investigate how water quality changes across different locales; however, that would require field trips that are both expensive and time intensive and therefore often infeasible. The WQS is designed to provide students with a simulated field trip through a prototypical watershed. The RiverWebSM Water Quality Simulation (WQS) Environment The WQS is a Web-based environment (http://mvhs1.mbhs.edu/riverweb/ index.html) developed at Maryland Virtual High School and linked to the RiverWebSM Program originating from the National Center for Supercomputing Applications (NCSA). Targeted at the grades 8 through 12 science and math curriculum, the WQS depicts the effects of various land uses on water quality in an archetypal watershed. Students access WQS monitoring stations to explore how different land uses (i.e., a pristine forest, agricultural, lumbering forest, residential area, commercial/industrial area, wetlands, and urban area) affect water quality. Each water monitoring station allows students to test for physical and chemical characteristics of the tributary (termed indicators), such as total flow and nitrogen concentration. By limiting each sub-watershed to one land use, the effect of that land use can be seen on the quality of the water that students “test” within its boundaries. The cumulative effect of the combined land use determines the water quality shown by the indicator values found at a common river outflow. After the user logs in, a map of the archetypal watershed appears. Water quality monitoring stations located throughout the watershed are depicted on the map. The user may click on the map to investigate any sub-watershed using the WQS graph window. After clicking on a station, the student is presented with a default graph that displays the variation of nitrogen over time in the top window and precipitation over time in the bottom window. However, other indicators may
SRL AND COLLABORATIVE LEARNING WITH A CBLE /
221
be selected; for example, the student could compare different indicators at the same station such as water temperature and toxins (as shown in Figure 1), or could compare nitrogen concentration between two stations (e.g., residential area vs. wetlands). In addition, by reducing the range of days for each graph, the student can zoom in on a particular time period and investigate daily or seasonal variations. A tour, which may be selected at login, uses frames to combine the WQS with instructions leading the user through most of the simulator’s capabilities. Students can examine time series reports and scatterplots: using the time series graphs, students are able to see the impact of seasonal changes on certain indicators; using the scatterplots, students are able to discover correlations between variables, such as the relationship between water temperature and dissolved oxygen. The goal is to foster student understanding of the complex relationships among the many land uses, indicators, and water quality in the watershed. To gain this understanding, students are first given instruction in graph interpretation. They then answer open-ended questions, using the WQS, which demand explanations of concepts and relationships, and finally they construct concept maps. Data Sources and Measures Several data sources were used to obtain an in-depth understanding of students’ emerging understanding of science phenomena while using the WQS. A total of 6.12 hours of video and audio data were collected on four dyads in each of the two science classes during all online science inquiry sessions with WQS. This allowed for an in-depth analysis of students’ behavior as well as teacher instruction and scaffolding while students engaged in collaborative science inquiry activities with the WQS. The students’ emerging understanding and ability to regulate their learning was assessed through an analysis of student and student-teacher interactions during these activities. In addition to the video and audio data from student pairs, we also collected students’ pretests and posttests. The paper-and-pencil materials consisted of a pretest and a posttest, which were constructed by a consulting science teacher based on information presented in the WQS. The pretest and posttest, which were identical, included seven complex questions which required students to: 1) demonstrate understanding of runoff in relation to different water quality indicators and sources and effects of different indicators; 2) describe and hypothesize daily and seasonal variations depicted on several line diagrams; 3) determine relationships between different indicators at different land use areas using scatter plots; and 4) construct a concept map illustrating the relationships between water quality indicators and land use. The range of total possible points for each test was 0–62. The questions were designed to give each student an opportunity to demonstrate his or her understanding of various issues related to water quality. Question 1 was designed to determine if students understood whether pollutants found at the
222
/
AZEVEDO, WINTERS AND MOOS
Figure 1. RiverWeb interface illustrating a student comparing the variation in water temperature (top window) and toxins (bottom window) over time in the wetlands area (Station 5).
SRL AND COLLABORATIVE LEARNING WITH A CBLE /
223
mouth of a river come from runoff from land uses areas upstream (range 0–5). Question 2 was designed to assess whether students understood the importance of trees as a stream buffer, particularly in developed areas (range 0–6). Question 3 was designed to assess whether students understood how land use can cause an increase in runoff and, subsequently, more sediments and toxins in the water (range 0–6). Question 4 was designed to assess whether students understood that there is seasonal variation in these variables and why that might be (range 0–6). Question 5 was designed to assess two concepts; each scored separately, whether students understood that the scale on the y-axis of any graph must be included in the interpretation of the graph, and that land usage affects nitrogen concentration (range 0–9). Question 6 also had two parts, scored separately, and was designed to assess whether students understood that the scale on the x-axis of any graph must also be included in the interpretation of the graph and that both precipitation and land use affect runoff (range 0–8). In question 7, students had to draw a concept map to show that they understood how various factors affect water quality (range 0–22). With regard to format, questions 1, 2, and 3 were constructed-response questions. Questions 4 and 5 combined time series graph interpretation and constructed response. Question 6 was a scatter plot graph interpretation question, and question 7 asked the students to draw and label a concept map. Procedure For this study, students in the environmental science course spent two weeks using the WQS. For most of the students, this was their first experience learning about water quality. On the first day of week one, the students took a pretest to assess their understanding of water-quality indicators and land use. They were then introduced to the concept of water quality and engaged in mini-labs such as testing water for nitrogen or identifying the directionality and strength of relationships between graphed water-quality indicators. During the end of week one, the students were given a tour of the WQS and taught how to use the various tools to analyze data. During week two, the students participated in three “jigsaw” activities using the WQS. In Jigsaw 1, students used a time-series graph to compare and contrast two variables, and then compared the indicators in their land use area to those in the pristine forest. In Jigsaw 2, students used the WQS to learn about two assigned indicators in each land use area. In Jigsaw 3, students used scatter plots in the WQS to analyze the relationships of all the different indicators for their specific land use area. The groups then reunited and composed concept maps showing the links between the different indicators for each area and presented their maps to the whole class. During the final class period of week two, the teachers administered the posttest to all students. Performance data, in the form of pretests and posttests, were collected for all 49 students. Process data, in the form of videotapes and audiotapes, were collected
224
/
AZEVEDO, WINTERS AND MOOS
on two separate occasions over the two-week period from a sample of eight dyads from two classes. One consulting teacher acted as a complete participant during the data collection period. She is an experienced environmental science teacher who introduced the science activities to all of the students and provided scaffolding to individual student pairs during activities. The regular classroom teacher and the second author, a former science teacher, also provided scaffolding to students during their activities. The consulting teacher and the two classroom teachers were not instructed to foster students’ SRL nor were they asked to use specific SRL process (e.g., planning, creating sub-goals, metacognitive monitoring) during this naturalistic classroom study. The first author and several other graduate assistants acted as complete observers rather than participants in the classrooms, remaining on the sidelines to take notes and manage taping equipment, interacting minimally with the students and teachers during class periods. Taping was done for whole class periods during which the teachers moved in and out of interaction with individual groups as they tackled the tasks in the jigsaw assignments. Data were therefore gathered on groups of students while they worked both with and without teacher assistance. Coding and Scoring In this section, we describe the scoring of the students’ answers to the pretests and posttests, the coding scheme for student and teacher SRL behaviors, and inter-rater agreement measures. Students’ Answers to Pretest and Posttest Questions
The second author and the consulting teacher constructed a rubric for scoring the students’ responses to the pretest and posttest questions by initially scoring a subset (30%) of all the pretests and posttests. The rubric was refined and the two teachers scored all 98 tests individually using the revised rubric. Questions 1, 2, 3, and 7 were each given a single score; question 5 had two parts so it was given two scores; and questions 4 and 6 each had two parts so the two questions were given four scores each. This yielded 14 separate scores for each student’s pretest and posttest, which were combined to calculate each student’s overall (questions 1–7 on the pretest and posttest) score out of 62 for each test. Students’ and Teacher’s Regulatory Verbalizations and Behavior
The 6.12 hours (367 minutes) of audio and video tape recordings from 14 episodes provided the raw data for the analysis of students’ self-regulatory
SRL AND COLLABORATIVE LEARNING WITH A CBLE /
225
behavior and teacher’s scaffolding of SRL. Azevedo, Cromley, and Seibert’s (2004) original model of SRL was modified and used for analyzing the students’ and teacher’s regulatory behaviors and adapted to fit the use of the WQS in the classroom. Their model is based on several recent models of SRL (Pintrich, 2000; Winne, 2001; Winne & Hadwin, 1998; Zimmerman, 2001). It includes key elements of these models (i.e., Winne’s (2001) and Pintrich’s (2000) formulation of self-regulation as a four-phase process) and extends these key elements to capture the major phases of self-regulation in a complex technology-rich studentcentered science classroom. The classes, descriptions and examples of the planning, monitoring, strategy use, and task difficulty and demand variables used for coding the learners’ and teacher’s regulatory behavior are described in the Appendix. The classes are: a) planning and goal setting, activation of perceptions and knowledge of the task and context, and the self in relationship to the task; b) monitoring processes that represent metacognitive awareness of different aspects of the self, task, and context; and c) efforts to control and regulate different aspects of the self, task, and context; and d) various kinds of reactions and reflections on the self and the task and/or context. The model also includes behavior of the teacher when she provided scaffolding of and instruction in SRL-related behaviors in order to enhance students’ understanding. Teacher scaffolding refers to any teacherinitiated move that has the intention of facilitating students’ understanding through various self-regulatory behaviors. Teacher instruction refers to any teacher-initiated move that has the intention of specifically directing students’ SRL. For this study and in this context, the model was adapted to include verbal as well as non-verbal behaviors specific to the tasks students engaged in while learning with the WQS. As such, video data as well as audio data were used to provide evidence of teacher and student SRL behaviors. Inter-Rater Agreement
Each teacher scored each student’s questions on the pretest and posttest (i.e., 7 questions with a total of 14 scores × 49 students × 2 tests = 1,372 individual scores). Each teacher was instructed to independently code all 1,372 scores on the pretests and posttests using a scoring rubric developed by both science teachers. Then inter-rater agreement was established for the scoring of the learners’ pretest and posttest answers by comparing the individual coding of the high school teacher and the second author. There was agreement on 66 out of 66 randomly selected pretests and posttests, yielding an inter-rater agreement of 100%. For SRL behavior variables and time, inter-rater agreement was established on 289 out of 305 coded segments (27% of codes), yielding inter-rater agreement of 95%. Inconsistencies were resolved through discussion between the experimenters.
226
/
AZEVEDO, WINTERS AND MOOS
RESULTS Question 1: Can low-achieving high school students effectively use a water-quality simulation (WQS) collaboratively to learn about water quality indicators and land use? We examined the shift in students’ learning (from pretest to posttest) by using a paired samples t-test. There was a significant difference between the students’ mean pretest score and their mean posttest score (t[48] = –7.03, p < .001). However, while there was a significant shift in the test scores, the students’ mean score was low on both the pretest (M = 12.1%, SD = 4.5) and the posttest (M = 19.1%, SD = 6.0). Question 2: Do the students and teacher spend the same amount of time using various regulatory behaviors to facilitate students’ conceptual understanding? We examined whether the relatively small (yet significant) learning gains from pretest to posttest were due to differences in the amount of time that both students and teachers spent regulating students’ learning with the WQS across each of the four SRL variables (planning, monitoring, strategy use, and handling task difficulties and demands). A MANOVA was conducted to determine whether students, teacher scaffolding, and teacher instruction differed in the amount of time spent on each of these SRL variables (see Figure 2). The data met the statistical assumptions for a MANOVA. There was a significant difference in the mean time that learners spent regulating their own learning and the amount of time the teacher spent on externally regulating the students by using scaffolding and direct instruction (F[3, 117] = 19.32, p < .05). Overall, students spent significantly more time regulating their own learning than the teacher spent on regulating students’ learning either by instructing or scaffolding. There were also significant differences in the amount of time students and teacher spent using different strategies to regulate students’ learning. There were no differences in the amount of time spent between planning, monitoring, and handling task difficulties and demands. Multiple pairwise comparison tests found significant differences between the time spent by students and the teacher on scaffolding and instruction for four SRL variables. Learners spent significantly more time using strategies to regulate their learning (F[2, 39] = 10.96, p < .05) than the teacher did in providing either direct instruction or scaffolding students’ use of strategies (Mean = 294.4 sec, SD = 236.3; Mean = 143.0 sec, SD = 78.0; Mean = 39.0 sec, SD = 35.9, respectively). The teacher spent significantly more time instructing students how to plan their learning (F[2, 39] = 10.26, p < .05) than she did scaffolding their planning or that students spent planning their own learning (Mean = 36.0 sec, SD = 5.5; Mean = 12.7 sec, SD = 5.5; Mean = 1.8 sec, SD = 5.5, respectively). However, learners and teacher tended
SRL AND COLLABORATIVE LEARNING WITH A CBLE /
227
Figure 2. Mean time spent by students and teacher on SRL behaviors.
to spend the same amount of time regulating their own and her students’ monitoring activities (F[2, 39] = 0.20, p > .05), and handling task difficulties and demands (F[2, 39] = 3.14, p > .05). Question 3: Do the students and teacher differ in the frequency of use of self-regulatory behaviors to facilitate students’ conceptual understanding? Third, we examined how learners and teachers regulated the students’ learning of the ecology system using the WQS by calculating how often teacher instruction, teacher scaffolding, and/or student behavior reflected a variable relating to one of the four main SRL categories. We examined whether the frequency of use of each of the variables relating to the four main SRL categories of planning, monitoring, strategy use, and task difficulty were used significantly differently in teacher instruction, teacher scaffolding, and student behavior. A total of 1,143 segments were coded based on 6.12 hours (367 minutes) of video from the four dyads in each of the two classes. Chi-square analyses revealed a significant difference between the frequency of these variables used between students, teacher scaffolding, and teacher instruction (P2[6, 1143] = 145.16, p < .001) (see Table 1). Due to the overall significance, we probed further to examine what specific SRL behaviors the students engaged in and teachers instructed or scaffolded while learning with the WQS. In this section, we present a detailed account of each of the specific SRL variables used by students and teachers to regulate the students’ learning of ecology with the WQS.
228
/
AZEVEDO, WINTERS AND MOOS
Table 1. Number and Proportion of Students’ Self-Regulated Learning and Teachers’ Scaffolding of Students SRL Movesa Studentinitiated moves SRL variable
f
Teacherinstruction moves
Teacherscaffolding moves
%
f
%
f
%
Planning (Planning, goals, prior knowledge activation, and recycling goal in working memory)
7
1%
33
16%
5
7%
Monitoring (Judgment of learning, partner questioning, monitoring progress toward goals, teacher questioning, and feeling of knowing)
81
9%
21
10%
23
31%
Strategy Use (Follow directions, procedural, content evaluation, search, and selecting new informational source)
720
83%
131
64%
41
55%
Task Difficulty and Demands (Time and effort planning and help-seeking behavior)
56
7%
20
10%
5
7%
866
100%
205
100%
74
100%
Total a
This table and corresponding analyses include 73% of the codeable data (i.e., 1,143 out of 1,562 codes), since in 27% of the segments students were engaged in off-task behavior.
Planning
Chi-square analyses demonstrated that students and teachers engaged in very few behaviors associated with category of planning, accounting for only 4% of all the codes. Under planning, students and teachers were coded for behaviors related to planning (P), activating prior knowledge (PKA), stating goals (G), or recycling a goal in working memory (RG). Students displayed no behavior associated with prior knowledge activation or stating goals. Indeed, student moves were only associated with 1%2 of codes in this category, due to a few instances of planning 2
Percentages cited are the total moves made by students and teachers.
SRL AND COLLABORATIVE LEARNING WITH A CBLE /
229
(less than 1%) and a few instances of recycling goals in working memory (less than 1%). Teachers provided little in the way of instruction to students for planning (only 3% of total codes) and virtually no scaffolding of these SRL behaviors (less than 1% of total codes). The majority of teacher moves in this category were to instruct students on their goals (2%). Teachers did not often activate students’ prior knowledge (less than 1%) and there were only a few instances of teacher instruction of student planning (1%). Monitoring
Chi-square analyses demonstrated that students and teachers also engaged in very few behaviors associated with category of monitoring, accounting for 11% of all the codes. Under monitoring, students and teachers were coded for behaviors related to expressing a Judgment of Learning (JOL), partner questioning (PQ), monitoring their progress toward goals (MPTG), teacher questioning (TQ), and expressing a feeling of knowing (FOK). Student moves associated with monitoring were responsible for 7%; but the majority of student moves in this category were partner questioning (2%) and teacher questioning (4%). There were few instances of judgment of learning (1%), feeling of knowing (less than 1%), and monitoring progress toward goals (less than 1%). Teacher instruction and scaffolding of monitoring accounted for 4%, with teachers helping students monitor their progress toward goals (2% each for TI and TS). Teachers did not scaffold or instruct students with respect to questioning, judgments of learning, or feelings of knowing. Strategies
Chi -square analyses demonstrated that students and teachers engaged in many behaviors associated with category of strategies, accounting for 78% of all the codes. Under strategies, students and teachers were coded for behaviors related to following directions (FD – students only), giving procedural directions (Pro – teachers only), content evaluation (CE), searching the environment (S), and selecting a new informational source (SNIS). Student moves associated with strategies were responsible for 63% of total codes. The majority of student moves in this category were following directions (32%). Students also engaged in selecting new information sources (18%), searching (8%), and evaluating content (6%). These observations were expected, as the jigsaw tasks required these behaviors for completion. Teacher scaffolding of strategies accounted for 4% of moves, including helping students evaluate content (2%) and understand procedures (1%). Teacher instruction of strategies accounted for 11% of moves, including using procedures (which accounted for 10% of all moves) and instruction on content evaluation (which accounted for 1% of all moves). Teachers provided no other strategy help or instruction.
230
/
AZEVEDO, WINTERS AND MOOS
Task Difficulty and Demands
Chi-square analyses demonstrated that students and teachers engaged in very few behaviors associated with category of task difficulty and demands, accounting for only 7% of all the codes. Under task difficulty and demands, students and teachers were coded for behaviors related to time and effort planning (TEP) and help-seeking behavior (HSB). Student moves associated with monitoring were responsible for 5% of total codes. The majority of student moves in this category were help-seeking behavior (5%), usually concerning procedural questions. Students engaged in virtually no time and effort planning (less than 1%). In summary, although the analyses found significant differences between the students’ mean pretest and posttest scores and in the frequency of the students’ and teachers’ use of SRL variables, the learning gains were quite small. The following four qualitative descriptions provide valuable insight into how these students collaborated and self-regulated their learning of ecological systems with the WQS. The typical pattern for students who were on-task working together is demonstrated in example #1 (see Table 2). These students were involved in selecting variables, looking at the graphs, and determining an appropriate value by evaluating the graphs. Students seemed to be merely following directions (e.g., in turn 4, “What do we put that under—nitrogen, right?”). There is no indication that they were thinking critically about the information they were writing down. When teachers were present and students demonstrated that they were lacking the necessary knowledge to effectively evaluate the content in the hypermedia environment, the teacher typically provided only procedural instruction. In example #2, the teacher initiated an interaction with two students by providing procedural instruction (see Table 2). However, when the teacher presented cognitive scaffolding in the form of evaluating the content (e.g., in turn 5, “So, what is this again?”), it becomes evident that the students lacked the necessary knowledge and/or skills to effectively use this scaffolding. Recognizing this inadequacy, the teacher returned to providing procedural instruction. Students often sought help from the teacher while completing the RiverWeb task. Example #3 demonstrates a typical teacher-student interaction in which the students sought help with carrying out a basic procedure for the task, even though the steps for the task had been explained to the whole class at the beginning of the session (see Table 2). The student response to such procedural instruction was merely to follow directions (e.g., in turns 10, 12, and 17, the student either chose variables or wrote as directed); there is no evidence that the students were attempting to understand the concepts. Example #4 illustrates various scaffolding and instructional techniques that teachers provided in order to facilitate students’ learning (see Table 2). The teacher responded to a student-initiated interaction and provided scaffolding for a relationship seen between indicators in the hypermedia environment. In this example, the teacher used both instructional and scaffolding strategies, including procedural
SRL AND COLLABORATIVE LEARNING WITH A CBLE /
231
help (e.g., in turn 4, “Put a w.”) as well as scaffolding the evaluation of the content (e.g., in turn 4, “Do you see it increasing or decreasing?”). In addition, the degree and type of instruction and scaffolding provided by the teachers usually reflected the level of students’ understanding of the information presented in the hypermedia environment. In this example, cognitive scaffolding, as opposed to the more superficial procedural instruction, was provided once the student demonstrated understanding through evaluation of content (e.g., in turn 3, “Okay, this is a weak relationship.”). DISCUSSION Our results show that low-achieving students tend to benefit little from Webbased hypermedia learning environments designed to foster their learning of challenging science topics. Despite performance data that show significant results, the process data including the time and frequency of students’ SRL and teacher’s scaffolding and instructional moves show the use of a disproportionate amount of time and number of SRL moves during learning. The process data provide evidence that students’ poor performance on the pretest-posttest shift is due to their failure to deploy key SRL processes and mechanisms which could have led to significant shifts in conceptual understanding. In addition, the same data show that the teacher rarely deployed scaffolding and instructional moves aimed at fostering students’ self-regulated learning of ecology. With regard to the first research question, the results of this study show that students did learn about the effects of land use during the two-week period in which they used the WQS collaboratively in the ecology class. Despite the statistically significant results on their scores from pretest to posttest, the shift in scores is rather small (7%) and there is ample room for increased learning of ecology, since the mean on the posttest was relatively low (19%). We can conclude that while these Web-based hypermedia learning environments can potentially foster students’ learning, not all students have the self-regulatory skills necessary to learn from these rich environments. Our results indicate that providing lowachieving students with complex, open-ended CBLEs without the appropriate embedded and contextual scaffolding (e.g., from the teacher, peer, tutor, etc.) will lead to inferior shifts in learning. This finding is consistent with the majority of studies on non-linear, random access hypermedia environments with flexible access and a high degree of learner control (e.g., Azevedo, Cromley, & Seibert, 2004; Azevedo, Guthrie, & Seibert, 2004; Greene & Land, 2000; Jacobson & Archodidou, 2000). This finding is consistent with previous research which indicates that students lack the strategies, self-monitoring, persistence, and other self-regulatory skills required to take advantage of the vast amounts of rich content contained in CBLEs such as Web-based hypermedia learning environments. We hypothesize that in addition to not having the self-regulatory skills, these students lack prior
232
/
AZEVEDO, WINTERS AND MOOS
Table 2. Qualitative Descriptions of Students’ and Teacher Interactions during Learning with RiverWeb [Non-Verbal Behavior] (SRL Code) Turn
Transcript
Example 1: 1
Student 2: [Selecting variables] (FD)
2
Student 1: Toxins. . . . Alrighty then, right now we can look for pristine forests. Let me see something. That don’t work. That sucks. What does this say— nitrogen concentration.
3
Student 2: [unintelligible] Um, average is .7. (CE)
4
Student 1: What do we put that under—nitrogen, right? 0.7 [writing on sheet]? (FD)
5
Student 2: [Unintelligible] [writing on sheet] (FD)
6
Student 1: [Selecting variables] There is runoff. (FD) (SNIS)
7
Student 2: 1.7- (CE)
8
Student 1: No, it is 1—1.797 (CE)
9
Student 2: I mean 1.797
10
Student 1: .5. [writing on sheet] (FD)
11
Student 2: Change it—to what? [Selecting variables] Sediments . . . 2.—(FD) (SNIS) (CE)
12
Student 1: - .6 (CE)
13
Student 2: 2.6 [writing on sheet] (CE) (FD)
14
Student 1: Water temp. [Selecting variables] (FD) (SNIS)
15
Student 2: . . . . And its . . . . [looking at screen and writing on sheet] (CE) (FD)
16
Student 1: Phosphorous . . . [Selecting variables] (FD)
17
Student 2: Phosphorous.
Example 2: 1
Teacher: Okay, so you are going to complete this. This first thing that I wanted you to do is put your station number here . . . Okay, so you are going to, you are going to fill out everything here so we know this belongs to. Ah, this is lumbering forest. You are going to look at average runoff. This is nitrogen. (TI-PRO) What station is this? (TS-PRO) Wait a minute, don’t, don’t change it yet. You already have the average nitrogen, okay. (TI-PRO). What’s the average? [referring to information in hypermedia environment] (TS-CE)
2
Student 1: 0.7865 (CE)
SRL AND COLLABORATIVE LEARNING WITH A CBLE /
233
Table 2. (Cont’d.) Turn
Transcript
3
Teacher: Remember, I said you can, you can round that to .79, would be good. (TI-PRO)
4
Student 1: Okay.
5
Teacher: Okay. So what is this again? [referring to information in hypermedia environment[ (TS-CE) Where do you find that on your table? Okay, for what station? (TS-PRO)
6
Student 2: I have no, I don’t know how to do none of this or understand the problem. (JOL)
7
Teacher: Uh, uh. Fill that, put that in there. You will continue and fill out the chart. (TI-PRO)
8
Student 1: What is this? (TQ)
9
Teacher: Well, you have, you have to fill this out first. (TI-PRO)
10
Student 1: How do I find about the [unintelligible]? (TQ)
11
Teacher: Well, again, click on here and it gives you all of your different indicators. (TI-PRO)
Example 3: 1
Student 2: [Hand is raised] I need to find out what station we are. (HSB)
2
Student 1: I think we are 6 [unintelligible].
3
Student 2: Yeah, I think we are 5 or 6. [Keeps hand raised] ......................
4
Student 1: Hey [to teacher] . . .
5
Teacher: Yes?
6
Student 1: What station? (HSB)
7
Teacher: Um, 6. So you can type in 6. (TI-PRO)
8
Student 2: What are we looking for now? (TQ)
9
Teacher: [referring to table on back of packet] Can I rip this? Ok. You are going to click on your land use area. Then you are going to use this table and you are going complete this for your indicators. You are station 6, so I want you to write “6” in here [points to place on sheet]. (TI-PRO)
10
Student 2: [Writes on sheet as directed] (FD)
11
Teacher: And then “urban”. Ok, now, you are going to complete this entire table, so the first thing you are going to do (TI-P) is click on this rectangle where it says air temp.—click there and choose runoff. (TI-PRO)
234
/
AZEVEDO, WINTERS AND MOOS
Table 2. (Cont’d.) Turn
Transcript
12
Student 2: [Clicks on rectangle and chooses runoff]. . . (FD)
13
Teacher: Ok, and then down here, what is the next one? (TS-CE)
14
Student 1: Sediment. (CE) [selects sediment]
15
Student 2: Sediment.
16
Teacher: Uh huh. Now when you change display—[points to screen] you have minimum value, maximum value, and I want you to write down the average, and again, you can round up. (TI-PRO) So that would be 21,939. You write that here and you are done. (TS-MPTG)
17
Student 2: [Writes down number on sheet] (FD)
Example 4: 1
Student 1: Sketch that? [Question directed towards teacher] (TQ)
2
Teacher: Uh, uh. (TI-PRO)
3
Student 1: Okay, this is a weak relationship. [referring to scatter plot graph in hypermedia environment] (CE)
4
Teacher: Uh, uh. Put a w. (TI-PRO). Negative? Do you see it increasing or decreasing? (TS-CE). Or, if you can’t if you can’t tell if it is going up or down, then there’s no relationship. (TI-CE)
5
Student 1: I can’t tell. [referring to scatter plot graph in hypermedia environment] (CE)
6
Teacher: Okay, then you put no relationship. Okay, then as you continue, you guys discuss it amongst yourselves. Make sure you both agree. (TI-PRO)
knowledge of the topic and also lack specific science skills such as rounding numbers. We acknowledge that one of the limitations of this study is that we used a one-group pretest-posttest pre-experimental design and combined it with process data. However, this was a necessary first step prior to conducting experimental manipulations given the infancy of this type of research and the complex nature of students’ SRL with CBLEs in the classroom (Mayer, 2003). With regard to the second and third research questions, our extensive process data indicated that students and the teacher differed in the amount of time and the frequency of use of self-regulatory behaviors during learning with the WQS. These students tend to spend more time on and use more self-regulatory skills, followed by teacher instructional moves and then teacher scaffolding. Also, the students and teacher use strategies as a predominant SRL variable during learning
SRL AND COLLABORATIVE LEARNING WITH A CBLE /
235
with the WQS. The second most often used SRL variable by students and teachers is monitoring. The quantity (i.e., amount and frequency of SRL variables) and quality of the strategy and monitoring processes used by students may explain why students gained so little knowledge of ecology with the WQS. The verbal protocols provide process data to indicate that they used these SRL processes, and together with the product data show that the use of these processes led to significant increases in their understanding of the science topic. Students regulated their learning by using mostly ineffective strategies and metacognitive monitoring and did very little planning. In contrast, the teacher’s instructional and scaffolding moves involved using strategies to facilitate students’ learning. These findings highlight the value of converging product and trace data—i.e., while the statistical analyses revealed significant differences in the time and frequency data, the explanation lies in the analysis of the quality of each of these SRL moves used by both students and teachers. For example, despite spending a significant amount of time regulating their learning, and spending a disproportionate amount of time using these strategies, the process data reveal students were using low-level strategies such as following directions and searching the WQS without a specific goal. Furthermore, the relative amount of time spent by the teacher on instructing and scaffolding moves was relatively less than students spent, and the quality of those moves can also be considered low-level in response to the students level of understanding. These findings are consistent with previous research on using trace methodologies to examine SRL processes used by learners during learning (Azevedo, Cromley, & Seibert, 2004; Azevedo, Guthrie, & Seibert, 2004; Winne & Hadwin, 1998; Winne & Stockley, 1998). They also highlight current theoretical and methodological concerns among SRL researchers that more research is required to understand the inter-relatedness and dynamics of SRL variables during learningcognitive, motivational/affective, behavioral, and contextual during the cyclical and iterative phases of planning, monitoring, control, and reflection (e.g., Butler, 2002; Meyer & Turner, 2002; Perry, 2002; Pintrich, 2000). This is definitively the case in the area of learning with Web-based hypermedia learning environments, where research tends to focus on learning gains (see Shapiro & Niederhauser, 2004 for examples) without considering the complexities of students’ SRL and the teachers’ scaffolding of SRL during extended learning with CBLEs. Implications for the Design of the CBLEs as MetaCognitive Tools to Enhance Students’ Learning The results of our study have implications and pose several challenges for the design of the WQS to foster low-achieving students’ self-regulated learning of ecology in technology-rich dynamic science classrooms. One challenge for designing a Web-based hypermedia environment is the role of scaffolding, which is critical in fostering students’ conceptual understanding and enhancing SRL. The
236
/
AZEVEDO, WINTERS AND MOOS
first issue related to scaffolding is the system’s ability to detect that students need assistance in regulating their learning. For example, how can the WQS detect that a student or the pair is/are using ineffective strategies or need(s) assistance in monitoring their learning? Second, what types and levels of scaffolding methods should be designed for low-self-regulating learners? According to our results, these students typically use ineffective learning strategies, do not monitor their learning, fail to engage in help-seeking behavior, and have difficulties in handling task difficulties and demands. So, how can we “build” system components (e.g., a student model, instructional model, interface, inference engine, etc.) to determine if: a) a learner, in the complex interactions during individual and collaborative knowledge construction is having difficulties with several phases and areas of SRL; and b) what effects will this determination have on the detection, monitoring, and fostering of learners’ overall self-regulation? How can we make our SRL model “visible” to the learners and flexible enough to allow learners to explore advanced topics related to the ecological systems, including its content and structure? How can the CBLE adapt and exhibit flexibility during learning? Recent research on pedagogical agents (e.g., Baylor, 2003; White et al., 2000) suggest that these agents could be used to facilitate students’ SRL by scaffolding students’ planning (e.g., include a planning net with a hierarchy of sub-goals, model the phases of science inquiry), monitoring (e.g., prompting student to periodically relate their emerging understanding with their prior knowledge and instructional goals), strategy use (e.g., embed video clips that model effective strategies such as making inferences and hypothesizing, model basic science skills such as rounding off numbers), and handling task difficulties and demands (e.g., prompts to help the student monitor their time and effort planning, provide a digital library of video clips that deal with FAQs related to basic science skills, inquiry skills, etc.). Another critical issue is the system’s ability to construct an adequate model of the learner’s emerging understanding of the domain (i.e., a mental model). How can we design ways of detecting, monitoring, and fostering shifts in learners’ mental models of ecological systems? Can we have students create concept maps and/or drawings which can be used to dynamically assess their existing mental model, which could interact with other system components? For example, what kind of instructional decisions should be made in a case where the system has determined that a student has a sophisticated mental model of the ecological system, but has expressed low interest in the task, versus a student who has constructed a less-sophisticated mental model has expressed a high level of interest in the topic, yet lacks the ability to plan learning and is using ineffective strategies (e.g., “blindly” searching the WQS to find the representations of information needed to explore the issues of the current goal)? Would making the learner construct an “external” visual representation of his/her emerging mental model of the domain allow him/her to self-regulate? At another level, would this visual
SRL AND COLLABORATIVE LEARNING WITH A CBLE /
237
representation allow a peer to assist with regulating their learning? Would this external representation allow learners to jointly regulate their co-construction of the topic? Can this information provide the WQS with other “variables” with which to make informed instructional decisions? Also, would these external representations, which are visible and accessible, allow the learner (and perhaps others such as peers, and teachers) to share, inspect, critique, modify, and assess the learners’ understanding of a complex system? These issues pose several challenges for the learning sciences community. First, there is a greater need to draw on the strengths of the members of our research community in dealing with the complexities of studying SRL in complex instructional settings and in designing and developing CBLEs to trace, detect, and foster students’ SRL. Second, the role of teachers needs to be examined in terms of how different instructional methods and scaffolding support students’ SRL, and their role as external regulatory agents creating instructional settings that promote students’ SRL (Corno & Randi, 1999; Meyer & Turner, 2002; Perry et al., 2002). Third, theoretical issues related to SRL challenge Derry and Lajoie’s (1993) initial characterization of three camps for using computers as cognitive tools—modelers, middle-campers, and non-modelers. Lajoie (2000) has recently argued that we should abandon these “camps” due to both the influx of computer technologies made available to teachers and educators and to recent challenges made by SRL researchers about the complexity of learning. Future research should explore the role of SRL and teachers’ fostering of students’ understanding of complex and challenging science topics in dynamic technology-enhanced classroom learning environments (Azevedo, 2002).
Searching memory for relevant prior knowledge either before beginning performance of a task or during task performance
Restating the goal (e.g., question or parts of a question) in working memory (WM)
Prior Knowledge Activation (PKA)
Recycle Goal in Working Memory (RG)
Student: “It says write the relationships.”
Tutor Instruction: “You need to remember what we did during the tour.”
Tutor Scaffolding: “Why might sediments and nitrogen go together?”
Tutor Instruction: “You will be writing relationships.”
Tutor Scaffolding: “I have given you an example to use to complete this page.”
Consist either of operations that are possible, postponed, or intended, or of states that are expected to be obtained. Goals can be identified because they have no reference to already-existing states
Goals (G)
Example
A plan involves coordinating the selection of Student: “Next we are going to look at that” operators. Its execution involves making [referring to new data entry] behavior conditional on the state of the problem and a hierarchy of goals and sub-goals Tutor Instruction: “When you are done, you can look at sediments”
Descriptiona
Planning Planning (P)
Variable
/
APPENDIX Classes, Descriptions, and Examples of the Variables Used to Code Learners’ Self-Regulatory Behavior and Co-Regulated Behavior (based on Azevedo, Cromley, & Seibert, 2004).
238 AZEVEDO, WINTERS AND MOOS
Learner becomes aware that they don’t know or understand everything they read Learner directs learning or procedural based question towards collaborative partner Assessing whether previously-set goal has been met
Learner directs learning or procedural based question towards teacher Learner is aware of having read something in the past and having some understanding of it, but not being able to recall it on demand Learner is involved in one of the following learning-based behaviors: (1) Reading worksheet, (2) entering data, (3) calculating, or (4) collaborating with partner Can be non-verbal behavior Teacher provides scaffolsding/instruction on procedure of learning task
Monitoring Judgment of Learning (JOL)
Partner Questioning (PQ)
Monitor Progress Toward Goals (MPTG)
Teacher Questioning (TQ)
Feeling of Knowing (FOK)
Strategy Use Follow Directions (FD)
Procedural (Pro)
Teacher Scaffolding: [Referring to worksheet] “Which land use is it for?” Teacher Instruction: “You only need to average”
Student: Types data into computer, calculates, writes information on worksheet, discusses task steps with partner.
Student: “I did read them” [but does not know the answer].
Student: [Directed towards teacher] “What are we suppose to do with the calculator part?”
Student: “We are done.” Tutor Scaffolding: “Are you guys finished?” Tutor Instruction: “You should be starting on your second chart.”
Student: [Directed towards partner] “How do we know what’s air temp?”
Student: [Referring to teacher scaffolding of content evaluation] “I don’t know how to use those graphs.”
SRL AND COLLABORATIVE LEARNING WITH A CBLE / 239
Monitoring content relative to goals
Searching the hypermedia environment without specifying a specific plan or goal Can be non-verbal behavior The selection and use of various cognitive strategies for memory, learning, reasoning, problem solving, and thinking. May include selecting a new representation, coordinating multiple representations, etc. Can be non-verbal behavior
Search (S)
Selecting New Informational Source (SNIS)
Descriptiona
Content Evaluation (CE)
Variable
Student: [Learner enters new data] then advances to new line or scatterplot graph
Student: Learner manipulates hypermedia environment by scrolling either up or down
Student: [Referring to scatterplot in hypermedia] “That’s positive; that’s strong.” Tutor Scaffolding: [Referring to scatterplot in hypermedia environment] “Do you see it increasing or decreasing?” Tutor Instruction: [Referring to scatterplot in hypermedia environment]: “Think about why the strong ones are strong.”
Example
/
APPENDIX (Cont’d.)
240 AZEVEDO, WINTERS AND MOOS
Learner exhibits behavior that is clearly not within the learning task, including talking off-topic with collaborative partner and/or peer. Can be non-verbal behavior
Due to video camera angle and/or audio, behavior and/or verbalization of learner(s) cannot be reliably coded using one of the SRL codes
Learner seeks assistance regarding either the adequateness of their answer or their instructional behavior Can be non-verbal behavior
Attempts to intentionally control behavior
Student: Raises hand and/or verbally solicits help from teacher or researcher.
Student: [To partner] “What do we do now?” Teacher Scaffolding: [Referring to one of the collaborative learners] “Are you helping him?” Tutor Instruction: “I will give you a couple more minutes.”
All codes refer to what was recorded in the classroom and captured on the audio and videotapes (i.e., what students read, saw, heard, or did), including what the teacher initiated and scaffolded during learning with the WQS. Some codes include non-verbal behavior.
a
Off-task (OT)
Other Uncodeable (UC)
Help Seeking Behavior (HSB)
Task Difficulty and Demands Time and Effort Planning (TEP)
SRL AND COLLABORATIVE LEARNING WITH A CBLE / 241
242
/
AZEVEDO, WINTERS AND MOOS
ACKNOWLEDGMENTS The authors would like to thank Stacy Pritchett for constructing the lesson plans, assisting the classroom teachers, and scoring the pretests and posttests; and Susan Ragan and Mary Ellen Verona at the Maryland Virtual High School for allowing us to use RiverWeb in this classroom study. We also thank Jennifer G. Cromley for comments on an earlier version of this manuscript; Laura Smith, Lucia Buie, and Jennifer G. Cromley for assisting in the data collection; and the high school teachers (Wendell Hall and Philip Johnson) and their students for participating in this study.
REFERENCES Azevedo, R. (2002). Beyond intelligent tutoring systems: Computers as MetaCognitive tools to enhance learning? Instructional Science, 30(1), 31-45. Azevedo, R., Cromley, J. G., & Seibert, D. (2004). Does adaptive scaffolding facilitate students’ ability to regulate their learning with hypermedia, Contemporary Educational Psychology, 29, 344-370. Azevedo, R. Cromley, J. G., Winters, F. I., Xu, L., & Iny, D. (2003). Is strategy instruction effective in facilitating students’ ability to regulate their learning with hypermedia? In U. Hoppe, F. Verdejo, & J. Kay (Eds.), Artificial intelligence in education: Shaping the future of learning through intelligent technologies (pp. 193-200). Amsterdam: IOS Press. Azevedo, R., Guthrie, J. T., & Seibert, D. (2004). The role of self-regulated learning in fostering students’ conceptual understanding of complex systems with hypermedia. Journal of Educational Computing Research, 30, 87-111. Azevedo, R., Verona, M. E., & Cromley, J. G. (2001). Fostering students’ collaborative problem solving with RiverWeb. In J. D. Moore, C. L. Redfield, & W. L. Johnson (Eds.), Artificial intelligence in education: AI-ED in the wired and wireless future (pp. 166-175). Amsterdam: IOS Press. Baylor, A. (2003). Evidence that multiple agents facilitate greater learning. In U. Hoppe, F. Verdejo, & J. Kay (Eds.), Artificial intelligence in education: Shaping the future of learning through intelligent technologies (pp. 377-379). Amsterdam: IOS Press. Biswas G., Schwartz, D., Bransford, J., & CTGV. (2001). Technology support for complex problem solving: From SAD environments to AI. In K. Forbus & P. Feltovich (Eds.), Smart machines in education. Cambridge Press, MA: MIT Press. Boekaerts, M., Pintrich, P., & Zeidner, M. (2000), Handbook of self-regulation. San Diego, CA: Academic Press. Butler, D. (1998). The strategic content learning approach to promoting self-regulated learning: A report of three studies. Journal of Educational Psychology, 90(4), 682-697. Butler, D. (2002). Individualizing instruction in self-regulated learning. Theory Into Practice, 41(2), 81-92. Clark, D., & Linn, M. (2003). Designing for knowledge integration: The impact of instructional time. Journal of the Learning Sciences, 12(4), 451-493.
SRL AND COLLABORATIVE LEARNING WITH A CBLE /
243
Corno, L., & Randi, J. (1999). A design theory for classroom instruction in selfregulated learning? In C. Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory (Vol. II; pp. 293-318). Mahwah, NJ: Erlbaum. Derry, S. J., & Lajoie, S. P. (1993). Computers as cognitive tools (1st ed.). Hillsdale, NJ: Erlbaum. Greene, B., & Land, S. (2000). A qualitative analysis of scaffolding use in a resourcebased learning environment involving the World Wide Web. Journal of Educational Computing Research, 23(2), 151-179. Hickey, D., Kindfield, A., Horowitz, P., & Christie, M. (2003). Integrating curriculum, instruction, assessment, and evaluation in a technology-supported genetics learning environment. American Educational Research Journal, 40(2), 495-538. Jacobson, M., & Archodidou, A. (2000). The design of hypermedia tools for learning: Fostering conceptual change and transfer of complex scientific knowledge. Journal of the Learning Sciences, 9(2), 149-199. Jonassen, D. (1996). Computers as mind tools for schools. Columbus, OH: Merrill. Kozma, R., Chin, E., Russell, J., & Marx, N. (2000). The roles of representations and tools in the chemistry laboratory and their implications for chemistry learning. Journal of the Learning Sciences, 9(2), 105-144. Lajoie, S. P. (2000). Computers as cognitive tools II: No more walls: Theory change, paradigm shifts and their influence on the use of computers for instructional purposes. Mahwah, NJ: Erlbaum. Lajoie, S. P., & Azevedo, R. (2000). Cognitive tools for medical informatics. In S. P. Lajoie (Ed.), Computers as cognitive tools II: No more walls: Theory change, paradigm shifts and their influence on the use of computers for instructional purposes (pp. 247-271). Mahwah, NJ: Erlbaum. Lajoie, S. P., Guerrera, C., Munsie, S., & Lavigne, N. (2001). Constructing knowledge in the context of BioWorld. Instructional Science, 29(2), 155-186. Mayer, R. (2003). Learning environments: The case for evidence-based practice and issue-driven research. Educational Psychology Review, 15(4), 359-366. McCaslin, M., & Hickey, D. (2001). Self-regulated learning and academic achievement: A Vygtoskian perspective. In B. Zimmerman & D. Schunk (Eds.), Self-regulated learning and academic achievement: Theoretical perspectives (pp. 227-252). Mahwah, NJ: Erlbaum. Meyer, D. & Turner, J. (2002). Using instructional discourse analysis to study the scaffolding of student self-regulation. Educational Psychologist, 37(1), 17-25. Paris, S. G., & Paris, A. H. (2001). Classroom applications of research on self-regulated learning. Educational Psychologist, 36(2), 89-101. Perry, N. (2002). Introduction: Using qualitative methods to enrich understandings of self-regulated learning. Educational Psychologist, 37(1), 1-3. Perry, N., VendeKamp, K., Mercer, L., & Nordby, C. (2002). Investigating teacherstudent interactions that foster self-regulated learning. Educational Psychologist, 37(1), 5-15. Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 451-502). San Diego, CA: Academic Press.
244
/
AZEVEDO, WINTERS AND MOOS
Rasku-Puttonen, H., Etelapelto, A., Arvaja, M., & Hakkinen, P. (2003). Is successful scaffolding an illusion? Shifting patterns of responsibility and control in teacher-student interaction during a long-term learning project. Instructional Science, 31, 377-393. Reiser, B., Tabak, I., Sandoval, W., Smith, B., Steinmuller, F., & Leone, A. (2001). BGuILE: Strategic and conceptual scaffolds for scientific inquiry in biology classrooms. In S. Carver & D. Klahr (Eds.), Cognition and instruction: Twenty-five years of progress (pp. 263-306). Mahwah, NJ: Erlbaum. Schunk, D. (2001). Social cognitive theory and self-regulated learning. In B. Zimmerman & D. Schunk (Eds.), Self-regulated learning and academic achievement: Theoretical perspectives (pp. 125-152). Mahwah, NJ: Erlbaum. Shapiro, A., & Niederhauser, D. (2004). Learning from hypertext: Research issues and findings. In D. H. Jonassen (Ed.), Handbook of research for education communications and technology (2nd ed). Mahwah, NJ: Erlbaum. Slotta, J. D., & Linn, M. C. (2000). The knowledge integration environment: Helping students use the Internet effectively. In M. Jacobson & R. Kozma (Eds.), Learning the sciences of the 21st century. Mahwah, NJ: LEA. Suthers, D., & Hundhausen, C. (2003). An experimental study of the effects of representational guidance on collaborative learning processes. Journal of the Learning Sciences, 12(2), 183-218. Vye, N., Schwartz, D., Bransford, J., Barron, B., Zech, L., & CTGV. (1998). SMART environments that support monitoring, reflection, and revision. In D. Hacker, J. Dunlosky, & A. Graesser (Eds.), Metacognition in educational theory and practice (pp. 305-346). Mahwah, NJ: Erlbaum. White, B. Y., Shimoda, T. A., & Frederiksen, J. R. (2000). Facilitating students’ inquiry learning and metacognitive development through modifiable software advisers. In S. P. Lajoie (Ed.), Computers as cognitive tools II: No more walls: Theory change, paradigm shifts and their influence on the use of computers for instructional purposes (pp. 97-132). Mahwah, NJ: Erlbaum. Williams, M. (1996). Learner control and instructional technologies. In D. Jonassen (Ed.), Handbook of research on educational communications and technology (pp. 957-983). New York: Scholastic. Winne, P. H. (2001). Self-regulated learning viewed from models of information processing. In B. Zimmerman & D. Schunk (Eds.), Self-regulated learning and academic achievement: Theoretical perspectives (pp. 153-189). Mahwah, NJ: Erlbaum. Winne, P., & Hadwin, A. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. Graesser (Eds.), Metacognition in educational theory and practice (pp. 277-304). Hillsdale, NJ: Erlbaum. Winne, P. H., & Perry, N. E. (2000). Measuring self-regulated learning. In M. Boekaerts, P. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 531-566). San Diego, CA: Academic Press. Winne, P., & Stockley, D. (1998). Computing technologies as sites for developing self-regulated learning. In D. Schunk & B. Zimmerman (Eds.), Self-regulated learning: From teaching to self-reflective practice (pp. 106-136). New York: Guilford. Zimmerman, B. (2001). Theories of self-regulated learning and academic achievement: An overview and analysis. In B. Zimmerman & D. Schunk (Eds.), Self-regulated learning and academic achievement: Theoretical perspectives (pp. 1-37). Mahwah, NJ: Erlbaum.
SRL AND COLLABORATIVE LEARNING WITH A CBLE /
245
Zimmerman, B., & Schunk, D. (Eds.). (2001). Self-regulated learning and academic achievement (2nd ed.). Mahwah, NJ: Erlbaum.
Direct reprint requests to: Roger Azevedo, Ph.D. University of Maryland Dept. of Human Development 3304 Benjamin Bldg., 3304E College Park, MD 20742 e mail:
[email protected]
