Self-Regulation of Learning with Multiple Representations in ...

Self-Regulation of Learning with Multiple Representations in Hypermedia Jennifer CROMLEY, Roger AZEVEDO, and Evan OLSON Department of Human Development and Cognition and Technology Laboratory, University of Maryland, 3304 Benjamin Building, College Park, MD 20742, USA [email protected], [email protected], [email protected]

Abstract. A body of research has demonstrated when multiple representations of content help students learn. Few studies, however, have used process measures to understand what different cognitive processes students enact when learning from different representations. We collected pretest, posttest, think-aloud, and video data from 21 undergraduate students learning about the human circulatory system using a hypermedia encyclopedia. We measured learning as a change in a participant’s mental model of the circulatory system from pretest to posttest. Students who learned more tended to spend less time in Text. While viewing Text alone, amount of learning was most strongly associated with verbalizing a smaller proportion of Feeling of Knowing, Free Search, and Selecting a New Informational Source. For Text + Diagrams, the amount of learning was most strongly associated with verbalizing a larger proportion of Inference and Self-Questioning. For Animation, the only significant variable was Summarizing. When not using the hypermedia environment, the significant variables were Feeling of Knowing, Prior Knowledge Activation, and Taking Notes. We close with implications for designing hypermedia environments for learning about complex science topics.

Introduction Hypermedia environments, by definition, present learners with multiple representations of content (e.g., text, tables, diagrams, video clips). Learning, however, is not always improved by including multiple representations, either in Computer-Based Learning Environments (CBLEs) or in paper text [1]. A number of studies have shown that learners have difficulty coordinating different representations of the same content (e.g., [2]). A body of research has demonstrated when multiple representations help students learn complex science topics. A series of studies by Ainsworth and colleagues [3,4,5]; Chandler, Cooper, Sweller, and colleagues (e.g., [6]) and Mayer and colleagues (e.g., [7,8,9]), together with other studies (see the special issue of Learning and Instruction [10]), suggest that learning is improved when illustrations highlight important information, authors and designers avoid distracting information, and modalities are combined in ways that do not overload working memory. Few studies, however, have used process measures to understand how students learn from multiple representations, that is, what different cognitive processes they enact when learning from different representations. We feel that better understanding the cognitive processes involved in using different representations can offer important guidelines for the design of CBLEs [11]. We begin by reviewing the handful of studies that we were able to identify that have collected process data from participants using multiple representations. We considered studies of learning with CBLEs or paper text, across different domains, and using different theoretical frameworks. We then describe our research questions.

Using a self-explanation framework, Ainsworth and Loizou [3] compared undergraduate students learning about the circulatory system from either paper text or diagrams. They prompted both groups to self-explain while learning. Participants completed pretests and posttests that included matching terms with definitions and drawing the flow of blood through the body. Students who were given diagrams had significantly higher scores on the diagram and flow measures at posttest. The researchers conclude on the basis of the verbal protocols that diagram participants engaged in more self-explanation while learning. Using a cognitive strategy approach, Moore and Scevak [12] collected think-aloud, written free recall, standardized comprehension, and answers to literal and inferential questions from 119 of the highest-skilled readers in 5th, 7th, and 9th grades reading text and a diagram in science text. Older students tended to use a larger variety of different strategies while learning, and more often coordinated text and diagram than did younger students. From an Information Processing Theory perspective, Hegarty & Just [13] used eye tracking to study cognitive processes when learning about complex pulley systems in text-anddiagram format on a computer. The researchers found that subjects integrated reading the text with fixating on the diagram rather than reading the complete text first and then fixating on the diagram. The location of the interruption in the subjects’ reading of text tended to be at the end of a clause or sentence. Using an expert-novice paradigm, Kozma and Russell [14] had both professional and undergraduate student chemists sort different representations—animations, equations, graphs, and videotapes of chemical processes—and verbally explain why they had sorted them in the way they did. The representations could be viewed on a computer, and were depicted on cards which were sorted. Whereas novices tended to sort different representations together (e.g., several videos together), experts made more multiple-media groupings. As with experts in physics (e.g., [15]) experts tended to give explanations based on laws and principles, whereas student explanations tended to describe surface features of the problem (e.g., movement, color). Using a cognitive strategy approach, Lewalter [15] had undergraduate students think aloud while learning about how stars bend light in three different computer-based formats: text only, static diagrams, and animated diagrams. While students in both diagram conditions learned significantly more than those in the text condition, the think-aloud protocols showed that the static and animated diagram groups used different learning strategies. Most verbalizations were restatements of the text with little paraphrasing. However, the animated diagram group did verbalize more feeling of knowing, while the static diagram group engaged in more planning. In summary, researchers have in a few cases collected process data from participants using multiple representations, but in only two studies did participants use hypermedia. There is therefore a need to collect process data from students while they are learning using multiple representations in hypermedia environments. We designed a research study to investigate the relationship of Self-Regulated Learning (SRL) strategies used while learning from different representations (Text, Text + Diagrams, Animation, and Not in Environment) to learn about the circulatory system from a hypermedia environment. We measured learning as a change in a participant’s mental model of the circulatory system from pretest to posttest—based on Azevedo and Cromley [17] and Chi [18]. The research questions were: 1) Which SRL variables are used while learning from different representations in hypermedia? 2) For each of the four different representations, what is the relationship between learning and amount of time spent in the representation? 3) For each of the four different representations, what is the relationship between learning and proportion of use of SRL variables?

1. Method 1.1 Participants Participants were 21 undergraduate students (19 women and 2 men) who received extra credit in their Educational Psychology course for their participation. Their mean age was 22.4 years and mean GPA was 3.3. Forty-eight percent (n = 11) were seniors, 52% (n = 10) were juniors. The students were non-biology majors and the pretest confirmed that all participants had average or little knowledge of the circulatory system (pretest M = 5.29, SD = 2.61; posttest M = 8.52, SD = 2.64).

1.2 Materials and Equipment In this section, we describe the hypermedia environment, participant questionnaire, pretest and posttest measure, and recording equipment. During the experimental phase, the participants used a hypermedia environment to learn about the circulatory system. During the training phase, learners were shown the three most relevant articles in the environment (i.e., circulatory system, blood, and heart), which contained multiple representations of information—text, static diagrams, photographs, and a digitized animation depicting the functioning of the heart. Of the three most relevant articles, the blood article was approximately 3,800 words long, had 7 sections, 8 sub-sections, 25 hyperlinks, and 6 illustrations. The heart article was approximately 10,000 words long, had 6 sections, 10 subsections, 58 hyperlinks, and 28 illustrations. The circulatory system article was approximately 3,100 words long, had 5 sections, 4 sub-sections, 24 hyperlinks, and 4 illustrations. During learning, participants were allowed to use all of the features incorporated in the environment, such as the search functions, hyperlinks, and multiple representations of information, and were allowed to navigate freely within the environment. The paper-and-pencil materials consisted of a consent form, a participant questionnaire, a pretest and identical posttest. The pretest was constructed in consultation with a nurse practitioner who is also a faculty member at a school of nursing in a large mid-Atlantic university. The pretest consisted of a sheet on which students were asked to write everything they knew about the circulatory system, including the parts and their purposes, how they work individually and together, and how they support the healthy functioning of the human body. The posttest was identical to the pretest. During the learning session, all participant verbalizations were recorded on a tape recorder using a clip-on microphone and the computer screen and work area were recorded on a digital videotape.

1.3 Procedure The first two authors tested participants individually. First, the participant questionnaire was handed out, and participants were given as much time as they wanted to complete it. Second, the pretest was handed out, and participants were given 10 minutes to complete it. Participants wrote their answers on the pretest and did not have access to any instructional materials. Third, the experimenter provided instructions for the learning task. The following instructions were read and presented to the participants in writing. Participant instructions were: “You are being presented with a hypermedia environment, which contains textual information, static diagrams, and a digital animation of the circulatory system. We are trying to learn more about how students use hypermedia environments to learn about the circulatory system. Your task is to learn all you can about the circulatory system in

40 minutes. Make sure you learn about the different parts and their purpose, how they work both individually and together, and how they support the human body. We ask you to ‘think aloud’ continuously while you use the hypermedia environment to learn about the circulatory system. I’ll be here in case anything goes wrong with the computer and the equipment. Please remember that it is very important to say everything that you are thinking while you are working on this task.” Participants were provided with pen and paper with which they could take notes, although not all did so.

1.4 Data Analysis In this section, we describe scoring the pretest/posttest, coding the think-aloud protocols, and interrater reliability for the coding. To code the participants’ mental models, we used a 12-model coding scheme developed by Azevedo and Cromley ([17]; based on Chi [18]) which represents the progression from no understanding to the most accurate understanding of the circulatory system: (1) no understanding, (2) basic global concepts, (3) basic global concepts with purpose, (4) basic single loop model, (5) single loop with purpose, (6) advanced single loop model, (7) single loop model with lungs, (8) advanced single loop model with lungs, (9) double loop concept, (10) basic double loop model, (11) detailed double loop model, and (12) advanced double loop model. The mental models accurately reflect biomedical knowledge provided by the nurse practitioner. A complete description of the necessary features for each mental model is available in [17, pp. 534-535]. The mental model “jump” was calculated by subtracting the pretest mental model from the postest mental model. To code the learners’ self-regulatory behavior, we began with the raw data: 827 minutes (13.8 hr) of audio and video tape recordings from the 21 participants, who gave extensive verbalizations while they learned about the circulatory system. During the first phase of data analysis, a graduate student transcribed the audio tapes and created a text file for each participant. This phase of the data analysis yielded 215 single-spaced pages (M = 10 pages per participant) with a total of 71,742 words (M = 3,416 words per participant). We used Azevedo and Cromley’s [17] model of SRL for analyzing the participant’s self-regulatory behavior. Their model is based on several recent models of SRL [19, 20, 21]. It includes key elements of these models (i.e., Winne’s [20] and Pintrich’s [19] formulation of self-regulation as a fourphase process) and extended these key elements to capture the major phases of self-regulation: Planning, Monitoring, Strategy Use, Task Difficulty and Demands, and Interest. See Table 2 for the specific codes for each phase; for definitions and examples of the codes, see Azevedo and Cromley [17, pp. 533-534]. We used Azevedo and Cromley’s SRL model to re-segment the data from the previous data analysis phase. This phase of the data analysis yielded 1,533 segments (M = 73.0 per participant) with corresponding SRL variables. A graduate student coded the transcriptions by assigning each coded segment one of the SRL variables. To code the videotapes, we viewed each time-stamped videotape along with its coded transcript. We recorded time spent in each representation with a stopwatch and noted on the transcript which representation was being used for each verbalization. We defined Text + Diagrams as text together with any diagram, so long as at least 10% of the diagram remained visible on the computer screen. We defined Not in Environment as any time the participant read his or her notes (or verbalized in response to reading those notes), subsequently added to those notes without looking back at the screen (similar to Cox and Brna’s External Representations [22]), or read the task instructions. Inter-rater reliability was established by recruiting and training a graduate student to use the description of the mental models developed by Azevedo and Cromley [17]. The graduate student was instructed to independently code all 42 selected protocols (pre- and posttest

descriptions of the circulatory system from each participant) using the 12 mental models of the circulatory system. There was agreement on 37 out of a total of 42 student descriptions yielding a reliability coefficient of .88. Similarly, inter-rater reliability was established for the coding of the learners’ self-regulated behavior by comparing the individual coding of the same graduate student, who was trained to use the coding scheme with that of one of the experimenters. She was instructed to independently code 7 randomly selected protocol segments (30% of the 1,533 coded segments with corresponding SRL variables). There was agreement on 458 out of 462 segments yielding a reliability coefficient of .98. Inconsistencies were resolved through discussion between the experimenters and the student.

2. Results 2.1 Descriptive Statistics Descriptive statistics on time spent in the four representations are shown in Table 1. On average, participants spent the most time in Text + Diagram (with little variability) and the least time in Animation, but with great variability in all representations other than Text + Diagram.

2.2 Research Question 1—Which SRL variables are used while learning from different representations in hypermedia? Participants verbalized fewer SRL variables in representations other than Text + Diagram. See Table 1 for the number of SRL variables verbalized; not all SRL variables could be verbalized in all representations, e.g., Control of Context could only be enacted in the hypermedia environment. See Table 2 for which specific SRL variables were verbalized in each representation. Table 1. Descriptive Statistics for Time Spent in Representations and Number of SRL Variables Verbalized

Representation Text + Diagram Not in Environment Text Animation

Time Mean (SD) in min 19.03 (3.48) 9.00 (6.17) 8.62 (5.17) 2.72 (1.82)

No. SRL Variables Verbalized (% of possible) 30 (100%) 19 (73%) 26 (90%) 15 (56%)

2.3 Research Question 2—For each of the four different representations, what is the relationship between learning and amount of time spent in the representation? We computed Spearman rank correlations between the amount of time spent in each representation and jump in mental models. These results indicate which representations are associated with a higher jump in mental models from pretest to posttest. Proportion of time in Text had the highest correlation and the only significant correlation with mental model jump (rs [21] = -.47, p < .05). The other representations had smaller and non-significant correlations: Text + Diagram (rs [21] = .30, p > .05), Not in Environment (rs [21] = .17, p > .05), and Animation (rs [21] = .18, p > .05). Participants who spent a higher proportion of time in Text only had lower mental model shifts. We hypothesize that Text is either not as instructive as the other representations, or is more confusing than the other representations.

2.4 Research Question 3—For each of the four different representations, what is the relationship between learning and proportion of use of SRL variables? In order to correct for the different number of verbalizations per participant and the different amounts of time spent in each representation, we transformed the raw counts of verbalizations of each SRL variable in each representation. We then multiplied the proportion of verbalizations for each SRL variable times the proportion of time spent in each representation. Finally, we computed Spearman rank correlations between the transformed proportion of use of SRL variables and jump in mental models for each representation. Results are shown in Table 2. Table 2. Spearman Rank Correlation Between Proportion of Use of Each SRL Variable and Mental Model Jump, for Each Type of Representation Variable [Raw number of verbalizations] Text Text+Diagram Animation NIE Planning Prior Knowledge Activation [78] -.322 -.060 -.3001 .447* Planning [10] -.179 — .171 — Recycle Goal in Working Memory [29] .0941 -.261 — -.005 Sub-Goals [40] -.208 .210 .2101 .094 1 Monitoring Feeling of Knowing [105] .205 .300 -.523* .545* Judgment of Learning [70] -.304 .170 .021 .152 Monitoring Progress Toward Goals [13] -.232 -.057 — .136 Identify Adequacy of Information [14] -.273 .017 — — Self-Questioning [11] .022 — .3001 .435* Content Evaluation [58] -.086 NA -.375*1 -.420* Strategy Use Draw [23] .0941 .216 .107 .292 Summarization [125] -.347 .170 — .435* Taking Notes [321] -.188 .347 -.062 .470* Read Notes [77] NA NA NA .181 Knowledge Elaboration [14] .008 .136 — -.2061 Coordinating Informational Sources [42] NA .041 — .360 Find Location in Environment [6] .278 .041 — NA Selecting New Informational Source [50] .257 .352 .300 1 -.513* Goal-Directed Search [12] .059 .266 NA NA Free Search [32] -.255 NA NA -.441* Mnemonics [9] — .296 — .0941 Inferences [29] — .371 .379* .392* Re-Reading [97] -.089 .276 -.053 — Memorization [5] — -.114 — .0941 Task Difficulty and Demands Time and Effort Planning [19] .181 -.169 -.300 -.188 Control of Context [186] -.103 .040 NA -.438* Help Seeking Behavior [7] .045 -.071 .094 — Expect Adequacy of Information [13] NA .101 .307 NA Task Difficulty [14] -.240 .300 — -.435* Interest Interest Statement [28] -.316 .131 .085 — * p < .10, — Dashes indicate the SRL variable was not used by any participants in that representation, NA indicates code was not possible in that representation, 1 indicates code was used by only one participant.

While viewing Text alone, amount of jump was significantly associated with verbalizing a smaller proportion of Feeling of Knowing (FOK), Free Search (FS), Selecting a New Informational Source (SNIS), Control of Context (COC), Task Difficulty (TD), Content Evaluation (CE), and with a larger proportion of Inference (INF). While viewing Text + Diagrams, amount of jump was significantly associated with verbalizing a larger proportion of Inferences and Self-Questioning (SQ). While viewing the Animation, amount of jump was significantly associated with verbalizing a larger proportion of Summarizing (SUM). And when not using the hypermedia environment, amount of jump was most strongly associated with verbalizing a larger proportion of Feeling of Knowing, Prior Knowledge Activation (PKA), and Taking Notes (TN). Looking at the same codes across representations, PKA was positively associated with jumping when it was verbalized Not in Environment, but was negatively associated with jumping when verbalized in Text. FOK was likewise positively associated with jumping when it was verbalized Not in Environment, but was negatively associated with jumping when verbalized in Text (that is, participants appeared to have some false sense of understanding when in text). SQ was positively associated with jumping when it was verbalized in Text + Diagram, but not in the other representations. CE was negatively associated with jumping when it was verbalized in Text (that is, participants appeared to have some false sense of the content being irrelevant when in text). SUM was positively associated with jumping when it was verbalized in the Animation (participants rarely took notes while watching the animation), whereas Taking Notes was positively associated with jumping when it was verbalized Not in Environment (i.e., adding to already-existing notes). SNIS was negatively associated with jumping when it was verbalized in Text (in this context, switching to the Animation from Text), but was non-significant when it was verbalized in Text + Diagrams or Not in Environment. FS (skimming) was also negatively associated with jumping when it was verbalized in Text. Inferences were positively associated with jumping when verbalized in Text or Text + Diagram. COC (frequently using the back arrow or hyperlinks) and TD were negatively associated with jumping when they were verbalized in Text.

3. Implications for Research and Design of Computer-Based Learning Environments Our findings suggest certain guidelines for the design of hypermedia environments (see also Brusilovsky [23]). When students are using Text alone, they generally should be encouraged to switch to a different representation. However, to the extent that Text alone contains valuable information, students should be encouraged to draw inferences. For example, after the student reads 1-2 paragraphs, the environment could display a question that requires the student to draw inference from just-read text. In Text + Diagrams, the environment should encourage students to draw inferences, and should also encourage self-questioning. One simple way to do this would be to ask the student to write a question; the quality of the question need not be graded or scored, but we hope that by asking students to write a question, we would encourage monitoring and control processes. In Animation, students should be encouraged to summarize. In our current research, we have successfully used experimenter prompts to get students to summarize; this could easily be embedded in a CBLE. Finally, when Not in Environment, students should be encouraged to judge their Feeling of Knowing, engage in Prior Knowledge Activation, and Take Notes. In our current research [17], we have also successfully used experimenter prompts to get students to judge their Feeling of Knowing; this could easily be embedded in a CBLE. Also, before students move to a new section in the environment, they could be prompted to read over their notes, recall what they learned previously, and consider revising their notes.

Acknowledgments This research was supported by funding from the National Science Foundation (REC#0133346) awarded to the second author. The authors would like the thank Fielding I. Winters for assistance with data collection. References [1] [2] [3] [4]

[5]

[6] [7] [8] [9] [10] [11] [12] [13] [14]

[15] [16] [17] [18]

[19] [20]

[21]

[22] [23]

Goldman, S. (2003). Learning in complex domains: When and why do multiple representations help? Learning and Instruction, 13, 239-244 Ainsworth, S., Wood, D.J. , & Bibby, P.A. (1996) Co-ordinating Multiple Representations in Computer Based Learning Environments, Proceedings of EUROAI-ED, Lisbon. Ainsworth, S.E & Th Loizou, A. (2003) The effects of self-explaining when learning with text or diagrams. Cognitive Science, 27, 669-681. de Jong, T., Ainsworth, S., Dobson, M., van der Hulst, A., Levonen, J., Reimann, P., Sime, J., van Someren, M., Spada, H. & Swaak, J. (1998) Acquiring knowledge in science and math: the use of multiple representations in technology based learning environments in Spada, H., Reimann, P. Bozhimen, & T. de Jong (Eds) Learning with Multiple Representations (pp. 9-40) Amsterdam: Elsevier Science. Van Labeke, N., & Ainsworth, S.E. (2002). Representational decisions when learning population dynamics with an instructional simulation. In S. A. Cerri & G. Gouardères & F. Paraguaçu (Eds.), Intelligent Tutoring Systems (pp. 831-840). Berlin: Springer-Verlag. Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13(4), 351-371. Mayer, R. E., & Anderson, R. B. (1992). The instructive animation: Helping students build connections between words and pictures in multimedia learning. Journal of Educational Psychology, 84(4), 444-452. Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91(2), 358-368. Mayer, R. E., Heiser, J., & Lonn, S. (2001). Cognitive constraints on multimedia learning: When presenting more material results in less understanding. Journal of Educational Psychology, 93(1), 187-198. Schnotz, W., & Lowe, R. (2003). External and internal representations in multimedia learning. Learning and Instruction, 13, 117-123. Lajoie, S. P., & Azevedo, R. (in press). Teaching and learning in technology-rich environments. In P. Alexander & P. Winne (Eds.), Handbook of educational psychology (2nd Ed.). Mahwah, NJ: Erlbaum. Moore, P. J., & Scevak, J. J. (1997). Learning from texts and visual aids: A developmental perspective. Journal of Research in Reading, 20(3), 205-223. Hegarty, M., & Just, M. A. (1993). Constructing mental models of machines from text and diagrams. Journal of Memory & Language, 32(6), 717-742. Kozma, R. B., & Russell, J. (1997). Multimedia and understanding: Expert and novice responses to different representations of chemical phenomena. Journal of Research in Science Teaching, 34(9), 949968. Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121-152. Lewalter, D. (2003). Cognitive strategies for learning from static and dynamic visuals. Learning and Instruction, 13, 177-189. Azevedo, R., & Cromley, J. G., (2004). Does training on self-regulated learning facilitate students' learning with hypermedia? Journal of Educational Psychology, 96, 523-535. Chi, M. T. H. (2000). Self-explaining expository texts: The dual process of generating inferences and repairing mental models. In R. Glaser (Ed.), Advances in Instructional Psychology (pp. 161-238). Mahwah, NJ: Lawrence Erlbaum. Pintrich, P. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 452-502). San Diego, CA: Academic Press. Winne, P. (2001). Self-regulated learning viewed from models of information processing. In B. Zimmerman, & D. Schunk (Eds), Self-regulated learning and academic achievement: Theoretical perspectives (pp. 153-189). Mawah, NJ: Lawrence Erlbaum. Zimmerman, B. (2001). Theories of self-regulated learning and academic achievement: An overview and analysis. In B. Zimmerman, & D. Schunk (Eds), Self-regulated learning and academic achievement: Theoretical perspectives (pp. 1-38). Mawah, NJ: Lawrence Erlbaum. Cox, R. and Brna, P. (1995). Supporting the use of external representations in problem solving: the need for flexible learning environments. Journal of Artificial Intelligence in Education, 6(2/3), 239-302. Brusilovsky, P. (2001). Adaptive hypermedia. User Modelling & User-Adapted Interaction, 11, 87-110.