Running head: SELF-EXPLANATION AND PREDICTIVE REASONING
The Effect of Self-explanation and Predictive Reasoning when Learning to Play Chess
Anique B.H. de Bruin, Remy M.J.P. Rikers, and Henk G. Schmidt Erasmus University Rotterdam, The Netherlands
Correspondence concerning this article should be addressed to: A. de Bruin Department of Psychology Erasmus University Rotterdam, WJ5-09 P.O. Box 1738 3000 DR Rotterdam The Netherlands, e-mail:
[email protected]
Self-explanation and prediction 2 Running head: SELF-EXPLANATION AND PREDICTIVE REASONING
The Effect of Self-explanation and Predictive Reasoning when Learning to Play Chess
Self-explanation and prediction 3 Abstract The present study was designed to test whether self-explanation and prediction are effective instructional techniques for novices learning to play chess. In the learning phase of the experiment, participants in three conditions either observed (i.e., the control condition), predicted, or self-explained and predicted the moves of the computer playing an endgame of chess. In the test phase participants played five versions of the endgame against the computer. The participants in the self-explanation condition more often checkmated the black king than those in the two other conditions. No differences emerged between the prediction and control condition. Moreover, the self-explanation condition showed better understanding of the principles of the endgame than the two other conditions. Even for novices, and in a visuospatial task, generating self-explanations while studying had a positive effect on learning. Participants in the self-explanation condition showed better understanding of the principles that underlie the endgame, as indicated by the fact that they more often made the optimal move.
Key words: self-explanation, chess, skill acquisition, novices
Self-explanation and prediction 4 The effect of self-explanation and predictive reasoning when learning to play chess In a classic study by Chi and colleagues (Chi, Bassok, Lewis, Reimann, & Glaser, 1989), individual differences in self-generated explanations of students while solving worked-out examples in the domain of physics were investigated. It was found that good problem solvers produced more extensive think-aloud protocols, containing more physics related explanations than the poor problem solvers’ protocols. Moreover, good problem solvers’ think-aloud protocols contained more signs of self-monitoring by explaining the consequences of an action, or providing a goal for a set of actions. Not only did the good problem solvers study the examples longer, they also expressed higher metacognitive awareness by verbalizing comprehension failures more often than the poor learners. Chi and colleagues (1989) concluded that the extent to which learners benefit from the physics worked examples depended directly on how well participants had explained the study materials to themselves. The high quality, problem-related verbalizations produced by good problem-solvers were termed self-explanations (Chi et al., 1989). According to Neuman, Leibowitz, and Schwarz (2000) self-explanations can be defined as utterances that are primarily focused at clarification of the problem situation and inference of new knowledge. The beneficial effect of self-explanation during problem solving was soon replicated and extended in several experiments (e.g., Bielaczyc, Pirolli, & Brown, 1995; Chi, Chiu, & LaVancher, 1994; Neuman & Schwarz, 1998; Renkl, 1997; Renkl, Stark, Gruber, & Mandl, 1998). These studies showed that not only high quality selfexplanations fostered learning, but also that self-explaining by itself resulted in greater learning gains than merely reading the study material twice. The studies by Chi and colleagues, however, suffered from a lack of control for time on task: participants in the selfexplanation condition varied significantly in time spent studying the material, which could in part be responsible for the learning effect. However, Renkl (1997) showed that when
Self-explanation and prediction 5 study time was fixed, the quality of the self-explanations still differentiated the successful from the unsuccessful learners. As to the cognitive mechanisms that might mediate learning from self-explanations, several possibilities have been proposed. VanLehn and Jones (1993) argue that verbalization during learning requires learners to process the material in a more deliberate way, thereby leading learners to identify and fill knowledge gaps. Moreover, Chi and colleagues (1994) emphasize that self-explanations stimulate the integration of newly learned information with prior knowledge. For example, thirty percent of the self-explanations in Chi’s data could be classified as an integration of new information with previous knowledge. Finally, Neuman and Schwarz (2000) add that self-explanations support the execution of existing problem solving strategies and thereby aid learning. In previous research on the self-explanation effect participants usually possessed a certain amount of prior knowledge of the topic of study. Participants were for example high school or university students who studied a new problem from a topic they had been studying for some time or that required solution procedures students were already familiar with (e.g., biology, Chi et al., 1994; Ainsworth & Loizou, 2003; mathematics, Mwangi & Sweller, 1998; probability calculation, Renkl, 1997). Therefore, Renkl and colleagues (Renkl, Atkinson, & Große, in press) argue that generating self-explanations when learning is particularly helpful for students at the beginning of the intermediate phase of skill acquisition, and less so for students in the early or late phases. In the intermediate stage of skill acquisition, learners have basic domain knowledge and have to direct attention to learning how to solve problems. By contrast, the early phase of skill acquisition is characterized by acquiring basic knowledge about the principles of the domain of study (VanLehn, 1996). Although Chi and colleagues (1994) argue that self-explanation involves to a large extent the integration of new information with prior knowledge, other processing
Self-explanation and prediction 6 characteristics of self-explanations have been identified that might promote understanding when learners do not possess prior knowledge. For example, Neuman and Schwarz (1998) emphasize that self-explanations provide support in uncovering the deep structure of the problem representation, whereas Mwangi and Sweller (1998) add that self-explanations ensure that learners attend appropriately to the instructional material. Whether generating self-explanations during the early phase of skill acquisition can promote learning has however never been studied empirically. It is therefore interesting to examine to what extent the self-explanation effect remains intact when participants have no prior topic knowledge. Most research on the self-explanation effect provided learners with instructional material that explicitly contained the to be learned information (e.g., expository text, Chi et al., 1994, worked examples, Mwangi & Sweller, 1998, programming lessons, Bielaczyc et al., 1995). In these cases, the self-explanation manipulation was used to foster deep processing and thorough understanding of the material to be learned. To our knowledge, no studies exist that test whether providing self-explanations promotes learning when no explicit information about the principles of the domain is given. The question arises to what extent generating self-explanations fosters learning when it is employed not as a learning support, but rather as the primary instructional technique to acquire domain-relevant skills. For example in chess, a distinction can be made between the rules and the principles that are needed to adequately play the game. The rules cover the legal moves the pieces can make and knowledge of concepts as check, checkmate, and stalemate. Without knowledge of these rules, it is impossible to learn the principles that are necessary to play the game. Therefore, understanding of the rules is a necessary condition to learn the principles of the game. The principles that for example underlie an endgame of rook and king against king include knowing how to limit the space of the king by collaboration of the pieces and knowing that a king is always checkmated at the edge of the board. It is clear that merely knowing the rules
Self-explanation and prediction 7 does not make one a chess master. This requires extensive analysis of numerous chess games. By actually studying chess games, players can infer the principles and improve their playing strength. A clear distinction is therefore made between the rules and principles of chess. For example, understanding of the rules can be accomplished relatively fast, whereas learning to play at grandmaster level may take more than ten years (Simon & Chase, 1973). When using chess as the domain of study, novices will first have to get acquainted with the basic chess rules before proceeding to the actual study of chess games. Using chess as the topic of study provides the opportunity to examine the effect of self-explanations when explicit information about the principles of the domain is absent. Learners are required to deduce the principles by actively studying multiple examples of chess games. When relevant information is not explicitly given, but is left to the discovery of the learner, generating self-explanations might result as an effective method to structure thinking and discover and store relevant procedures (Neuman & Schwarz, 1998). Under these circumstances self-explanations cannot be based on rereads or rephrasing of textual information (cf. Chi et al., 1994). Learners will have to pass on directly to generating possible explanations for the steps observed in the problem solving procedure. The present study was undertaken to investigate the influence of generating selfexplanations when learning to play chess without having prior knowledge. Participants were novices who upon entering the experiment had no knowledge about chess whatsoever. These participants are in the early phase of skill acquisition according to VanLehn’s (1996) classification. After studying an introductory presentation covering the basic rules of chess, participants entered the learning phase and had to learn to play the chess endgame of rook and king against king through observing a set of games played by a computer. Participants were assigned to one of three conditions. In the self-explanation condition, participants were asked to explain and predict the next move that the computer would make for white (rook
Self-explanation and prediction 8 and king). The requirement to predict the next move was chosen to approach a normal chess game as accurately as possible, which can be described as generating the best move given a certain chess position. Moreover, asking learners to predict the next move required them to reflect not on the computer move, but rather on the quality of their understanding of the computer move and on possible discrepancies between prediction and computer move. This procedure has correspondence with Ericsson’s (Ericsson, Krampe, & Tesch-Roemer, 1993; Ericsson, 1996) description of one of the aspects of deliberate practice, that is, providing opportunities for repetition and correction of errors: learners are required to repeatedly produce measurable output (predict the next move) and afterwards receive feedback on its accuracy, which allows them to correct errors in understanding. In order to be able to separate a possible self-explanation effect from an effect of predicting the next move, the prediction condition (cf. Stark, 1998) only asked participants to predict the next move of the computer without further verbalizations. The control (or observation) condition merely observed the chess play of the computer without predictions or other verbalizations. In the test phase, all participants had to play five examples of the endgame against the computer and to checkmate the black king in as few moves as possible. We hypothesized that participants in the self-explanation condition would be better able to predict the next move of the computer and in more test exercises be able to checkmate the black king than the other two conditions. Moreover, because prediction alone can be considered a metacognitive activity as well, we predicted that the prediction condition would outperform the observation condition in the test phase. As to the discovery of the chess principles that underlie the endgame of rook and king against king, we hypothesized that the same pattern would occur: the self-explanation condition would discover more principles of the endgame than the other two conditions, whereas the prediction condition would outperform the observation condition.
Self-explanation and prediction 9 Method Participants Participants were 45 first-year psychology students from Erasmus University Rotterdam, The Netherlands. They were randomly assigned to one of the three conditions. To prevent differences in prior knowledge and skill, only students who had never played chess before, nor were aware of any of the rules of chess, were allowed to participate. The students participated to fulfill course requirements, and moreover were awarded a small financial compensation. Materials At the beginning of the experiment all participants were shown a computerized presentation providing information about the basic rules of chess. Since it was impossible to learn the complete game during the length of one experiment, a subpart of the game that could be studied in isolation was selected. This subpart involved the theoretical endgame of rook and king against king. In this endgame the side with the rook and king (the white side in this study) is theoretically always able to checkmate the black king. On a computer screen the rules regarding the legal moves for king and rook were visualized, as well as the basic rules of taking, check, checkmate, and stalemate. The presentation consisted of the general rules of chess. It is important to note that these can be clearly distinguished from the principles that underlie the endgame: knowing what the legal moves of a rook and king are does not tell a learner anything about how to accurately checkmate a king in this endgame. Participants were allowed to study the presentation in a self-paced way. This took on average about 20 minutes. The chess computer program ‘Shredder 6’ was used to provide chess exercises, which consisted of examples of the endgame of rook-king against king played by the computer. This chess program plays at Grandmaster level (ELO rating of approximately
Self-explanation and prediction 10 2700). The chess exercises that the program generated ranged in difficulty from checkmate in 6 to checkmate in 10 moves. Moreover, a digital chessboard was connected to Shredder 6. It was used for three purposes. First, after the introductory presentation, the experimenter tested the participants’ knowledge of the basic chess rules by asking questions about positions presented on the chessboard. For the chess exercises the experimenter moved the pieces on the chessboard suggested by the computer. Finally, in the test exercises the digital chessboard registered the moves that the participants made, and Shredder 6 generated a countermove, which the experimenter played on the board. Move prediction times and selfexplanations were recorded by an audio tape recorder. Procedure Given the complexity and duration of the task, the experiment was run during three separate sessions, each lasting about one and a half hour. For all participants the three sessions were planned within a one-week interval. In the first session participants studied the computer presentation about the basic rules of chess. Afterwards the experimenter tested in a questionand-answer format whether the participant had understood the rules. The questions were of the kind: “In this situation, what moves can the rook make?”, or “In this situation, is the black king in checkmate?”. The purpose of this phase was to ensure a minimum baseline of understanding before proceeding to the practice phase. Participants were not allowed to proceed to the learning phase of the experiment before being able to answer all questions correctly. The chess exercises were designed to enable participants to infer the principles that are specific to the endgame of rook and king against king and are essential to be able to play adequately against the computer. These principles were nowhere named, but had to be discovered by the participants by studying the computer moves. An example of a chess position used in the experiment is provided in Figure 1.
Self-explanation and prediction 11 -----------------------------------------------------------------------------------------------------------Insert Figure 1 about here -----------------------------------------------------------------------------------------------------------Participants in the so-called prediction and self-explanation condition (hereafter referred to as PSE) were asked to predict every next move the computer would make for white (rook and king), and to verbally explain which considerations led them to predict this move. Participants could take as long as they wished to do this. If participants predicted a move different from the one subsequently produced by the computer, they were asked to try to explain the discrepancy between predicted and actual move. Participants in the prediction only (PO) condition also predicted the next move of the computer, but were not allowed to provide self-explanations when generating a prediction. Participants in the observation condition were instructed to observe the moves the computer made in the chess exercises without any verbalizations or predictions. To control for time on task, the mean study times participants in the PSE condition needed to predict the next move was used to determine the prediction and observation times for the different moves in the prediction and observation condition. Participants studied four chess exercises in the first session. The second session started with a short rehearsal of the basic rules of chess in the same question-and-answer format as in the first session. Afterwards, six chess exercises were studied under the same instruction as the first session. Participants were either asked to observe, predict, or selfexplain and predict the next moves of white. The third session consisted of five test exercises in which participants were required to play the endgame with white (rook and king) against the computer and checkmate the black king in as few moves as possible. The difficulty of the test exercises was comparable to the chess exercises in the learning phase, ranging from mate in 6 to mate in 10 moves. Participants were given a maximum of 15 minutes for each test situation. Following the
Self-explanation and prediction 12 rules of the World Chess Federation, a game ended in a draw when 50 moves were made and no piece was taken, the black king took the white rook (no checkmate possible anymore), or the same position reoccurred three times in the same game. After finishing the test, participants were asked to fill out a short questionnaire containing questions about how participants rated their own level of understanding of the principles of the endgame and about how much they enjoyed studying the endgame. Analysis The predictions of the participants in the PSE and PO condition in the learning phase were compared to the computer’s optimal move. When a predicted move was identical to the move of the computer it was scored one, whereas predicted moves that were different were scored zero. Since there were often several correct options, the predicted move was also scored one if it was equally good as the computer move. To acquire more specific information about the principles that the participants applied in their predictions in the learning phase and in the moves in the test exercises, these were analyzed by a chess master (a 21-year old female member of the Dutch youth team). The chess master together with the first author identified the different principles that apply to the theoretical endgame rook and king against king. Six principles were identified, among them: limiting the space of the black king with the rook, driving the black king to the edge of the board, and minimizing the difference between the two kings. Because these principles do not differ much in difficulty and because they are all important to understand the endgame, they were analyzed not separately but as a whole. Next, the chess master scored the predictions in the chess exercises and the actual moves participants made in the test exercises. She indicated for each move whether the participant chose the optimal move and if so, which principle was applied. If the move was not optimal, the chess master registered
Self-explanation and prediction 13 which principle the participant failed to apply. Thus, every move was qualified as either correct or incorrect use of the principle that was currently applicable. Repeated measures ANOVAs were applied to the data. First, the relation between quality and number of correct predictions on the one hand and treatment on the other was tested, using study session (first or second) as a within-subjects factor and condition (PSE or PO) as the between-subjects factor. Moreover, repeated measures analyses were applied to the number of times checkmate occurred in the test phase, the quality of the moves in the test exercises and the number of errors made between the different conditions using subsequent test exercises as a within-subject factor and condition as a between-subject factor. A multivariate ANOVA was performed on the items of the motivation and understanding questionnaire. To allow for comparison between conditions, post-hoc analyses were conducted using Bonferroni’s method. Finally, through correlational analysis the relation between the amount of time spent on self-explanations and the number of checkmates in the test phase was analyzed. Interaction effects will only be mentioned when significant. All alpha levels were set at .05.
Results Learning phase To determine the study times per move for the PO and observation condition, the thinking times per move for the PSE condition were calculated. Participants in the PSE condition needed on average 42.5 seconds (SD = 19.6) to predict the next move. After the computer generated the optimal move, participants needed on average 12.3 (SD = 4.2) seconds to evaluate their prediction and explain possible discrepancies between their prediction and the computer’s move. The study times differed somewhat between the various chess exercises, with more complex situations resulting in longer study times.
Self-explanation and prediction 14 As to the number of accurate predictions participants in the PSE and PO condition made about the computer’s next moves, the difference between the conditions was significant, F(1, 28) = 6.71, MSE = 126.81, p < .05, η2 = .19. Participants in the PSE condition scored significantly higher than participants in the PO condition. Looking at the quality of the predictions, the results show that participants in the PSE condition were significantly better at applying the correct principles than participants in the PO condition, F(1, 28) = 14.47, MSE = 164.35, p < .001, η2 = .34. Moreover, there was an effect of session, F(1, 28) = 7.22, MSE = 59.03, p < .05, η2 = .21. In the second session, participants in both conditions applied the principles better than in the first session. Table 1 presents the percentage of correct predictions per study session and the percentage of predictions that reflected the correct application of the principles. To examine a possible learning effect over sessions, the results for the first session were analyzed separately as well. At the end of the first session, participants had studied four chess exercises under a PSE or PO condition. No difference was found between conditions for the percentage of moves predicted correctly, F(1, 28) = 2.12, whereas as to the principles applied correctly, participants in the PSE condition already scored significantly higher, F(1, 28) = 7.96, MSE = 104.65, p < .01, , η2 = .22. Thus, although participants in the PSE condition were in the first session not better at predicting the next move, they already had benefited from the self-explanation instruction and outperformed the PO condition in the application of the endgame principles. Since none of the participants had any prior knowledge about chess and the discovery of the endgame principles is not a straightforward endeavor, this result is attributable to the effect of the self-explanation manipulation. -------------------------------------------------------------------------------------------------------------Insert Table 1 about here
Self-explanation and prediction 15 --------------------------------------------------------------------------------------------------------------Test phase Repeated measures ANOVA revealed that the three conditions differed significantly in number of checkmates in the test exercises, F(2, 42) = 11.53, MSE = 4.49, p < .001, , η2 = .35. Posthoc tests using Bonferroni’s method indicated that the PSE condition checkmated the black king significantly more often than the PO condition, p < .001, and the observation condition, p < .01. The difference between the PO and observation condition was not significant. The mean number of checkmates for the PSE condition was 3.0 (SD = 1.77), whereas for the PO and observation condition this was 0.87 (SD = 0.92) and 1.33 (SD = 1.68), respectively. The results per test situation for percentage of checkmate per condition are represented in Figure 2. ---------------------------------------------------------------------------------------------------------Insert Figure 2 about here ---------------------------------------------------------------------------------------------------------When participants were able to checkmate the black king, they needed on average 18.8 moves (SD = 8.5), 25.1 moves (SD = 12.7), and 16.9 moves (SD = 12.5) in the PSE, PO and observation condition, respectively. These differences were not significant, F(2, 26) = 1.16. When analyzing the total number of moves participants needed in the test exercises, interpretation of the results is difficult. If a game ended after 6 moves, at least two options are possible: The black king took the rook or the game ended in checkmate. The first option represents inadequate understanding of the endgame, whereas the second option indicates accurate understanding. Therefore, the total number of moves is not necessarily a direct representation of the quality of play. Instead of analyzing the number of moves an exercise lasted, we analyzed the quality of the moves, by looking at the number of moves in the test that represented correct application of the endgame principles.
Self-explanation and prediction 16 Repeated measures ANOVA with test exercise as a within subject factor revealed that there was a significant difference in the correct use of the principles relevant to the rook and king against king endgame, F (2, 42) = 5.02, MSE = 4603.46, p < .05, η2 = .19. Posthoc analyses indicated that participants in the PSE condition more often applied the principles correctly than participants in the PO condition and the observation condition. ---------------------------------------------------------------------------------------------------------Insert Table 2 about here ---------------------------------------------------------------------------------------------------------Apart from the principle errors mentioned above, a second category of so-called procedural errors was identified. This category covered the number of times a game ended because 50 moves were made, 15 minutes had passed or the same position had occurred three times. The results indicated that there was a significant difference in the number of procedural errors between conditions, F(2, 42) = 3.86, MSE = 6.96, p < .05, η2 = .16. Posthoc analysis revealed that the PSE condition made fewer procedural errors than the PO condition. Participants in the PSE condition committed on average 1.5 procedural errors (SD = 1.3), while for the PO and observation condition this was 2.8 (SD = 1.2) and 2.6 (SD = 1.5), respectively. The correlation between the amount of time participants spent on self-explanation in the learning phase and the number of checkmates in the test phase was non-significant. The same was found for the relation between the amount of time participants in the PSE condition needed to evaluate the discrepancy between their prediction and the computer move of on one hand and the number of checkmates in the test phase on the other hand. Although the conditions differed in the number of times checkmate and in the correct use of principles, there were no differences in the time participants needed to select their next move in the test exercises, F < 1. Participants in the PSE, PO and observation condition
Self-explanation and prediction 17 needed on average respectively 22.9 seconds (SD = 7.7), 23.4 seconds (SD = 8.0), and 21.0 seconds (SD = 7.6) to select their next move. This indicates that although participants in the PO and observation condition performed worse than the PSE condition in the test phase, this was not due to a lack of motivation to study the test exercises thoroughly.
Self-ratings of understanding and motivation After the test phase, all participants filled out a short questionnaire containing 10 items that covered how well they thought they understood the principles of the endgame and how much they enjoyed studying the endgame. Except for one occasion, there were no differences between the conditions on these items. All three conditions enjoyed participating equally, F (2, 41) = 1.23, and were equally motivated to learn more about chess, F (2, 41) = 1.44. Moreover, there were no differences between conditions on the items considering the perceived difficulty of the learning phase, all Fs < 1. Only on the item “I understand the principles of the endgame” there was a significant difference between conditions, F (2, 41) = 3.45, MSE = 3.33, p < .05, η2 = .14. Posthoc analysis revealed that the observation condition rated higher on this item than the PO condition, p < .05, whereas there was no difference between the PSE and PO condition. These differences could be attributable to poor metacognitive awareness of the participants in the observation condition. Since they never received feedback about the quality of their predictions in the learning phase, participants in the observation condition were possibly unaware of their own level of understanding and overrated this on the questionnaire. Discussion This study examined the separate and combined effects of self-explaining and predicting while learning to play chess. Participants had to learn an endgame of chess by observing several chess exercises provided by a chess computer. In the prediction and self-explanation
Self-explanation and prediction 18 (PSE) condition, participants were required to explain and predict the next move of the computer. Participants in the prediction only (PO) condition explicated their prediction of the next move without further verbalizations. In the observation condition, participants merely observed the moves of the computer. In the test phase, participants played 5 versions of the endgame against the computer. The results indicated that predicting next moves and providing self-explanations while doing so fostered learning of the endgame: Participants in the PSE condition were in more test exercises able to checkmate the black king than participants in the PO or observation condition. Moreover, the results suggested that participants in the PSE condition showed better understanding of the principles that underlie the endgame, as indicated by the fact that they more often made the optimal move. When participants were required to independently play the endgame in the test phase, participants in the PSE condition proved to be better able to transfer the principles learned to novel problems. These results further strengthen the evidence for the self-explanation effect (Chi et al., 1989; Renkl, 1997). The finding that there were no differences in thinking times per move in the test phase between the three conditions and the results from the questionnaire indicate that there were no differences between the three conditions in enjoyment of the endgame and in motivation to do their utmost. In addition to differences in the number of times participants checkmated the black king, participants in the PSE condition were already in the learning phase better able to predict the next move of the computer and to apply the principles of the endgame than participants in the PO condition. This indicates that already early on in the learning phase participants in the PSE condition benefited from generating self-explanations. In previous self-explanation research, participants usually possessed a certain level of prior knowledge of the topic of study. Moreover, the information to be learned was generally more or less explicitly given in a textual or worked example format (e.g., Chi et
Self-explanation and prediction 19 al., 1989; Chi et al., 1994; Renkl, 1997). In the present study both of these conditions did not apply. Despite the fact that participants were new to the field of chess and received no explicit information about the principles of the endgame, generating self-explanations while studying chess positions nevertheless fostered discovery of the endgame principles. These data suggest that the effect of self-explanation also emerges when integration with prior knowledge is not possible. As a consequence, our results are not in agreement with Renkl’s assumption (Renkl et al., in press) that generating self-explanations is not beneficial for learning in the early phase of skill acquisition. To explain these findings, we consider the differences in the nature of the study material between previous research on self-explanations and the present study. When learners have to generate self-explanations of study material that is to a certain extent of a textual nature, it is possible that the self-explanations remain at a reread or problem restatement level. In that case, learners are not forced to meaningfully explain the material to themselves but mostly reformulate what was already said in the text. For instance, Mwangi and Sweller (1998) found that about 39% of the self-explanations in their study could be classified as rereads. In the present study, extensive restating of the problem situation was virtually impossible, simply because no explicit description of the problem situation or of the solution procedure was provided. All information participants received were a chess position and the requirement to predict the next move of the computer. Therefore, instead of remaining at a reread level, participants were forced to provide meaningful self-explanations about the possible moves white could make. One has to assume that this condition encouraged thoroughness of processing, supporting discovery of the endgame principles. This interpretation is supported by the finding that in the first session a performance difference between the PSE condition and the PO condition already appeared: Directly from the start of the learning phase the self-explanations participants generated were meaningful
Self-explanation and prediction 20 and therefore promoted understanding. The requirement to verbally explain the computer’s next move forced participants to more deliberately reflect on the study material and hence prevented shallow representation of implicit procedural knowledge (Aleven & Koedinger, 2002). There is however an alternative to the thoroughness-of-processing hypothesis. This alternative explanation for the present findings takes into account the visual nature of the material and the verbal nature of the self-explanations. Because of the requirement to selfexplain visual information, participants were stimulated to translate the problem from a visual to a verbal mode. This possibly led to a more extensive problem representation and prevented shallow understanding. This interpretation is supported by previous research on self-explaining when studying graphics (Ainsworth & Loizou, 2003; Aleven & Koedinger, 2002). However, in these studies learners always received a combination of graphics and text. In the present study, text was absent. Our study shows that the self-explanation effect when learning from graphics is not only due to the way graphics stimulate the explanation of the textual information, but possibly also to the way self-explaining promotes the construction of more elaborate representations in both verbal and visual form. The multimode character of the problem representation might provide an explanation for the self-explanation effect in novices. The present study is a first attempt to study the effect of self-explaining when learning to play chess. The results clearly show that, even for novices, generating self-explanations when predicting the next move fosters learning of the chess principles more than either predicting without verbalizing or merely observing. These findings appear to have implications for chess training. However, before being able to formulate training principles, some questions remain that require clarification in future research.
Self-explanation and prediction 21 First, it is unclear to what extent these results are replicated when learners of higher skill level (i.e., chess players of an intermediate or experienced level) are subjected to a similar manipulation. Research on the expertise reversal effect indicates that instructional techniques that have proven useful for novices can actually impede learning for experienced performers (for a review, see Kalyuga, Ayres, Chandler, & Sweller, 2003). For example, Kalyuga and colleagues (Kalyuga, Chandler, & Sweller, 1998) found that in an experiment on studying electrical circuits, experienced learners were hampered by an instruction that integrated textual explanations with diagrams, whereas inexperienced learners benefited from this procedure. For the experienced learners, the textual information was redundant and therefore best eliminated. Future research is needed to examine whether a similar effect is observed when asking experienced chess players to self-explain predictions of a computer’s chess moves. In the present experiment, the instruction for the PO and PSE condition departed from the learner’s perspective by asking participants to either predict or predict and self-explain the computer moves. This allowed participants to reflect on the accuracy of their own predictions and correct the principles they deduced concerning the endgame of rook and king against king. In this way, the computer moves functioned as feedback on the accuracy of participants’ predictions and the corresponding inferred principles. This procedure is in agreement with Ericsson’s (Ericsson et al., 1993; Ericsson, 1996) description of the repetition and correction of errors assumption of deliberate practice: Learners repeatedly predicted the computer moves and received feedback on its accuracy enabling them to correct errors in understanding. Instead, departing from the computer’s perspective by asking participants only to explain the computer moves (i.e., a self-explanation only condition) would stimulate learners less to reflect on their own level of understanding and correct errors, since learners are not required to self-generate solutions that are evaluated
Self-explanation and prediction 22 afterwards, but are limited to explaining computer provided moves, without self-reflection. Nevertheless, it would be interesting to further examine the importance of repetition and correction of self-generated errors during learning by contrasting in future research a prediction and self-explanation condition with a self-explanation only condition that asks learners to explain the computer moves. A final question regards the difference between the self-explanation and prediction condition in explanations of discrepancies between predicted moves and computer moves. Contrary to the PSE condition, participants in the prediction condition were not asked to self-explain any discrepancies between their predictions and the actual moves of the computer. It is unclear to what extent this attributed to the difference in performance between these two conditions. However, the time learners needed to explain the discrepancies was relatively short compared to the time learners took to predict the next move (12 seconds versus 43 seconds, respectively). Nevertheless, it would be interesting to investigate in future research and in a more explicit manipulation in what way the computer feedback stimulated the learning process. For example, Renkl (1999) mentions that having students generate self-explanations without providing feedback can lead to ‘illusions of understanding.’ If students do not receive feedback about the accuracy of the content of their self-explanations, they are not encouraged to identify and correct misconceptions and might persevere in illusions of understanding. Providing learners with feedback about the quality of the content of their self-explanations might be a possibility to immediately correct misconceptions and prevent illusions of understanding.
Self-explanation and prediction 23 References Ainsworth, S., & Loizou, A. Th. (2003). The effects of self-explaining when learning with text or diagrams. Cognitive Science, 27, 669-681. Aleven, V., & Koedinger, K. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based cognitive tutor. Cognitive Science, 26, 147-179. Bielaczyc, K., Pirolli, P.L., & Brown, A.L. (1995). Training in self-explanation and selfregulation strategies: investigating the effects of knowledge acquisition activities on problem solving. Cognition and Instruction, 13(2), 221-252. Chi, M. T. H.., Chiu, M., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439-477. Chi, M. T. H., Lewis, M.W., Reimann, P., & Glaser, R. (1989). Self-explanations: how students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182. Ericsson, K.A., Krampe, R.T., & Tesch-Roemer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100(3), 363-406. Ericsson, K.A. (1996). The road to excellence: The acquisition of expert performance in the arts and sciences, sports and games. Mahwah, NJ: Lawrence Erlbaum Associates. Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect. Educational Psychologist, 38(1), 23-31. Kalyuga, S., Chandler, P., & Sweller, J. (1998). Levels of expertise and instructional design. Human Factors, 40, 1-17. Mwangi, W., & Sweller, J. (1998). Learning to solve compare word problems: the effect of example format and generating self-explanations. Cognition and Instruction, 16(2), 173-199.
Self-explanation and prediction 24 Neuman, Y., & Schwarz, B. (1998). Is self-explanation while solving problems helpful? The case of analogical problem-solving. British Journal of Educational Psychology, 68, 15-24. Neuman, Y., Leibowitz, L., & Schwarz, B. (2000). Patterns of verbal mediation during problem solving: a sequential analysis of self-explanation. The Journal of Experimental Education, 68(3), 197-213. Neuman, Y., & Schwarz, B. (2000). Substituting one mystery for another: the role of selfexplanations in solving algebra word-problems. Learning and Instruction, 10, 203220. Pirolli, P., & Recker, M. (1994). Learning strategies and transfer in the domain of programming. Cognition and Instruction, 12(3), 235-275. Renkl, A. (1997). Learning from worked-out examples: a study on individual differences. Cognitive Science, 21(1), 1-29. Renkl, A., Atkinson, R.K., & & Große, C.S. (in press). How fading worked-out solution steps works: A cognitive load perspective. Instructional Science. Renkl, A. (1999). Learning mathematics from worked-out examples: analyzing and fostering self-explanations. European Journal of Psychology of Education, 14(4), 477-488. Renkl, A., Stark, R., Gruber, H., & Mandl, H. (1998). Learning from worked-out examples: the effects of example variability and elicited self-explanations. Contemporary Educational Psychology, 23, 90-108. Simon, H.A., & Chase, W.G. (1973). Skill in chess. American Scientist, 61, 394-403. Stark, R. (1998). Lernen mit Lösungsbeispielen. Der Einfluß unvollständiger Lösungsschritte auf Beispielelaboration, Motivation und Lernererfolg [Learning by worked-out examples. The impact of incomplete solution steps on example
Self-explanation and prediction 25 elaboration, motivation, and learning outcomes]. Unpublished dissertation. University of Munich, Germany. VanLehn, K. (1996). Cognitive Skill Acquisition. Annual Review of Psychology, 47, 513539. VanLehn, K., & Jones, R. (1993). What mediates the self-explanation effect? Knowledge gaps, schemas or analogies? In M. Polson (Ed.), Proceedings of the fifteenth annual conference of the cognitive science society (pp. 1034-1039). Hillsdale, NJ: Erlbaum.
Self-explanation and prediction 26 Author notes The authors would like to thank Jill Whittingham, Conny Hillebrand and Desirée Hamelink for their help in performing the experiment and analyzing the chess moves.
Self-explanation and prediction 27 Table 1 Mean percentage of correct predictions and predictions that expressed correct application of principles in chess exercises in the learning phase. Standard deviations between brackets. Session
PSE condition
PO condition
Percentage of correct predictions 1
64.80 (10.86)
59.27 (9.91)
2
67.47 (6.45)
57.93 (8.57)
Total
66.14 (8.66)
58.6 (9.24)
Percentage of principles correctly applied 1
58.89 (10.78)
48.35 (9.65)
2
66.27 (10.13)
51.10 (11.62)
Total
62.58 (10.46)
49.73 (10.64)
Self-explanation and prediction 28 Table 2 Mean percentage of moves that expressed correct use of chess principles. Standard deviations between brackets. Testsituation
PSE condition
PO condition
Observation condition
1
62.39 (25.46)
49.85 (21.69)
46.14 (27.27)
2
62.91 (20.69)
50.35 (15.38)
52.99 (18.28)
3
67.29 (14.61)
48.75 (15.80)
49.32 (20.38)
4
63.04 (19.31)
51.48 (15.23)
48.54 (14.96)
5
55.43 (16.67)
40.05 (19.71)
49.33 (15.21)
Total mean
62.21 (14.24)
48.10 (9.35)
49.26 (16.11)
Self-explanation and prediction 29 Figure Caption Figure 1. Example of a chess position used in the learning phase. Figure 2. Mean percentage of participants who checkmated the black king per condition.
Self-explanation and prediction 30
Self-explanation and prediction 31
Mean percentage checkmate
100 90 80 70 60
PSE condition PO condition Observation condition
50 40 30 20 10 0 1
2
3
Test exercise
4
5
