Representation interpretation versus representation construction: An ILE–based study using switchERII. Richard Cox Human Communication Research Centre University of Edinburgh
[email protected]
Abstract External representations (ERs) such as diagrams are useful aids to reasoning and they are exploited in many intelligent educational systems. However, such systems vary in the extent to which they a) permit the user to interact with representations presented by the system and b) permit users to construct their own representations. These issues are important from a constructivist point of view, but have not been widely studied to date. In this study, subjects’ performance on a diagram interpretation task was compared with their performance on an analytical reasoning task in which they constructed their own diagrams using switchERII, an interactive learning environment. SwitchERII incorporates the semantics of Euler’s Circles (a type of set diagram) and dynamically parses the user’s representation. Results indicate that some, though not all, types of error in diagram interpretation predict errors in subsequent external representation construction. Some subjects make interpretation errors and diagram construction errors and others make errors on one task, but not on both. Errors made during ER construction do not necessarily result in poor problem solving performance in terms of response accuracy on problem questions. Cognitive and attentional processes associated with representation construction and externalisation are discussed, together with the design implications for intelligent educational systems.
1 Introduction External representations (ERs) are an important part of many intelligent educational systems. In some systems, reasoning with ERs is central to the learning activity supported. Examples of this kind of system are provided by Hyperproof [1] which teaches first-order logic using a ‘heterogeneous’ approach (combined ‘Blocks world’ diagram plus logic syntax). Another example, in the domain of algebra word problems, is provided by the Algebra Word Problem Tutor [15]. That system provides a range of diagrams for students to choose between in the course of reasoning about surface area. In other systems, such as Bridge [2], the representation serves a more mediating function. In Bridge, tiling graphical icons in a jigsaw-like fashion helps students design computer programs – the representation places constraints on how program elements can be arranged. Such systems also differ in the degree to which students participate in the construction of the representation. Hyperproof allows students to interact with the blocks world – to add objects, move them, change their labels, sizes, etc. On the other hand, the Algebra Word Problem Tutor presents a range of several static diagrams for the student to choose between. From a constructivist perspective, the issue of interactivity and representation construction is very important. ER construction represents more than simple translation of information, for as Vygotsky observed, when signs (language, diagrams, etc) are included in an action, they do more than facilitate manoeuvres that are impossible in the absence of the sign system. They fundamentally transform the action [19,20]. However, much previous work on reasoning with ERs has failed to distinguish between performance under conditions where subjects interpret presented diagrams and situations in which subjects construct their own
Figure 1: SwitchERII screen showing problem stem, question window, subject’s representation and system feedback window. representations. A notable exception is provided by a study by Grossen & Carnine [8]. Students were taught to use a method based on Euler’s circles to reason about the relationships between plant species. A computer-based tutoring system was employed. Students in one group were required to construct diagrams before progressing through the resource material whereas the students in the other group used pre-drawn computer-based diagrams. Instruction plus self-constructed diagrams was more effective than instruction plus diagram selection. Students in the diagram-construction condition scored more highly on difficult problem types (without valid conclusions) and demonstrated fewer trials to mastery within the course. Gains were retained for at least the duration of a two week follow up. They conclude that active drawing produces deeper processing than more passive diagram selection. Grossen & Carnine [8] provide evidence for the importance of active participation in diagram construction in determining learning outcomes. However, the between-group experimental design utilised did not allow the comparison of representation interpretation and representation construction in the same subjects. This paper presents results from a study of problem solving with external representations (ERs) in the domain of analytical reasoning – i.e. constraint-satisfaction puzzles of the kind found in the US ‘GRE’ examination. An interactive learning environment (switchERII) was used in an investigation of the factors associated with effective ER use in situations where subjects reason with their own, self-constructed, ERs. Subjects’ performance on a pre-test requiring diagram interpretation was compared with their performance under conditions in which they constructed representations from scratch using switchERII. The diagram interpretation pre-test assessed prior knowledge and allowed misconceptions about the representational formalism to be identified. 1.1 SwitchERII – an intelligent learning environment (ILE) for supporting representation construction SwitchERII (Figure 1) was developed from an earlier system (switchERI) and from studies of subjects ‘workscratching’ paper-based external representations [4,5]. In those studies, it was noted, inter alia, that subjects frequently made errors during representation construction, often with negative consequences for problem solving. SwitchERII provides a range of ER construction environments for the user to choose and switch between (logic/text, matrix, graphical). SwitchERII’s graphical ER environment incorporates a representation of the semantics of a particular class of external representations (Euler’s Circles) and can dynamically parse the user’s
representation and provide feedback and advice during ER construction. For example, if the user represented the problem statement ‘All those who enjoy the poetry of Coleridge also enjoy the poetry of Donne’ by drawing a diagram similar to the 4th one in Figure 2, switchERII would respond with the message ‘There may be a problem with the representation...’. After a number of errors of this kind, the system suggests ‘Try using text instead of graphics ...’, as shown in Figure 1. Euler’s circles were chosen as the graphical formalism since they have a well-defined, computationally tractable, underlying semantics. Also, there is a relatively large amount of experimental data on how they are interpreted (e.g. [10,11,16,18]). Within switchERII, there are representations of Euler’s circle semantics, the current state of the user’s diagram and the analytical reasoning problem. Technically, the system’s representations permit detailed, diagnostic feedback regarding the precise nature of the error(s) in the user’s diagram. However, it was decided to provide more ‘flag’ like feedback [13] in order to encourage reflection and other metacognitive processes on the part of the user. SwitchERII also provides feedback about the user’s progress through the problem by means of ticks and arrows in the problem sentence window (Figure 1). The user may re-order the problem premise sentences either before or during ER construction. A fuller description of switchERII can be found in [4]. 1.2 Aims The first aim was to conduct a within-subject comparison of performance under two conditions – diagram interpretation and diagram construction. Studies [4] and [5] showed that subjects’ prior knowledge or repertoire of representational systems is an important predictor of success in reasoning with ERs, especially where indeterminate information must be represented. Another aim of the present study, therefore, was to further investigate the extent to which prior knowledge and misconceptions affect reasoning performance. A final aim was to analyse, in detail, ER construction errors in subjects’ use of a set diagrams and to study the effects of system feedback upon representation construction. The use of switchERII as the problem solving and ER construction environment, together with the logging of users’ interactions, enabled questions such as the following to be addressed:
In what order do subjects construct the elements of their representation? To what extent do subjects detect errors and modify their representations? How much time do subjects spend at various stages of reasoning? What is the effect of providing feedback about the accuracy of the user’s representation? Do subjects use multiple representions? If so, how? The methodological approach adopted here (i.e. the use of an ILE in an empirical investigation) is similar to that of Shute & Gawlick-Grendell [14] who investigated students’ learning in the domain of probability with StatLady, a system which, like switchERII, provides on-line support to the user. 2 Method 2.1 Subjects Sixteen University of Edinburgh students (7 females, 9 males) participated. They were randomly assigned to two conditions – ‘Feedback’ (FB) and ‘No feedback’ (NFB). Subjects in the FB condition used a version of switchERII which generated feedback and guidance during ER construction. The NFB subjects used a version of switchERII which did not display feedback messages, but which was identical in all other respects. Subjects were paid £5 for their participation. 2.2 The reasoning problems Two problems were selected, both of which are solvable using Euler’s Circle set diagrams. The first problem was a practice problem developed by the author. The experimental problem used for data collection was the ‘Poets’ problem employed in previous studies [4,5]. The poets problem is indeterminate in that many different, but valid, diagrammatic models can be constructed of the information. The problem consists of a list of ‘givens’ (problem stem information) together with a series of questions. The problem and one of its associated questions, as presented by switchERII, is shown in Figure 1.
B
A
AB
A
B
1
2
3
A
B
4
A
B
5
Figure 2: Euler’s Circle interpretation task diagrams. 2.3 Assessing prior knowledge – the diagram interpretation task The diagram interpretation pre-test was based on one used by Newstead [10]. In the test, five Euler’s circle diagrams are presented to the subject. Each diagram consisted of two circles – ‘A’ and ‘B’ (Figure 2). Below the diagrams, 4 premises were listed in the order ALL As are Bs, NO As are Bs, SOME As are Bs, and SOME As are NOT Bs. Adjacent to each premise were the numbers 1 to 5. Subjects were instructed: “Below this paragraph there are five circle diagrams labeled 1 to 5. They represent sets of objects (A’s and B’s). Below the circle diagrams there are four statements. Please circle the number(s) of the diagram(s) that the sentence is true of. If you think ‘All A’s are B’s’ is true of diagram 3, circle 3 alongside that sentence. You may circle more than one number per statement. Please interpret “some” to mean “at least one and possibly all”. 2.4 Procedure 2.4.1 Pre-test Subjects were first administered the pre-test of knowledge of Euler’s method of representing syllogistic premises. The pre-test typically took 5–6 minutes to complete. 2.4.2 SwitchERII trials Following the interpretation pre-test, subjects were introduced to the switchERII system via the practice example. The experimenter pointed out the various screen windows and regions – problem information, ER construction environment icons, question windows, the ER construction area, system message window, etc. Subjects in the FB group were told that messages would appear if the system ‘thought’ there was a problem with their representation. The sentence re-ordering facility was demonstrated to all subjects by the experimenter and each ER construction environment was demonstrated. Following a demonstration of the system features by the experimenter, the subject attempted the practice problem on their own. When the subject was ready, the experimental problem was run and the experimenter read out the following instructions “In this problem the questions are based upon a set of conditions. In answering, it may be useful to draw a diagram. For each question, select the best answer choice given.” Subjects were also instructed to begin their solution using set diagrams. They were further told, however, that if they preferred not to use set diagrams they could switch to an alternative ER environment at any time. When the subject indicated to the experimenter that they were ready to begin, a screenrecorder1 was started in order to log user-system interactions. Subjects were interviewed by the experimenter following the problem solving session. 3 Results 3.1 Pre-test data – Euler’s circle interpretation task Six subjects (3 FB, 3 NFB) demonstrated error-free performance on the Euler’s circle interpretation task. Correct responses are defined as diagrams 1 and 2 for the premise ‘All As are Bs’, diagram 5 only for ‘No As are Bs’, diagrams 1,2,3 and 4 for ‘Some As are Bs’ and diagrams 3,4 and 5 for ‘Some As are not Bs’ (see Figure 2). Seventy-five percent of subjects chose the correct diagrams for ‘All As are Bs’, 94% for ‘No As are Bs’, 50% for ‘Some’ and 62% for ‘Some not’. These results are similar to those previously reported [10,16] with replication of the usual response pattern across quantifiers (‘ALL’ easiest, ‘SOME’ most difficult). The majority of subjects (10 out of 16) made errors of omission and/or commission. At least three ‘syndromes’ could be identified in error patterns. These were errors of conversion, ‘Gricean’ interpretation errors and what might be termed ‘island’ responses. For each subject, 7 measures were derived 1
‘Cameraman’, Multimedia Utilities, Motion Works International, San Francisco, CA. This utility produces Apple ‘Quicktime’ format screenrecordings.
– conversion errors, Gricean errors on ‘Some As are Bs’, Gricean errors on ‘Some As are not Bs’, island responding on ‘No As are Bs’, island responding on ‘Some As are not Bs’, errors of omission not accounted for by conversion, Gricean or island errors and, finally, errors of commission not accounted for by conversion, Gricean or island errors. Conversion was deemed to have occurred when the subject chose diagram 1 alone as a model of ‘All As are Bs’. That is, the subject interprets ‘All As are Bs’ to be equivalent to the statement ‘All Bs are As’, i.e. they ‘convert’ the universal quantifier. Two subjects converted. Gricean errors are characterised by the adoption of a natural language interpretation of ‘some’ as excluding the possibility of ‘all’ [7]. Despite explicit instructions to adopt a logical and not a natural language interpretation of ‘some’ (recall that subjects were told in the instructions that ‘Some’ should be interpreted to mean ‘at least one and possibly all’), 6 subjects showed Gricean errors. Gricean errors on existential positive and existential negative premises were distinguished. The former (Gricean-some) errors are defined operationally as the selection of diagrams 3 and/or 4 only as being true of premise ‘Some As are Bs’. The latter (Gricean-somenot) errors are defined as the selection of diagrams 3 and/or 4 only as being true of the premise ‘Some As are not Bs’. Island responses – this term describes a phenomenon noticed in previous data [16]. It is defined operationally as a response in which the subject interprets the identity diagram (diagram 1 in Figure 2) and diagram 2 as valid models of ‘All As are Bs’ – but then subsequently interprets diagram 2 as being consistent with the premise ‘Some As are not Bs’ – that is, interpreting diagram 2 as representing an ‘island’ of As in a ‘sea’ of Bs. In other words, the phenomenon is one of inconsistent semantics in which the subject’s interpretation of spatial inclusion as metaphor for set membership holds for ‘All As are Bs’ but then the metaphor changes when ‘Some As are not Bs’ is interpreted – whereas for ‘All As are Bs’ the small circle A is taken to represent a set containing both As and Bs, in an ‘island’ response, the small ‘A’ circle is interpreted to represent an ‘island’ of As in a ‘sea’ of B’s. Hence, island interpretation errors differ from conversion and Gricean errors in that they are due to inconsistent application of the ‘spatial-containment-for-set-membership’ metaphor across quantifier conditions. Island-NO responses are defined as the selection of diagrams 2 and/or 3 for ‘No As are Bs’. Island-SN responses are defined as the selection of diagram 2 as being true of ‘Some As are not Bs’. One subject manifested an island interpretation of ‘Some As are not Bs’ and another subject did so in response to ‘No As are Bs’. For the former subject, island responding was associated with Gricean errors and for the latter subject island responding was associated with both Gricean and conversion errors. Errors of omission & commission consisted of erroneous responses that were not accounted for by those associated with conversion, Gricean or island responses. One subject made a commission error and 8 subjects made errors of omission. 3.2 SwitchERII results The switchERII screenrecordings were replayed and the experimenter recorded the frequency, timing and type of various events such as errors in ER construction, representation switching and responses to questions. A full analysis is reported in [4]. Subject responses to the 4 problem questions were scored – 1 subject scored 1 out of 4, 3 scored 2, 6 scored 3 and 6 scored 4. 3.2.1 Effect of switchERII system feedback upon performance The effects of switchERII’s explicit system feedback messages (e.g. ‘There may be a problem with the representation...’) were not striking. There was no significant difference between FB and NFB group subjects in terms of diagram construction errors (FB mean = 1.37, s.d.=1.19; NFB = 1.0, s.d.=1.07), problem score (FB group median = 3, NFB median = 3.5). Neither did the groups differ significantly in terms of time spent on the problem (FB mean = 20.5 minutes, s.d.= 4.47; NFB mean = 19.0 minutes, s.d. = 5.83). Several subjects reported that they did not attend to the message window or that they ‘didn’t notice’ the system messages because they were concentrating on the problem. Five of the 8 FB group subjects made reversal errors during construction, despite system feedback messages – probably because they were preoccupied with problem solving and had insufficient cognitive resources to also attend to system feedback messages. 3.2.2 Reversal errors in Euler’s circle ER construction The most common errors in graphical ER construction were reversals in which, for example, a subject might represent ‘All As are Bs’ by drawing a small circle ‘B’ inside a larger circle ‘A’. Seven subjects made reversal errors during ER construction resulting in invalid models (set diagrams) of the problem information. One FB group subject made a reversal error but subsequently corrected his diagram (possibly as a result of feedback
from switchERII). An example of a reversal error can be seen in Figure 1 in that subject’s representation of the relationship between Browning-likers (circle ‘B’) and Eliot-likers (circle ‘E’). Table 1: Relationship between errors on interpretation task, diagram construction errors & reasoning score. Group 1 2 3 Interp. errors [mean (s.d.)] 2.8 (1.5) 1.2 (0.5) Interp. errors [type (n of Ss)] gr(3) is(2) co(2) o(5) gr(3) o(3) Constr. errors [mean (s.d.)] 2.2 (1.1) 1.3 (.5) Constr. errors [type (n of Ss)] rev(4) wq(3) se(1) rev(3) wq(1) se(3) Reas’g score/4 [median] 2.0 3.5 3.5 Key to error type codes: Interpretation error types: gr – Gricean, is – island, co – conversion, o – other; Diagram construction error types: rev – reversal, wq – wrong quantifier represented (e.g. ‘some’ for ‘all’), se – sign error (e.g. ‘no’ for ‘all’). Note that in the diagram interpretation task a particular subject may make > 1 error of the same type (eg Gricean ‘some’ & Gricean ‘some not’). An individual subject may make > 1 error of the same type during diagram construction (eg 2 reversal errors in same diagram).
3.2.3 Relationship between diagram interpretation, diagram construction and reasoning The relationship between interpretation task performance, diagram construction errors, representation switching and reasoning score was analysed. Three patterns of performance were noted. Each subject, except one2, fell into one of three groups. The means and s.d.’s for interpretation task and diagram construction errors and the median scores (out of 4) for subjects in each of the three groups is shown in Table 1. The types of errors for both diagram tasks are also shown. Note that subjects from both FB and NFB conditions were evenly distributed over the three groups (Group 1 – 3FB, 2NFB; Group 2 – 2FB, 2NFB; Group 3 – 3FB, 3NFB). The first group consisted of subjects who made errors on both interpretation and diagram construction (n=5). The construction errors were associated with conversion and island responses on the pre-test and also with Gricean errors. Two subjects who showed conversion errors on the pre-test manifested reversal errors during ER construction. This was to be expected in these subjects since they regard ‘All As are Bs’ to be synonymous with ‘All Bs are As’. Both subjects who gave ‘island’ responses on the pre-test also made reversal errors. Gricean errors, though, were more or less equally divided between reversing subjects (3 Gricean responders) and non-reversing subjects (2 Gricean responders). Subjects in this group appeared not to comprehend the semantics of Euler’s circles. This group demonstrated the lowest median reasoning score. However, despite diagram errors the average was 2 questions correct out of 4. In many analytical reasoning problems it is possible to read-off correct answers to at least some of the problem questions from partially incorrect diagrams of the information. For this reason, errors in ER construction do not always result in poor reasoning performance. Of course, some types of ER construction error have more effect than others. It is also the case that an idiosyncratic diagram can be useful to the person that produced it if, when information is read-off from the representation, the idiosyncratic interpretation is applied consistently. Only one of the subjects in this group switched from the graphical (Euler’s circles) ER environment to the text environment in the course of problem solving – this suggests that these subjects were largely unaware of their representational errors. A second group consisted of subjects who made interpretation errors, but not diagram construction errors (n=4). Subjects in this group tended to make fewer, less severe errors on the interpretation task (mainly errors of omission and Gricean errors). These subjects scored well on the analytical reasoning problem questions. A ‘depth of processing’ [6] explanation probably accounts for the manner in which the performance of subjects in this group differed on the two tasks (i.e. deeper processing during self-construction of diagrams than when interpreting prefabricated representations). One subject in this group switched from graphics to text having (validly) represented 3 out of the 6 problem entities (poets) in his Euler’s circle diagram. Two subjects in this group used switchERII’s problem sentence re-ordering feature prior to constructing their representation. A third group of subjects (n=6) did not make errors on the interpretation task but did make diagram construction errors. The discrepancy between diagram production and diagram comprehension would, in this case, seem to indicate problems at the level of executing a diagram and not at the level of comprehending the representational semantics underlying the representation. A high proportion (4 out of 6) of the subjects in this group 2 S4 did not comply with the instruction to begin his solution by constructing a set diagram – he switched to the text ER environment immediately.
switched representations in the course of problem solving. Three subjects switched to text and then back to the diagram in the course of reasoning. A fourth subject switched to text on the third question. The ER switching behaviour of these subjects suggests that they were probably aware of the errors in their diagrams. Not only was the proportion of switchers higher in this group, but also the nature of the switching was qualitatively different from that of switchers in the other groups in that they did not abandon their diagrams, but used them selectively and in conjunction with textual representations. This behaviour, together with the high level of correct responding to the problem questions, indicates that these subjects recognised that their diagrams were (at least partially) incorrect and that they therefore used them judiciously and in conjunction with alternative representations in the linguistic modality. This finding might be regarded as a ‘compensatory’ rather than ‘complementary’ use of multiple representations. One subject in this group used switchERII’s problem-sentence re-ordering feature. 4 Discussion and conclusions The results indicate that not all errors of ER interpretation predict performance on tasks in which subjects construct and reason with ERs. This finding highlights the important differences between reasoning with ERs under those two conditions. However, the picture is somewhat complicated in that although some interpretation errors are associated with errors in ER construction, erroneously constructed ERs do not inevitably cause subjects to make incorrect responses to the problem questions. Awareness of representational error(s) on the part of the subject is crucial. Most of the subjects in the third group tended to remediate the deficiencies in their Euler’s circle diagrams via the use of multiple representations (inter-representational switching during reasoning). The situation for these subjects is analogous to that of skilled readers who find writing difficult, though it must be remembered that few people receive direct instruction in the production of diagrammatic representations to anything like the extent that they receive instruction in writing. On balance, however, the majority of subjects appeared to use graphical ERs reasonably effectively. Diagrams are useful aids to reasoning due to their cognitive and semantic properties (specificity, computational efficiency, etc) which have been reviewed elsewhere [4,5,17]. In addition to the semantic properties and computational efficiency of external representations, the process of externalisation can also aid reasoning. Externalisation can facilitate a shift of reasoning mode. Diagrammatics models, used in the development of a geometry proof, and compared to syntactic representations, can act to constrain the set of possibly provable statements and shift the mode of reasoning from deduction to induction [9]. Selecting and constructing one’s own representation differs in several crucial ways from using a presented, prefabricated representation. The motor activity associated with diagram execution is one important difference. Spatial and motoric representations are closely linked in visual imagery [12]. Externalisation is a process by which a mental image is converted into a percept. This conversion process can help to disambiguate (often) hazy mental images and can also assist via the offloading of working memory demands onto the ER [12]. The effectiveness of ERs may also be mediated by mechanisms that parallel the ways in which selfexplanations are associated with successful problem solving – the ‘self-explanation’ effect [3]. Attempting to construct a model of the problem information can help the problem solver decide whether a single model is adequately expressive. It seems likely that graphical ERs, by their limited ability to express abstraction [17] may provide more salient and vivid feedback to a comprehension-monitoring, self-explaining student than ‘self-talk’ in the linguistic modality. However, in the context of ER construction, self-monitoring requires that the reasoner recognise his/her diagrammatic errors – something that many subjects did not seem to do in the present study. What are the implication of these results for the design of intelligent educational systems? How might such systems facilitate externalisation? The cognitive load of analytical reasoning with ERs is very high and system feedback needs to minimise attentional demands. In the case of switchERII, feedback was provided in a ‘mismatched’ modality (i.e. textually when user was working graphically). Guidance could be provided in the same modality as the user is working in, i.e. via the diagram itself, perhaps by highlighting incorrect diagram elements. A study of this issue is planned. The results also suggest that the diagrammatic reasoning literature, much of which consists of diagram interpretation studies, should be interpreted cautiously by designers of interactive graphical systems. The challenge is to build intelligent educational systems that can analyse diagram construction behaviour in relation to diagram use in reasoning and which can assist the subject who has difficulty with diagram construction. For this, systems must incorporate a representation of the semantics of the formalism and use it to parse the users’ input and provide feedback – preferably dynamically during diagram execution. Ideally, the system should also check the consistency of the user’s conclusions against his/her representation. A longer term goal would be build systems capable of providing intelligent assistance at the level of helping the reasoner assign appropriate representations to particular tasks taking into account user characteristics such as expertise and cognitive style and the semantic and cognitive properties of representational systems.
Acknowledgements Thanks to Dr. Paul Brna for his comments on drafts of this paper. Thanks also to my colleagues in the Graphics and Language Working Group of the Human Communication Research Centre. 5 References 1. Barwise, J. & Etchemendy, J. (1994) Hyperproof for the Macintosh. CSLI Lecture Notes Number 42, Centre for the Study of Language & Information, Leland Stanford Junior University: CSLI Publications. 2. Bonar, J. & Cunningham, R. (1988) BRIDGE: Tutoring the programming process. In J. Psotka, L. Massey, & S. Mutter (Eds.), Intelligent tutoring systems: Lessons learned. Hillsdale, NJ: Lawrence Erlbaum Associates, 409–434. 3. Chi, M.T.H., Bassok, M., Lewis, M.W., Reimann, P. & Glaser, R. (1989) Self explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145–182. 4. Cox, R. (1996) Analytical reasoning with multiple external representations. PhD thesis, Department of Artificial Intelligence, University of Edinburgh. 5. Cox, R. & Brna, P. (1995) Supporting the use of external representations in problem solving: The need for flexible learning environments. Journal of Artificial Intelligence in Education, 6(2), 239–302. 6. Craik, F. & Lockhart, R. (1972) Levels of processing: A framework for memory research. Journal of Verbal Learning & Verbal Behavior, 11, 671–684. 7. Grice, H.P. (1975) Logic and conversation. In P. Cole & J.L. Morgan (Eds.), Syntax and semantics: Volume 3 – Speech Acts, New York:Academic Press, 41–58. 8. Grossen, G. & Carnine, D. (1990) Diagramming a logic strategy: Effects on difficult problem types and transfer. Learning Disability Quarterly, 13, 168–182. 9. Koedinger, K.R. & Anderson, J.R. (1990) Abstract planning and perceptual chunks: Elements of expertise in geometry. Cognitive Science, 14, 511–550. 10. Newstead, S.E. (1989) Interpretational errors in syllogistic reasoning. Journal of Memory & Language, 28, 78–91. 11. Newstead, S.E. (1995) Gricean implicatures and syllogistic reasoning. Journal of Memory and Language, 34, 644–664. 12. Reisberg, D. & Logie, R. (1993) The ins and outs of working memory: Overcoming the limits on learning from imagery. In B. Roskos-Ewoldson, M.J. Intons-Peterson & R.E. Anderson (Eds.), Imagery, creativity and discovery: A cognitive perspective, Amsterdam: North Holland, Elsevier, 39 – 76. 13. Schooler, L.J. & Anderson, J.R. (1990) The disruptive potential of immediate feedback. Proceedings of the 12th Annual Conference of the Cognitive Science Society, Hillsdale, NJ: Lawrence Erlbaum Associates, 702–708. 14. Shute, V. & Gawlick-Grendell, L. (1993) An experiential approach to teaching and learning probability: StatLady. In P. Brna, S Ohlsson & H. Pain (Eds.), Proceedings of the World Conference on Artificial Intelligence in Education (AIED’93), AACE, 177–184. 15. Singley, M.K., Anderson, J.R., Gevins, J.S. & Hoffman, D. (1989) The algebra word problem tutor. In D. Bierman, J. Breuker & J. Sandberg (Eds) Artificial intelligence and education. Proceedings of the 4th International Conference on AI and Education, May, Amsterdam, IOS, 267–275. 16. Stenning, K. & Cox, R. (1995) Attitudes to logical independence: traits in quantifier interpretation. In J. D. Moore & J. Fain Lehman (Eds.), Proceedings of the 17th Annual Conference of the Cognitive Science Society, Lawrence Erlbaum & Associates, 742–747. 17. Stenning, K. & Oberlander, J. (1995) A cognitive theory of graphical and linguistic reasoning: Logic and implementation. Cognitive Science, 19(1), 97–140. 18. Stenning, K., Yule, P. & Cox, R. (1996) Quantifier interpretation and syllogistic reasoning: An individual differences account. Proceedings of the 18th Annual Conference of the Cognitive Science Society, Hillsdale, NJ: Lawrence Erlbaum Associates, 678–683. 19. Wertsch, J. (1991) Voices of the mind: A sociocultural approach to mediated action, Cambridge, MA: Harvard University Press. 20. Wertsch, J. & Toma, C. (1995) Discourse and learning in the classroom: A sociocultural approach. In L.P. Steffe & J. Gale (Eds.) Constructivism in education, Hillsdale, NJ: Lawrence Erlbaum Associates, 159–174.
