Design of argument diagramming for case-based group learning John Dowell & Thomas Gladisch Department of Computer Science University College London Gower St., London. UK
[email protected];
[email protected];
ABSTRACT
Argument diagramming tools can improve reasoning and learning. They are likely to have a significant place in future virtual learning environments whose design will be dominated by the concepts of self-directed enquiry and shared constructive learning. In such environments, where communication is typically computer-mediated and predominantly textual, communication can be integrated with argument diagramming. Put simply, people will be able to draw their arguments as they discuss. We report on an exploratory investigation of this design concept. We built a virtual learning environment supporting remote collaborative learning with computer mediated communication by chat text and argument diagramming. We examined the effect of using an explicit argumentation scheme to integrate the discussion with the drawing of the argument. We report the results of an evaluation study conducted with this system. We found that learners were able to successfully use the argument diagramming system to represent their discussion. Although use of the argumentation scheme was limited, the study nevertheless demonstrates the viability of using explicit argumentation schemes and suggests the potential for embedding schemes more directly into communication tools in virtual learning environments. Keywords
Argument diagramming, collaboration, case-based learning, computer-mediated communication INTRODUCTION Future learning environments
Self-directed inquiry and the shared construction of explanations will be central concepts in the design of future virtual learning environments (VLEs). Within such an environment, learners will search for, select and interpret evidence and related information and they will discuss and argue about the explanations they construct. Much communication will be computer mediated and much of it by visual text. This design vision represents a distinct departure from the current day learning environment which characteristically relies on learners
completing prescribed tasks in order to acquire relatively circumscribed knowledge (Pea, 2004). The future of VLEs is perhaps signalled by Knowledge Forum (Scardamalia, 2003), a learning environment which is claimed to enable self-directed learning and group explanation construction and is claimed to achieve high levels of learner engagement with improved learning outcomes. Argumentation skill and argument adequacy
Future VLEs will need to help users to produce better explanations through arguing more effectively with evidence. People are naturally skilful at the discursive process of argument, appropriately deploying ‘moves’ (challenging, conceding, etc) and elaborating their argumentation to take account of the local and global structure of the discourse (turn taking, reference to previous contributions etc) (Kuhn, Shaw & Felton, 1997; Resnick et al 1993). Learners maintain their ability for arguing even when communication is computer mediated and textual. Nevertheless, to counteract the negative effects of computer mediated communication on peoples’ dialogue management, various tools have been developed (e.g., Academic Talk) to help learners better structure their argumentation with menus of pre-formed sentence openers, and threading to make visible the local context of contributions. These communication tools are designed to overcome the tendencies for dialogue to become incoherent in synchronous chat systems, and for it to lose focus in asynchronous forum systems (Herring, 1999). Whilst people are typically adept at argumentation as a dialogical process, the arguments they construct over evidence are often inadequate when judged by objective criteria (Schwarz & Glassner, 2003). Arguments often rely on non-justified beliefs rather than articulated reasons or theory, they often consist of detached reasons and may give no recognition of alternatives or rebuttal of counterarguments (Kuhn, 1991). People frequently accept the plausibility of explanations without paying sufficient attention to their consistency with the available evidence (Brem & Rips, 2000). These observations about the weakness of informal reasoning
over evidence apply to both everyday and expert domains. For example clinicians have been found (Christiensen and Elstein 1991) to exhibit significant bias in their reasoning, mirroring the kinds of bias often found in decision-making. They will, for example, appropriate assertions or facts which confirm their own argument and ignore those that contradict. Argument diagramming tools
One promising way of improving collaborative evidence-based reasoning is to provide people with argument diagramming and knowledge mapping tools. Argument diagramming tools typically provide simple and generic notations for representing argument content, (for example the pioneering IBIS system used primitives of themes, issues, positions and arguments to record and facilitate complex decisions). The value of argument diagramming and knowledge mapping tools for learning has been repeatedly demonstrated since students first using the CSILE system were reported to “greatly surpass students in ordinary classrooms on measures of depth of learning and reflection, awareness of what they have learned or need to learn, and understanding of learning itself“ (Scardamalia and Bereiter, 1994). As with Knowledge Forum, the Belvedere system (Suthers and Jones 1997) has achieved some uptake in US schools. Belvedere allows groups of learners to create knowledge maps (evidence maps, concepts maps, causal diagrams etc). It achieves a high degree of accessibility through its epistemological and notational simplicity: it is only capable of representing ‘for’ and ‘against’ relationships between evidence and theories and therefore cannot capture the subtle nuances of arguments (Andriessen, Baker and Suthers, 2003). Belvedere is designed to be used collaboratively and provides a simple chat (asynchrous messaging) tool to allow remote groupwork. This tool is entirely uncoupled from the argument diagramming tool. Similarly, Baker et al (2004) report investigations with a prototype VLE in which learners collaboratively construct argument diagrams while communicating via rudimentary chat text. The learners Baker observed found the argument diagramming tools unwieldy to use to such a degree that a comparison group of learners working with the chat alone produced more successful arguments and achieved better learning outcomes. This is a surprising finding given the evidence elsewhere of the benefits of argument diagramming tools for learning outcomes (Kischner et al, 2003) and points to the simple lack of integration of environments for collaborative learning involving the constructive representation of reasoning and knowledge. Design challenge: linking dialogue to argument representation
So a significant challenge for the next generation of knowledge building environment is to develop transactional mapping tools which link dialogue tools directly to representation tools. For example, in such an integrated environment learners would create an argument diagram consisting of the component
arguments and linked knowledge. This environment could make use of the fact that arguments exhibit recurrent argument schemas or patterns of inference (Walton, 2005). For example, ‘slippery slope’ arguments are a stereotypical form of argument in which conceding one proposition leads inevitably to conceding others. The Araucaria argument diagramming tool (Reed and Rowe, 2004) exploits definitions of argument scheme to enable argument graphs to be drawn in text analysis. It is possible then that argumentation schemes could be used as an organising device to support the integration of discussion with argument diagramming in a VLE. We now describe a prototype VLE which was developed to explore this design possibility. DEVELOPMENT OF THE LEARNING ENVIRONMENT
From a technical point of view, the VLE consists of four main tools with communication handled by an XML server [@Sock] set up on a (remote) computer (Gladisch, 2007). These tools are: the chat system; the case materials library; the argumentation scheme display; and the diagramming tool. The four tools in themselves are not novel, but the combination of them in one environment provides a novel working and learning experience for users. Users use the chat tool to discuss the case materials they can access in the case library. They build an argument diagram to focus their reasoning and discussion. Users are encouraged to build the diagram within the framework provided by the argumentation scheme. The diagramming tool is the focus of the system to which activity with the other tools is directed. The diagram can only be modified by one user at a time and the so-called “annotator” can synchronise the changes he made on his whiteboard with the whiteboards of the other users. In this way, all other users can see who the annotator is and what he or she is changing on the whiteboard. To visualise their findings, the users can extract selected contents of the chat, as well as contents of the learning material, to place direct quotations of statements or facts within the diagram. By using a chat as the medium of communication, the users achieve more time to construct their arguments before responding to other participants. In this way, the discussion becomes more structured and efficient (Weinberger, 2006). The users’ dialogue is guided by the set of critical questions representing the argumentation scheme. Users can modify and ask these pre-defined questions, in order to focus the chat discussion on specific concerns. The argumentation scheme we have exploited is Walton’s abductive inference argumentation scheme (Walton, 2005) expressed as a series of critical questions which the learners are requested to respond to during the task. The scheme itself provides essentially two kinds of premise that can lead to a conclusion. The scheme guides the user in constructing a corresponding form of diagram. To make the scheme more usable for the diagram creation, there is also a “scheme checker”, which analyses the diagram for correspondence with the
argumentation scheme. The argument diagramming tool uses a notation similar to Belvedere consisting of two node types. Data nodes contain facts and given information of the learning material or external sources, whereas hypothesis nodes hold assumptions or conclusions that have been made by the users. The nodes can be linked to each other with different types of connections. These can either support or refute arguments. Furthermore, these connections can be evaluated by the users to show specific criteria of the
connection by placing text next to it. This kind of evaluation is similar to the evaluation modifiers of Araucaria, but allows for a more flexible arrangement of arguments in the diagramming area, since all nodes can be arranged freely and a single node can support and refute multiple nodes at once. A screen shot of the VLE is shown below, indicating the contents of the case library, the chat messaging system, and the argument diagramming tool showing a partially completed diagram. The VLE is available from the authors.
View of the VLE interface taken midway through an evaluation session with one of the groups of three learners. Visible in this view is the case library, chat window containing the discussion between the users, and the argument diagram which here is partially complete.
EVALUATION:
An evaluation of the VLE was conducted to assess the integration of dialogue and diagrams, the quality of argumentation and the effectiveness of the argumentation scheme tool, and learning outcomes. The effect of the argumentation scheme on the dialogue was of particular interest given its intended role of coordinating communication and diagramming. The evaluation then would involve groups of students
experiencing the VLE with realistic learning materials to achieve specific learning objectives. The case materials used were based on the case materials used by undergraduate medical students at UCL taking a course in which they learned about Medical Law. The case describes the details of an alleged medical negligence which became the subject of a court hearing. For our evaluation, the students were required to explain how the judge arrived at his verdict. The case was an
adaptation of a real case and although the verdict was far from ‘open or shut’, no specialist knowledge either of medicine or law was needed to find at least a simple explanation for the judge’s verdict. Five groups of three students were recruited for the evaluation, drawn from different faculties at UCL. Data collected in the evaluation was, first, the dialogue between the users as recorded in the chat tool log; second, data consisted of sequences of stills of the developing argument diagrams for each group; third, questionnaire responses collected from each user and probing their views of using the VLE and of the learning experience. The data and analysis will be presented in the full paper. CONCLUSIONS
The paper concludes with claims which can be drawn about the design of learning environments which integrate communication and argument diagramming. Although there have been previous attempts to develop learning environments in which shared argument diagramming is available with computer mediated communication (see Introduction), reports on their trial use suggest that users found them unworkable to the detriment of the learning outcomes. Our evaluation has examined the integration of these functions using argumentation schema. Our evaluation has shown the workability of this solution, with users finding little difficulty in concurrently diagramming their argumentation whilst maintaining a flowing dialogue. In spite of this finding, the use of the argumentation scheme had mixed success. This appears to be more a consequence of the way in which the argumentation scheme was implemented in our VLE, rather an intrinsic issue with the schema. We consider alternative ways in which the schema could be realised. We set this in the context of the development and uptake of VLEs which is gathering pace. REFERENCES
AcademicTalk: http://learning.north.londonmet.ac.uk/ltri/academictalk/ Andriessen J., Baker M and Suthers D., (2003), Argumentation, computer support and the educational context of confronting cognitions. In Andriessen J., Baker M and Suthers D. (ed.s) Arguing to learn. Kluwer Academic: Netherlands. Baker, M.J., Quignard, M., Lund, K. and Séjourné, A. (2003). Computer-supported collaborative learning in the space of debate. In B. Wasson, S. Ludvigsen & U. Hoppe (Eds.) Designing for Change in Networked Learning Environments : Proceedings of the International Conference on Computer Support for Collaborative Learning 2003, pp. 11-20. Dordrecht: Kluwer Academic Publishers Brem, S. K. & Rips, L. J. (2000). Explanation and evidence in informal argument. Cognitive Science,
24(4), 573-604. Christiensen C. and Elstein S., (1991). Informal reasoning in the medical profession. In Voss J., Perkins D. and Segal J (ed.s) Informal reasoning and education. Hillsdale NJ: Erlbaum. Gladisch, T. (2007) The COALA collaborative argumentation and learning environment. UCL Department of Computer Science (internal report). Herring S. (1999). Interactional coherence in CMC. Journal of Computer-Mediated Communication, 4(14). Kirschner P., Buckingham Shum S. and Carr C. (Eds.), (2003) Visualizing Argumentation: Software Tools for Collaborative and Educational Sense-Making. SpringerVerlag: London. Kuhn, D. (1991). The skills of argument. Cambridge: Cambridge University Press. Kuhn, D., Shaw, V., & Felton, M. (1997). Effects of dyadic interaction on argumentive reasoning. Cognition and Instruction 15(3), 287-315. Pea, R. (2004). The social and technological dimensions of scaffolding and related theoretical concepts for learning, education, and human activity. The Journal of the Learning Sciences, 13(3), 423-451 Reed C. and Rowe G., (2004): Araucaria: Software for Argument Analysis, Diagramming and Representation. International Journal on Artificial Intelligence Tools 13(4): 983. Resnick, L. B., Salmon, M., Zeitz, C. M., Wathen, S. H. & Holowchak, M. (1993). Reasoning in conversation. Cognition and Instruction, 11(3&4), 347-364. (@Sock) SourceForge.net: FlashSock: http://sourceforge.net/projects/flashsock/ Scardamalia M. and Bereiter C. (1994), Computer support for knowledge-building communities. The Journal of the Learning Sciences 3(3), 265-283. Scardamalia, M. (2003). Knowledge Forum (Advances beyond CSILE). Journal of Distance Education, 17, 3, 23-28 Schwarz B. and Glassner A., (2003) The blind and the paralytic: supporting argumentation in everyday and scientific issues. In Andriessen J., Baker M and Suthers D. (ed.s) Arguing to learn. Kluwer Academic: Netherlands. Suthers D., and Jones D., (1997) An architecture for Intelligent Collaborative Educational Systems, AI-Ed 97, World Conference on Artificial Intelligence and Education, Japan, pp. 55-62. Walton D., 2005, Abductive Reasoning. Alabama University Press. Weinberger, A., Clark, D., Erkens, G., Sampson, V., Stegmann, K., Janssen, J., Jaspers, J., Kanselaar, G. and Fischer, F.: Argumentative Knowledge Construction in CSCL. ICLS ’06, June 27 – July 1, pp. 1094-1100, Bloomington, USA, 2000.
