User-centred Design & Development of an Applied Web-based ITS Manolis Mavrikis, Antony Maciocia School of Mathematics, The University of Edinburgh, EH9-3JZ,Edinburgh, UK {M.Mavrikis,A.Maciocia}@ed.ac.uk
Abstract This paper describes how a user-centred engineering cycle is employed during the development of a web-based instructional system for first year science and engineering students learning mathematics. After an overview of the relevant methodology and appropriate theory that influences the system’s design, we briefly present the system together with the steps that led to its integration in the teaching environment as well as to reusable materials and components for other environments.
1. Inroduction E-learning is already changing our educational systems and fortunately researchers and technologists have already realised that projects should be driven by ‘learning pull’ rather than ’technology push’. Obviously, universities are among the stakeholders of this change and similarly, the School of Mathematics of the University of Edinburgh, is investigating the support of its students through a webbased instructional system, WALLIS [7]. Such an approach seems necessary mainly because the School delivers mathematics courses to the whole range of science and engineering students who come from diverse backgrounds but who also often lack interest and motivation since they are studying for different degrees. In addition, science departments cannot afford to spend a lot of lecture time in reinforcing basic mathematical skills that should have been covered at school. These are common problems and one of the steps that universities take to alleviate them is to apply e-learning solutions. On the other hand, it is often easy to overlook the close relation of pedagogy and technology and the fact that educational technology, in order to be effective, needs to be closely integrated with curriculum goals, related texts and teacher practices [5]. It was along these lines that, during the design and implementation of WALLIS, a usercentred methodology, and appropriate theory were adopted to allow the a more learner-centred application.
Duncan Abela, John Lee School of Informatics. The University of Edinburgh, EH8-9LW, Edinburgh, UK
[email protected] ,
[email protected]
2. Background From a software engineering perspective many researchers [eg. 3, 4, 5, 9] observe the importance of careful user study. Taking it even further, Clancey argues that “computer systems as artifacts” must be developed “incrementally, with relatively quick periods of use, observation, reflection, and redesign...in a context that includes the user’s everyday adaptations” [2] implying that students and teachers must participate during the development process. Along the same lines, Persistent Collaboration Methodology (PCM) [3] provides a suitable framework for developing and applying WALLIS as it is inspired by action research, which (despite the danger of producing results that are difficult to generalise) is the only transformational research method in education that looks at changes of the environment. Therefore, it is appropriate when a new approach is to be applied. As its name suggests, one of PCM features is collaboration between students, teachers, researchers and technologists during iterations of ‘observation, reflection, designing’. The other feature is persistence, which refers to a “long lasting chance, in classroom materials but also in the evolving methods and beliefs of the collaborating partners” [3]. Finally, PCM is stimulated by a ‘wheel of techniques and tools’ that influence the development of the system, but also theories of teaching and learning that influence the system’s pedagogy. The above process is exactly what we followed during the development of WALLIS. Although, formal collaborations were not always feasible, all of the phases were realised taking significant feedback from students and lecturers, and consultation from other experts. More importantly, this process was strongly influenced by learning theories and relevant research. For instance, recognising that learners must remain involved, active, and challenged to learn, we decided to employ interactive activities to discourage passive learning. In addition, issues like intelligent feedback and help-seeking seem important and previous research has shown that are very
Proceedings of the The 3rd IEEE International Conference on Advanced Learning Technologies (ICALT’03) 0-7695-1967-9/03 $17.00 © 2003 IEEE
effective. One of the theories that addresses these issues and influenced WALLIS’ intelligent component (DANTE [6]) is scaffolding and contingent instruction [10]; notions proposed to describe the need for a balance between the children’s capacity to selectively ask for help and the teacher’s effort to take appropriate actions. In order to achieve that, the tutor (or a system) should establish and maintain an orientation towards task relevant goals, highlight critical features, demonstrate how to achieve them and help control frustration ensuring that the student is neither left to struggle alone nor given too little scope for involvement and initiative in the task [10].
3. From theory to practice During the initial observation period, expert consultation helped looking into theoretical aspects, but also investigating the available technologies, and other technical issues. Moreover, the main developer was directly involved in the whole range of the teaching and learning activities. Lecture observation and discussions with lecturers provided information about the system’s materials, its knowledge base, the use they would make of it, what problems they considered as most important, what notation they use, and their didactical approach. In addition, the delivery of tutorials allowed exploration of common concepts that students find problematic and what kind of help would be useful for them. Afterwards, we designed and implemented a prototype [for details see 7] and added appropriate materials. Following a constructivist view, the materials comprise interactive parts which allow students to freely explore aspects that cannot be covered in static pages. Moreover, apart from the obvious toolbars and inline links, we provide students with a tree-like map in a popup navigation tool and a feedback frame at the bottom of the screen. Although we tried to eliminate problems that we anticipated, further investigations proved very useful. Live observations, questionnaires, and students’ interviews helped us locate and improve several problematic aspects that would otherwise have been neglected. For instance, students complained (and the observations confirmed) that they often found the mathematical linear format quite annoying. This was alleviated by using WebEQ’s (www.dessci.com) applets and MathML for inputting answers and the DOM2 interface for providing feedback. In addition, problems with the navigation tool were overcame by developing an adaptive one according to their preferences (location, size, current concept etc). The same methodology was used for several other components. For instance, although research on an animated agent [1], which was using WALLIS as a framework, was proven useful for other contexts, our questionnaire results and relevant research did not provide appropriate evidence of an advantage and therefore it was
not integrated in the on-line version of the system. This shows how a user-centred methodology can help take not only cost-effective decisions but also develop software, not just because of the ‘technology hype’ but because it is useful, efficient and helps students learn.
4. Current & Future work While still evaluating findings relating to WALLIS’ effectiveness in a course running in first semester of 2002/3, we are in a constant observation phase as the system continues to be applied and used. In order to be able to share (at least) our content we are experimenting with the OMDoc (www.mathweb.org/omdoc) standard and its use in ActiveMath [8], as well as other appropriate standards. In addition, in order to make our components available to others, significant effort is directed towards implementing appropriate specifications such as the IMS (www.imsproject.org/specifications.cfm) Learner Information Package for student’s representation and the IMS Content Packaging for the navigation tool. Finally, now that the system grows, there is a demand for an authoring tool [similar to 9] and extensions for more formal assessment. We plan to continue with this methodology that marries research findings with user needs and develop, or better integrate and extend useful and re-usable components when available. This process is very important, not only from a technical point of view, but also because it can facilitate fast development of other prototypes for more realistic and fruitful research.
References [1] D. Abela. Improving a web-based ITS with ANA; an animated agent. Master’s thesis, The University of Edinburgh, School of Informatics; Artificial Intelligence, 2002. [2] W. Clancey. Guidon-manage revisited: A socio-technical systems approach. IJAIED, 4(1):5–34, 1993. [3] T. Conlon and H. Pain. Persistent collaboration: a methodology for applied AIED. IJAIED, 7:219–252, 1996. [4] B. du Boulay and R Luckin. Modelling human teaching tactics and strategies. IJAIED, 12:235–256, 2001. [5] K. R. Koedinger. Cognitive tutors as modeling tool and instructional model; in K. D. Forbus, P. J. Feltovich, and E. Canas (eds), Smart machines in education. MIT Press, 2001. [6] M. Mavrikis and J. Lee. Towards Contingent & Affective Microworlds. To appear in the proceedings of the 11th International Conference on AIED, 2003. [7] M. Mavrikis and A. Maciocia. Developing WALLIS; a webbased system to enhance mathematics teaching. ICTM 2002. [8] E. Melis, E. Andres, A. Franke, A. Frischauf, G. Goguadse, P. Libbrecht, M. Pollet, and C. Ullrich. ActiveMath: A web-based learning environment. IJAIED, 12, 2001. [9] M. Moundridou and M. Virvou. Analysis and design of a web-based authoring tool generating intelligent tutoring systems. Computer & Education, 40:157–181, 2003. [10] D. Wood and H. Wood. Contingency in tutoring and learning. Learning and Instruction, 6(4):391–397, 1996.
Proceedings of the The 3rd IEEE International Conference on Advanced Learning Technologies (ICALT’03) 0-7695-1967-9/03 $17.00 © 2003 IEEE
