WALLIS: a Web-based ILE for Science and Engineering Students Studying Mathematics Manolis MAVRIKIS1 and Antony MACIOCIA School of Mathematics, The University of Edinburgh Abstract. This paper, after briefly describing our context and the problems faced by science and engineering students studying mathematics, presents WALLIS, a webbased system that is being developed to address some of these problems. Keeping in mind pedagogy and the development of a learner-centred application, we first describe the iterative methodology which we employed, and theories (such as constructivism and contingent instruction) that influence the system’s design and the feedback it provides. Finally we briefly present some results that the system’s integration provides despite its early stages, and could be particular useful for similar projects.
1. Introduction The last few years researchers and university teachers are more concerned than ever with the evident problem of the growing deficiency in mathematical skills amongst science and engineering students. Although most of the problems have their origins at school, universities have to cope with several problems such as their diverse backgrounds and levels, and they often fail to recognise the importance of mathematics for their main degree. All these problems make additional support (tutorials, formative assessment etc.) difficult and, in conjunction with the increased intake of students, time consuming. In order to improve this situation our School is investigating student support by applying an Interactive (and “intelligent”) Learning Environment (ILE); named WALLIS after the 17th century mathematician. On the other hand, developers often fail to notice the close relation of pedagogy and technology and, as [1] suggests, that technology needs to be closely integrated with curriculum goals, related texts and teacher practices. It was along these lines that, during the design, implementation and integration of WALLIS, a specific user-centred methodology was adopted. The following sections, before presenting the system itself, describe the methodology and theories that influenced its design so far. 2. Background 2.1 User-centred software engineering It has long been argued that educational software can be effective only when based on continuous refinement of system behaviour [2] and on careful user study [1, 3]. Along this line of thought, there are several socio-technical [e.g. 4] or user-centred approaches, which employ iteration phases [e.g. 5] of software development instead of a more classical software engineering model. Taking it even further, Clancey argues that “computer systems as artifacts” must be developed “incrementally, with relatively quick periods of use, observation, reflection, and redesign...in a context that includes the user’s everyday 1
Corresponding author:
[email protected], JCMB, Kings Buildings, EH9-3JZ, Scotland, UK.
adaptations” [6] implying that both students and teachers (or lecturers in our case) must participate from the very beginning of the development process. One of the methodologies that takes all these into account is Persistent Collaboration Methodology (PCM [7]). In particular, PCM provides a suitable framework for developing and applying WALLIS as it is inspired by action research, which (despite the danger of producing results that are difficult to generalise) is the only transformational research method in education that looks at changes on the environment, addresses practical questions, and is appropriate when a new approach is to be integrated in an existing context. As the methodology’s name suggests, it requires collaboration between students, teachers, researchers and technologists through all the phases of the ‘observation, reflection, designing, action’ cycle. Although, in our case, formal collaborations were not always possible, the phases were all realised with the cooperation of lecturers, experts in HCI or evaluation fields, and students. The other feature of PCM is persistence, which refers to a “long lasting chance, not just in classroom materials but also in the evolving methods and beliefs of the collaborating partners” [7]. The above process is stimulated by ‘a wheel’ of tools (or techniques) which influence the design and development of a system, and theories of teaching and learning which influence designers’ decisions with pedagogical aspects, which could otherwise be neglected. On the other hand, this stimulation can often be both ways. This is what happened during the development of WALLIS as research in the AI&ED field strongly influenced its design but even the decision to experiment with the application of such theories to computer-based applications came from previous observations [for example, 8] with similar technologies that sometimes lacked the appropriate feedback. 2.2 Theories influencing WALLIS Following a more constructivist view and recognising that learners must remain “involved, active, and challenged to think about and learn the presented material” [2], we took the view that interactive activities would help bypass drawbacks of passive learning. In addition, issues like intelligent feedback and help-seeking seem important and research has shown that, when coupled carefully, are very effective. Recent results indicate the importance of emotions and motivation in human learning through educational software (see 9 for appropriate references and more details on this). Affective tutoring, in the sense of targeting student’s emotional and motivational state, is an effective approach and needs to be further explored. One of the theories that addresses such issues to some extent and influenced the development of WALLIS is contingent instruction [10,11] and consequently scaffolding; notions proposed to describe exactly the need for a balance between the children’s capacity to selectively ask for help and the teacher’s effort to make her actions contingent upon activities of the individual learner. This is achieved by recruiting student’s interest on the task, establishing and maintaining an orientation towards task relevant goals, highlighting critical features that a student might overlook, demonstrating how to achieve goals and helping control frustration ensuring that the student is neither left to struggle alone nor given too little scope for involvement and initiative in the task [see 10]. An important issue that arises when one tries to teach in such a way, is to know when and how much help a student needs to complete a task. A possible solution comes from the learners’ use of help seeking to influence the tutoring process. It is exactly then that an adult (or a system) must intervene to promote further progress and this is where the students can play a role influencing the tutor’s behaviour by their own help-seeking. Of course there is a huge array of affective factors involved during this process and the AI&ED field gradually recognises that and provides results. These can augment theories such as the above and provide more insight for a system’s design. In the following sections it will be made clearer how this theory influenced the design of the feedback mechanism so far.
3. The development of WALLIS and its application 3.1 Initial Observations During the initial observation period of the system’s design, collaboration and expert consultation helped elaborating on theoretical aspects, investigating the technologies that the students were already using, and looking into other technical issues. In addition, the main developer was directly involved in the whole range of the teaching and learning activities; delivering tutorials and observing lectures. This allowed exploration of common concepts that students find problematic, the kind of help that would be particularly useful for them, the problems that lecturers consider important to address as well as the notation they use, and their didactical approach. A final step of this phase was discussions about the systems’ contents, its knowledge base, lecturers’ attitude towards the system, and the use they would make of it. 3.2 Designing and developing WALLIS After the initial observation period we developed a prototype and added appropriate content. As mentioned before, we decided that the materials would comprise, where applicable, interactive exploratory parts which would allow students to freely explore some aspects that cannot be covered in static pages. Based on Wood’s view on contingent instruction, we wanted students to be able to ask and receive help (see section 4 for details on the feedback mechanism). Previous observations showed that pop-up windows were considered quite annoying [8,12] and in the particular case of WALLIS, where student actions during the interactive parts would produce feedback, this approach was not even feasible. Therefore we employed a feedback area at the bottom of the window where help and suggestions are provided (figure 1).
Figure 1. The overall environment of WALLIS
Since WALLIS would be used in a specific context, the pages only need to adapt, at least at this phase, to student preferences (colours, sizes etc) and very broad aspects of their profile (see figure 2) such as the course they attend rather than different didactical styles or learning paths like other adaptive systems (such as ActiveMath [13]) do. Moreover from related literature [eg.14] it was evident that students are easily confused and lost by many hyperlinks. This is a common problem in web-based environments which we currently address by having the system suggesting further pages to study (see next section) and by using a popup window with a tree-like map of the contents (see figure 1).
Figure 2. The framework of WALLIS and how its components communicate.
3.4 Adapting the environment to student needs Through the iterative phases that we mentioned above, we were able to adapt the feedback mechanism, set the misconceptions that it targets, and improve certain aspects of the system based on students’ interactions and comments. Thanks to observers’ help we were able to conduct live observation of students’ interactions. The students were allowed to use the system as they wished and in their own time. This produced more realistic actions which were correlated with the logged actions and yielded valuable results. For instance, from the prototype’s application, we realised that students faced serious problems typing their answers in a linear format. This was considered very annoying for most of them, regardless their level of competence and computer literacy. Similarly, they complained that too much effort was needed to quickly understand the linearly typed mathematics at the feedback frame. We believe that this factor strongly influences other problems noticed such as the fact that some of them despite struggling with the chosen activity were not paying careful attention at the feedback provided. Although it can be argued that mathematics students should learn to type and understand linearly typed mathematics, it is surely not the right context to do so. The cognitive load to understand and type in this format (which many mathematical learning environments have to employ) obstructs their learning of more important aspects at this stage.
3.4.1 Delivering more usable mathematics Based on the above, we made several changes at the initial prototype to deliver a more student friendly notation. For the input, we employed WebEQ2 which has the ability to transform ones’ input to MathML3 which we then sent to a Computer Algebra System4 that transforms it to the equivalent linear format that the student would have typed (figure 3).
Figure 3. Using WebEQ to input sqrt(2*x+1)/6 and its underlying MathML.
Moreover, the pages are written using templates and LaTEX syntax for the mathematics. As there was no need at this phase to dynamically adapt the pages, they are transformed (using appropriate stylesheets and tth5) to their HTML format in advance. As for the feedback (text and mathematics) this is stored in the page’s JavaScript and according to users actions (see next section) it is delivered to the feedback window using the DOM2 interface6 that allows the dynamic change of a page without refreshing it. 3.4.2 Other components Similar methodology was followed with other components of WALLIS. For instance, to make it easier for the students to use the navigation tree, after we improved some commonly reported problems, it was replaced with an adaptive one that changes according to their preferences and the active concept. Several setups were presented to the students and they were asked to comment on these. In general, the qualitative comments collected [12,15] were very interesting and fruitful, as they provided insight to different issues and proved how a user-centred methodology can help designers take decisions that are useful and effective for the students and not developed just because of the ‘technology hype’. For instance, colour, size and other accessibility preferences where developed immediately while various ideas on the user interface (such as the ability to move components around) had to be dropped because they were not considered so important by our students. Similarly, an animated agent (ANA, [15]) despite being effective in other research was not finally applied because relevant research [e.g.15,16] did not provide appropriate evidence of an advantage (particularly for adult students). Moreover, some of the observed actions helped verify certain things (such as their problems with the linear format) and other aspects that would have been otherwise neglected and help conduct more realistic research. All these observations will prove useful for further investigation, improvements of the system and designing any subsequent pilot-test and research on other aspects of the system. WebEQ provides an equation editor applet (see http://www.dessci.com) http://www.w3.org/Math 4 In this case, for convenience, we use the commercial CAS MAPLE (www.maplesoft.com) but similar results can be achieved with CAS that support MathML or OpenMath (such as yacas: http://yacas.sourceforge.net/) 5 tth translates tex to HTML (see http://hutchinson.belmont.ma.us/tth/) 6 http://www.w3.org/TR/DOM-Level-2-Core 2
3
4. Contents and feedback that WALLIS provides WALLIS currently addresses various concepts such as functions (function domain, odd and even functions, slope and gradient for linear functions etc.), differentiation, integration, and vectors by presenting the corresponding theory pages and employing (where appropriate and feasible) exploratory activities. 4.1. Feedback mechanism during exploratory activities The system’s ‘intelligence’ relies upon JavaMath7, and an adaptation of the feedback mechanism developed for DANTE [8], which provides feedback during the exploratory activities presented in a microworld-like environment (built using JavaMath and some inhouse developed graphical objects). The feedback mechanism tracks the goals that the author of the activity sets and students have to achieve. The goals (which are defined using a programmatic interface rather than a detailed tool yet) depending on the activity involve selecting an answer from a multiple choice question, putting objects into certain positions, giving numerical answer and so on. Along these, there is appropriate feedback associated with each goal that students have to try, and other messages to be delivered when no action is taken or when they achieve a goal etc. More details about the feedback mechanism can be found at [8] but in general this follows a three level hinting process that is increased according to the time passed and the amount of help a student requested (incrementally demonstrating the answer if necessary) but also reducing the hinting level when the student follows the system’s suggestions. For instance, at the simple exploratory activity seen at figure 1 students can explore conic sections. By moving the sliders they change the plane’s coefficient that intersects with the double cone. Then they have to select from a list the conic section produced. During their interaction, they receive feedback that helps them explore the activity more efficiently such as hints to move the sliders in different positions, prompts to provide an answer, to rotate the double cone etc. Similarly, when they select an answer (note that this is one of the goals for this activity) students receive feedback based on the preset rules (see figure 1 for a short example and [9] for more). Finally, when a student explicitly asks for help, depending on her current goal and level, the system provides feedback on how to achieve that (for example, “Think first how many branches the conic section has” or “Drag and turn the cone to see it from a different position” that would potentially lead to the system’s providing an answer if the student keeps on failing). 4.2. Self-practice activities In addition, there are other interactive activities and formative assessment parts (see [17]) that provide incremental hints and feedback on known misconceptions that the author sets. We employ tailored JavaBeans through a servlet or a MAPLE process (similarly to AIM [18]) that validates an answer for syntactic, semantic or algebraic equivalence and returns an appropriate value of correct, not-correct or a know misconception helping the system to provide more effective feedback. For instance, at the self-practice activity seen at figure 4, students have to answer a question that comprises several steps each of which affords a different type of answer (mcq, matrix, numerical, fractions) and asks students to find the associated matrix and its eigenvalues for a quadratic equation, calculate the standard form, classify it, find the associated rotation matrix, the angle of this rotation and its eccentricity. By clicking the hint button students get incremental hints about the current JavaMath (maths.hws.edu/javamath) is a collection of graphical and other mathematical objects that help validate input, calculate integrals, derivatives etc.
7
step. Additionally, when they attempt an answer the receive feedback on their misconception. From the lecturers’ experience and previous pilot-tests of the questions one can set certain common misconceptions and by evaluating the student’s answer the system can predict her ‘bug’ and provide more tailored feedback as well as suggestions as to what page she should study afterwards. For instance the eccentricity of an ellipse is given by the formula e2 = 1-(b/a) 2 and from the student’s answer we can quite safely predict the wrong formula she used (e.g. + instead of -), if she unnecessarily squared a and b from the standard form and so on.
Figure 4. A self-practice activity
4.3. Feedback on abandoning pages Finally another type of feedback comes when students try to leave a page without having completed the set activity, or explored it in a satisfactory way, they are prompted to stay in the page and complete it, if they want. Moreover, the feedback mechanism, upon completion of the activities and if the student requests so, suggests study materials (by providing links in the feedback frame window) either towards the completion of the chosen study path or targeting one of the faced misconceptions during their interactions. The feedback mechanism is currently enhanced from results of ongoing research [see 9] that looks into defining what would constitute a satisfactory interaction and how to target students’ effort and motivation based on observations of the actual students’ behaviour. 5. Future work The incremental application, and the methodology we briefly mentioned, apart from facilitating relevant research, helped in developing user-friendly and useful components. Since the main obstacle in component re-usability seems to be that systems follow tailored approaches we now try to modularise the components following appropriate specifications. For instance, the adaptive tree and the student representation lies upon mapped elements of the corresponding IMS specifications (see http://www.imsproject.org). The above process is very important, not only from a technical point of view, but also from a research point of view as it could help fast development of prototypes which could be used for research.
Equally and probably more important is the ability to share and easily author content, activities and questions. Therefore we are experimenting with the OpenMath8 and OMDoc specifications and its extensions in ActiveMath 9 [13,19]. At present it is quite time consuming to author activities such as the ones described above (not to mention that it often requires some programming experience) but we are looking into ways of developing more friendly templates by using tools such as QMath 10 and collaborating with similar approaches [eg.19] that could ease authoring of this type of activities. Currently, after evaluating findings relating to the effectiveness of the WALLIS system, in a course running in first semester of 2002/3 [see 17], we are in a constant observation phase as the system continues to be applied and used. There is now a demand for an authoring tool for lecturers and its extension to more formal assessment. We plan to continue with this methodology that marries research findings with user needs and develop, or better integrate, re-use and extend useful tools. This is only possible through the use of worldwide accepted specifications and properly evaluated tools. On the other side, further research on the theoretical aspects of the system [e.g. 9] will improve its capabilities and provide a solid application which will have the ability to target, apart from the student’s cognitive state, their effort and possibly other affective characteristics. References 1 2 3 4 5 6 7 8 9 10 11 12 13
K. R. Koedinger. Cognitive tutors as modeling tool and instructional model. In [20], 2001. B.P.Woolf, J.E. Beck, C. Eliot, and M.K. Stern. Growth and maturity of ITS. In [20]. 2001. B. du Boulay and R Luckin. Modelling human teaching tactics and strategies. IJAIED, 12:235–256, 2001. M. Sharples, N. Jeffery, B. du Boulay, D. Teather, and B. Teather. Socio-cognitive engineering. European conference on Human Centred Processes, 1999. P. Krutchen. Rational Unified Process - an introduction. Addison - Wesley, Reading, Mass, 1999. W. Clancey. Guidon-manage revisited: A socio-technical systems approach. IJAIED, 4(1):5–34, 1993. T. Conlon and H. Pain. Persistent collaboration: a methodology for applied AIED. IJAIED, 7, 1996. M. Mavrikis. Towards More Intelligent and Educational DGEs. Master’s thesis, The University of Edinburgh, School of Informatics; Artificial Intelligence, 2002. M. Mavrikis, A. Maciocia, and J. Lee. Targeting the Affective State of Students Studying Mathematics on a web-based ILE. In Young Researchers Track of the 11th International Conference on AIED, 2003. D. Wood and H. Wood. Contingency in tutoring and learning. Learning and Instruction, 6(4), 1996. Wood, H. Help seeking, learning and contingent tutoring. Computers & Education, 33(2-3), 1999. M. Mavrikis and A. Maciocia. Developing WALLIS; a web-based system to enhance mathematics teaching. ICTM, Crete, Greece, 2002. E. Melis, E. Andres, A. Franke, A. Frischauf, G. Goguadse, P. Libbrecht, M. Pollet, and C. Ullrich. ActiveMath: A web-based learning environment. IJAIED, 12, 2001.
14 Conklin J. (1987). Hypertext: An Introduction and Survey. IEEE Computer pp.17-41 Sep 1987. 15 D. Abela. Improving a web-based ITS with ANA; an animated agent. Master’s thesis, The University of Edinburgh, School of Informatics; Artificial Intelligence, 2002. 16 D. Dehn and S. van Mulken. The impact of animated agents: a review of empirical research. Internation Journal of Human-Computer Studies, 52:1–22, 2000. 17 M. Mavrikis and A. Maciocia. Incorporating Assessment in a web-based ILE. LTSN maths-caa series (available online at http://ltsn.mathstore.ac.uk/articles/maths-caa-series/index.shtml), Jul 2003 18 Klai, S., Kolokolnikov, T., Van den Bergh, N. Using Maple and the web to grade mathematics tests, International Workshop on Advanced Learning Technologies, Palmerston North, New Zealand, 2000 (also http://sourceforge.net/projects/aimmath and http://ltsn.mathstore.ac.uk/newsletter/feb2002/pdf/aim.pdf). 19 G. Goguadze, E. Melis., C. Ullrich, and P. Cairns. Problems and Solutions for Markup for Mathematical Examples and Exercises. Proceedings of Second International Conference MKM, Springer, 2003 . 20 K. D. Forbus, P. J. Feltovich, and E. Canas, (eds). Smart machines in education. AAAI/MIT Press, 2001.
OpenMath allows more semantical encoding (see http://www.openmath.org). OMDOC (http://ww.mathweb.org/OMDoc) takes account of structural elements of mathematical documents and ActiveMath’s extensions take pedagogical aspects into account 10 QMath is a batch processor that produces an OMDoc file (see http://www.matracas.org) from plain text. 8 9
