CAME 2007
STACK: making many fine judgements rapidly C J Sangwin Maths Stats and OR Network, University of Birmingham, United Kingdom
[email protected]
Abstract This paper concerns a computer aided assessment system for mathematics known as STACK, a System for Teaching and Assessment using a Computer algebra Kernel. STACK makes use of the computer algebra system Maxima for a variety of tasks, the most important of which is establishing mathematical properties of student’s answers and constructing feedback based on this.
1
Introduction
This paper concerns computer aided assessment (CAA) of mathematics. Assessment is a fundamental part of the learning cycle and is often the primary driver of students’ learning. The outcome of assessments is feedback of various kinds and an item of assessment has a number of potential purposes. Formative assessment is to support and inform students’ learning. Feedback here could be qualitative, eg written comments tailored to the student’s answer, or brief indications of where students’ written work departs from model solutions. Summative assessment is to establish the achievement of the student. In mathematics, feedback to a summative assessment is most often quantitative, either a mark or a percentage. [16] also describe evaluative assessment which is to measure the effectiveness of the teaching or the assessment of students. Such assessments could have quality enhancement or quality audit functions. The ability to automatically generate data about an individual student or across a cohort is one particular strength of CAA, allowing regular and detailed evaluative assessment. For pragmatic reasons, an individual item of assessment may be used for a number of different purposes. For example, a traditional “exercise sheet” may have a primarily formative function, with written feedback and a single numerical mark which is itself a crude formative measure. This mark could also be used for summative purposes, perhaps as a small contribution to the final course mark. Qualitative comments could be aggregated as an evaluative assessment to inform subsequent teaching (quality enhancement). Marks for a particular piece of work, such as a summative examination, could be used as an evaluative assessment for quality audit. Strong messages are communicated to students by the choices made for assessment particularly when both formative and summative purposes are combined. It is relatively common in United Kingdom undergraduate mathematics to set weekly exercise sheets. While the original purpose is formative, to encourage student engagement the numerical mark provides a small summative contribution to the overall course. Here a “reward for sustained achievement” needs to be balanced against an “opportunity to learn from mistakes”. It is the use to which the outcomes of an assessment are put which defines the purpose of the assessment not the form of these outcomes. 1
Although assessment is a key part of the learning process for the student, for the teacher marking is repetitive, time consuming and difficult. When undertaking assessment the teacher is required to make many fine judgements rapidly. It is natural therefore to seek to automate this process. In mathematics, “the process” often involves establishing various mathematical properties of a student’s work. This could include the teacher asking “has an appropriate method been selected and correctly used?”, “is the final answer algebraically equivalent to my answer?”, or “is this expression fully simplified?”. For this process to be automated it is necessary to have software tools with which mathematical expressions can be manipulated and tested against objective criteria. Computer algebra systems (CAS) are certainly designed specifically to manipulate expressions, but their ability to establish properties for assessment is something we examine here. In particular this paper describes a computer aided assessment (CAA) system for mathematics known as STACK: a System for Teaching and Assessment using a Computer algebra Kernel. As the names implies, STACK relies on a computer algebra system (CAS) at its heart to support a variety of tasks. The application of CAS to support an online assessment system is quite different from the roles to which a CAS is traditionally put. These include a reduction in the computational load, automation of graphical representations, and the ability to perform rapid re-calculation to facilitate explorations. As an illustration, consider the situation in which a student enters his or her response to a mathematical question into a CAA system. The system then uses a CAS to subtract the student’s response from the teacher’s response and to simplify the resulting expression algebraically. If the result is zero an algebraic equivalence between the student’s answer and the teacher’s answer has been established. Note that the CAA system evaluates the student’s answer which contains mathematical content, rather than allowing selections from a list of teacher provided answers, such as in multiple choice or multiple response questions. The system then takes appropriate action, ie providing feedback, and storing these outcomes in a database. Of course, establishing algebraic equivalence is only a prototype test. Other more sophisticated response processing is normally required, some of which can be achieved with the support of a CAS.
2
Background to CAS supported CAA
The primary design goal for STACK was to support the evaluation of student provided answers entered as algebraic expressions, for use in the teaching and learning of mathematics in higher education. Many existing generic CAA systems provide only question types in which teacher provided answers are selected by students. This occurs in multiple choice questions (MCQ) or similar (multiple response etc). While a well constructed multiple choice question presents a list of plausible distracters, which ideally will be constructed from knowledge and understanding of common student errors, the teacher is essentially forced to ‘give the game away’ by presenting these choices up front. There is also the possibility that the student will remember the distracter and not the correct answer, thereby the testing process could well lead to incorrect learning. In any case, the purpose of many questions is grotesquely distorted by using the MCQ format. For example, solving an equation from scratch is significantly different than checking whether each potential response is indeed a solution. Another problem particular to mathematics is where the difficulty of a reversible process is markedly altered in different directions. For example, expansion versus factorization of algebraic expressions, or integration versus differentiation. The strategic, and one might argue astute, student does not answer the question as set, but checks each answer in reverse. 2
Figure 1: An example STACK question We preferred a system which evaluates student provided answers. The use of systems in which the processing of student’s answers is supported by computer algebra has gradually increased in higher education since 2000. Perhaps the first system to make CAS a central feature was the AiM system, described by [4], with subsequent technical developments described in [14]. This system operates using Maple, as does the Wallis system of [6]. Other systems have access to a different CAS, such as CalMath which uses Mathematica, CABLE, see [7], which uses Axiom and the STACK system which uses the CAS Maxima. From private correspondence, the author is also aware of systems which use Derive in a similar way. Of course, it is not necessary to use a mainstream CAS to process student responses in a CAA system, and there are many examples of CAA and computer based learning systems in which a student is required to enter a mathematical answer. Examples include the CALM system of [1] and the Metric system of Ramsden and May (see [8]). Developers of such systems are essentially implementing “computer algebra” tailored to this particular application.
3
The student’s view
The student interface with STACK is a simple website built on a “subject”, “quiz”, “question” hierarchy. A quiz presents a list of questions on a single page, or one at a time, at the discretion of the student, thereby allowing questions to be tried in any order. Repeated attempts are encouraged and the quizzes may have have a “due date”. We concentrate on interactions with a single question, rather than with the content management system. The student enters a response to the question in the form provided on the web page, and selects a button to indicate whether the answer is to be “validated” or “marked”. They must use a typed linear syntax, although it would be straightforward to incorporate an “equation editor” interface if this was deemed appropriate. Within the validate step the following rather different checks are performed. 1. Perform a syntax check The CAS has a strict syntax which does not conform to traditional written mathematics. Some CAA systems, such as AiM, force the user to adopt this syntax. The student, on the other hand, 3
expects to type what they see. These issues might, to the experienced professional, appear to be rather trivial. To the student they are not. Experience consistently demonstrates that traditional notation exerts such a powerful force on the mind that students forget the requirement for a strict syntax. For example, they type x(2x+1) instead of x*(2*x+1), or 1
