WebALT Deliverable Template

4 downloads 39810 Views 587KB Size Report
Apr 6, 2005 - These templates are questions in which random variables ...... For some platforms, automated email reminder messages can be sent via the.
WebALT Web Advanced Learning Technologies EDC-22253-WEBALT

European Digital Content on the Global Networks

State of the Art in Mathematical E-learning D1.1 WebALT

Deliverable Number:

D1.1

Type of Deliverable:

R – Report

Contractual Delivery Date:

1 April 2005

Actual Delivery Date:

06 April 2005

Dissemination Level:

PU – Public

Workpackage:

WP 1

Document Author/s:

Hans Cuypers, Karin Poels, Rikko Verrijzer, Olga Caprotti, Jouni Karhima, Matti Pauna, Andreas Strotmann

Document Editor:

Olga Caprotti, Karin Poels

Document Status:

Release

Key Words:

e-learning platform, MathML, OpenMath

EDC-22253-WEBALT

Abstract In this deliverable we describe the state of the art in mathematical e-learning. This is done by dividing the general area of e-learning in a number of e-learning components. Then, for every e-learning component, evaluation issues are composed with which the corresponding e-learning component is evaluated. We evaluate a representative number of existing e-learning platforms by first regarding each platform as a combination of e-leaning components. Then, the evaluation issues belonging to the components are studied to evaluate the platforms. As a part of the evaluation of e-learning platforms in general, we describe and evaluate the standards used in e-learning. Not only the standard ways e-learning platforms deal with mathematics, but also the standardized methods that designers of e-learning platforms use to deal with certain e-learning aspects, such as the creation of questions.

The information contained in this report is subject to change without notice and should not be construed as a commitment by any members of the WebALT Consortium. In the event of any software or algorithms being described in this report, the WebALT Consortium assumes no responsibility for the use or inability to use any of its software or algorithms. The information is provided without any warranty of any kind and the WebALT Consortium expressly disclaims all implied warranties, including but not limited to the implied warranties of merchantability and fitness for a particular use. This document may not be copied, reproduced, or modified in whole or in part for any purpose without written permission from the WebALT Consortium. In addition, to such written permission to copy, acknowledgement of the authors of the document and all applicable portions of the copyright notice must be clearly referenced. © COPYRIGHT 2005-2006 WebALT Consortium.

I

EDC-22253-WEBALT

Table of Contents 1

INTRODUCTION ........................................................................................... 4

2

E-LEARNING PLATFORMS: COMPONENTS..................................................... 5

3

2.1

THE E-LEARNING COMPONENTS ....................................................................... 5

2.2

EVALUATION ISSUES CORRESPONDING TO E-LEARNING COMPONENTS ............................ 6

2.3

PLATFORMS EVALUATED WITH RESPECT TO E-LEARNING COMPONENTS ........................... 8

MARKUP LANGUAGES IN E-LEARNING SYSTEMS ........................................ 11 3.1

3.1.1

CAM: Content Aggregation Model...................................................... 11

3.1.2

RTE: Runtime Environment .............................................................. 13

3.1.3

SN: Sequencing and Navigation........................................................ 14

3.2

5

6

IMS E-LEARNING STANDARDS....................................................................... 15

3.2.1

IMS Question and Test Interoperability (QTI)...................................... 16

3.2.2

IMS Learning Design (LD) ................................................................ 17

3.3 4

SHAREABLE CONTENT OBJECT MODEL (SCORM)................................................. 11

REMOTE QUESTION PROTOCOL (RQP) ............................................................. 18

MARKUP LANGUAGES IN MATHEMATICS.................................................... 20 4.1

TEX AND LATEX ....................................................................................... 20

4.2

MATHML ............................................................................................... 20

4.3

OPENMATH ............................................................................................. 21

4.4

OMDOC ................................................................................................ 22

4.5

MATHDOX .............................................................................................. 22

STATE OF THE ART IN LEARNING MATERIAL ............................................. 23 5.1

ADDING LEARNING MATERIAL TO A PLATFORM ...................................................... 23

5.2

SHARING LEARNING MATERIAL WITH OTHER TEACHERS/AUTHORS ............................... 23

5.3

MATH SUPPORT IN LEARNING MATERIAL ............................................................ 24

STATE OF THE ART IN ASSESSMENTS ........................................................ 25 6.1

SUPPORTED QUESTION TYPES & AUTOMATIC GRADING .......................................... 25

6.2

HINTS & FEEDBACK ................................................................................... 27

6.3

MATH SUPPORT ........................................................................................ 28

6.3.1

Representing & Grading................................................................... 28

6.3.2

Interactive Mathematical Questions................................................... 29

6.4

STORING & OFFERING QUESTIONS .................................................................. 31

6.5

MARKUP LANGUAGES IN ONLINE ASSESSMENT IN MATHEMATICS................................ 31

6.5.1

Helsinki Learning System................................................................. 32

6.5.2

LeActiveMath ................................................................................. 33

6.5.3

Serving mathematics ...................................................................... 34

6.5.4

The MathDox system ...................................................................... 34

II

EDC-22253-WEBALT

7

STATE OF THE ART IN COMMUNICATION ................................................... 35 7.1

VIDEO SERVICES ...................................................................................... 36

7.2

DISCUSSION FORUM .................................................................................. 36

7.3

REAL-TIME CHAT ROOMS ............................................................................. 37

7.4

INTERNAL E-MAIL ...................................................................................... 37

7.5

CALENDAR/PROGRESS REVIEW ...................................................................... 37

7.6

FILE EXCHANGE ........................................................................................ 38

7.7

WHITEBOARD .......................................................................................... 38

7.8

ONLINE JOURNAL/NOTES ............................................................................. 38

8

STATE OF THE ART IN MANAGEMENT ......................................................... 39 8.1

AUTHENTICATION ...................................................................................... 39

8.2

COURSE AUTHORIZATION ............................................................................ 40

8.3

HOSTED SERVICES .................................................................................... 40

8.4

REGISTRATION ......................................................................................... 40

8.5

STUDENT TRACKING .................................................................................. 41

8.6

CURRICULUM MANAGEMENT .......................................................................... 41

8.7

CUSTOMIZED LOOK & FEEL .......................................................................... 41

8.8

OPEN SOURCE ......................................................................................... 42

8.9

MULTIPLE LANGUAGES ................................................................................ 42

9

PLATFORM POLITICS ................................................................................. 42

10

CONCLUSIONS ........................................................................................ 45

ACKNOWLEDGEMENTS...................................................................................... 45 A.

DEFINITIONS OF ACTORS AND ABBREVIATIONS....................................... 46 ACTORS .......................................................................................................... 46 STANDARDS AND FURTHER CONCEPTS ........................................................................ 47

B.

BACKGROUND INFORMATION CONCERNING NAMING OF E-LEARNING

SYSTEMS .......................................................................................................... 48 REFERENCES..................................................................................................... 51

III

EDC-22253-WEBALT

1 Introduction The WebALT project has as aim to combine existing standards for representing mathematics on the web and existing linguistic technologies to enable the creation

of

language-independent

mathematical

content. Combining these

existing standards will result in an e-learning platform that can deal with mathematics in a multi-lingual and multi-cultural environment. To be able to develop an e-learning platform that satisfies the conditions mentioned above, it is crucial to know how existing platforms deal with issues relevant to the WebALT goals. This deliverable addresses this need by a study of the current state of the art in e-learning tools with a special attention to their use and features that (may) support mathematics. The set-up of this document is as follows. In Chapter 2, we define several elearning components, each describing a certain set of functionalities concerning e-learning platforms. Subsequently, every evaluated e-learning platform is divided into the e-learning components. In this way, the concept of e-learning becomes more transparent and comprehensible. In Section 2.2, evaluation issues are composed for every e-learning component, with which the e-learning components of a platform are evaluated. In Section 2.3, a representative number of

existing

e-learning

platforms

is

divided

into

the

defined

e-learning

components. Before we subsequently continue with discussing the evaluation issues for all evaluated platforms, we study the standards for managing e-content in Chapter 3 and the different markup languages in mathematics in Chapter 4. In Chapters 5, 6, 7 and 8 the state of the art corresponding to the e-learning components is given by means of the evaluation issues corresponding to every elearning component. Chapter 9 deals with a piece of politics concerning the evaluated platforms; which platforms are mostly used, what licenses do these “biggest” platforms have etc.. In Chapter 10 we give a conclusion about the evaluations we made. In Appendix A, a list of the description of actors can be found and also a list of abbreviations of terms and concepts used in this document. In Appendix B, background information can be found about the usual naming of e-learning platforms, like a VLE, MLE, LMS, etc. To avoid confusion, in this document, we will always talk about e-learning platforms, e-learning systems, or just platforms or systems. WebALT Deliverable D1.1

4

EDC-22253-WEBALT

2 E-learning Platforms: Components An e-learning platform, or simply platform, is a software system that contains the tools and resources necessary to support the learning system actors (teachers, authors, students, and administrators) to do anything that these actors would want

to

do

in

a

class

both

locally

and

telematically

using

PC-based

communication. The advantages of such systems are manifold. The web communities formed in this way can enrich their interaction and work flow and save time and energy accessing material. The teacher is able to streamline teaching preparation. Currently, there is an enourmous amount of e-learning platforms available throughout the world, most of them obeying this definition. E-learning platforms are used by teachers and authors to create or upload questions, assessments consisting of these questions and learning material (possibly including questions) and subsequently offer these questions and learning material to students. Besides creating or uploading content and offering content to students, e-learning platforms also offer other services. These are services with which students can communicate about the content among each other or with the teacher (e.g. chat rooms, forum and email) but also services to manage and distribute content. On the platform, students can look up homework, find background material for their homework and afterwards, even upload the made homework on the platform or send the homework to the teacher by internal email. In this chapter we categorize e-learning platforms depending on the set of features a platform offers. The area of e-learning is therefore divided into coherent components in Section 2.1. Then, in Section 2.2., essential evaluation issues are composed for every e-learning component which allow to do proper evaluations on components of e-learning platforms. Finally, in Section 2.3, a matrix is represented in which all e-learning platforms evaluated are divided into the e-learning components. The matrix gives more insight in the platforms we evaluated by giving a quick overview of their functionalities.

2.1 The E-learning Components We divide the area of e-learning into four e-learning components, each of them representing a certain group of functionalities. Every existing e-learning platform can be divided into these components. The e-learning components we consider within this evaluation are defined as follows. WebALT Deliverable D1.1

5

EDC-22253-WEBALT



Learning material Learning material tools offer electronic learning material to students on the platform. This can be in the form of articles, syllabi, abstracts, etc.. In general, teachers upload learning material to the platform or create learning material on the platform itself. In the latter case, the platform offers authoring tools to the teacher.



Assessments Assessment tools offer electronic assessments to students on the platform in the form of e.g. homework, automatic graded test or just a set of questions for self-assessment. In general, teachers can acquire assessments by uploading it to the platform or creating it on the platform itself.



Communication Communication tools realize communication between students, between teachers and between a student and a teacher. Communication tools include a forum, chat rooms, internal e-mail, student webpages, address books, grade books etc..



Management Management tools include administration (think of registering students, providing students with a login name and password, administering gradebooks, etc.), but also the automatic grading of assessments, controlling teaching material and assessments submitted by authors or teachers. Management tools customize the look & feel of the platform and can set a language for the platform.

We note that it is not the case that every system has to consist of all the components. There are systems which consist of just one of the components, and there are systems which consist of all of them. For more information, see [1]. 2.2 Evaluation issues corresponding to E-learning Components In the following, we describe evaluation issues corresponding to the e-learning components as defined in the previous section. These issues will subsequently be discussed in Chapters 5,6,7 and 8, in which the state of the art in e-learning is described. •

Learning material WebALT Deliverable D1.1

6

EDC-22253-WEBALT

When evaluating the learning material component of an e-learning platform, the evaluation issues are as follows. 1. Can learning material be created by the teacher/author or can it merely be down- or uploaded? 2. What are the possibilities concerning sharing learning material with other teachers/authors? 3. How does the learning material deal with mathematics? •

Assessment When evaluating the assessment component of an e-learning platform, the issues looked at are as follows. 1. Which question types are supported by the platform? If automatic grading is possible for a certain question type, how is it done? 2. What are the possibilities for giving hints and feedback in questions? 3. How do the different question types deal with mathematics, - with respect to representation and grading? - with respect to interactive mathematical questions? 4. Storing questions and offering questions to students. - How are questions stored by the teacher? - How are assessments created from these questions? 5. What internal representation (markup) is used for the encoding of mathematical questions by the platforms using markup language for creating assessments in mathematics?



Communication For the evaluation of the communication component, an inventory is made of the communication tools offered to students and teachers by the evaluated platform. These include video services, real-time chat and internal email.



Management For the evaluation of the management component, an inventory is made of the management tools offered by the evaluated platforms. These include WebALT Deliverable D1.1

7

EDC-22253-WEBALT

authentication, student tracking, and curriculum management. Further issues that will be discussed are as follows. 1. Can the Look & Feel of the platform be customized? 2. Are multiple languages supported by the platform? 3. Is the platform open source?

2.3 Platforms evaluated with respect to E-learning Components We present a table with all evaluated platforms in alphabetical order and in which they are divided into the four e-learning components “Lesson Material”, “Assessment”, “Communication” and “Management”. We also include the way

Management

Communication

Assessment

Learning material

these platforms deal with mathematics.

Representation of Mathematics

CAS

*

LaTeX

Maple

Aim [2]

*

Angel [3]

*

*

*

MathML

-

*

*

*

*

-

-

*

*

*

*

MathML,WebEQ

*

*

*

MathML

Atutor [4] Blackboard [5] Cable [6] Calculus &

Maple TA

Axiom Mathematica

Mathematica

*

-

[7] Claroline [8] Content-e [9]

*

*

*

*

*

*

LaTeX

*

LaTeX

-

WebALT Deliverable D1.1

8

EDC-22253-WEBALT

Cose [10] Desire2Learn [11] Dmath [12] eCollege [13]

*

*

*

-

*

*

*

-

*

*

*

*

*

*

MathML

*

*

*

-

Educator [14]

WebEQ

Helsinki

-

(Web)Mathematica -

-

-

Learning

*

MathML

System [15] Ilias [16]

*

IntraLearn [17] Ischolar [18]

*

*

*

*

Figures

*

*

*

-

*

OpenMath

*

Jones eeducation

-

Maple, Darwin, Scilab -

*

*

*

*

Whiteboard

*

OmDoc

*

Figures

*

-

*

LaTeX

*

MathML

[19] LeActiveMath

*

[20] LearnWise [21] Logicampus [22] LON-CAPA [23] Maple TA [24]

*

*

*

*

*

*

*

*

*

*

Wiris, Maple, Maxima -

-

-

Maple

WebALT Deliverable D1.1

9

EDC-22253-WEBALT

Math-kit [25]

*

LaTeX, MathML Math

-

Metric [26]

*

*

Moodle [27]

*

*

*

*

LaTeX

-

Mumie [28]

*

*

*

*

LaTeX,MathML

-

Math

Maxima, Axiom

Stack [29]

*

MuPAD

*

*

Interpreter

Interpreter

Teknical Virtual

*

*

*

*

Figures

Campus [30] The Learning Manager

*

*

*

*

*

*

-

[31] The MathDox System [32]

OpenMath

Various via Brokers CAS Supporting

Wallis [33]

*

*

*

MathML

MathML or OpenMath

WebCT [34] WebTutor [35] Wiley’s eGrade [36]

MathML,

*

*

*

*

*

*

*

*

-

*

*

*

Figures

-

WebEQ -

-

WebALT Deliverable D1.1

10

EDC-22253-WEBALT

3 Markup Languages in E-learning Systems In this chapter we describe the different markup languages used in e-learning systems for the management of content (e.g. exporting and importing of content).

3.1 Shareable Content Object Model (SCORM) The Sharable Content Object Reference Model (SCORM) aims to foster creation of reusable learning content as "instructional objects" within a common technical framework for computer and Web-based learning. SCORM describes that technical framework by providing a harmonized set of guidelines, specification and standards. Current version is SCORM 2004. SCORM is being developed by ADL (Advanced Distributed Learning) Initiative, sponsored by the U.S. Department of Defence in collaboration with industry and academia. The SCORM standards are accepted as IEEE standards. The material in this section is gathered from the SCROM books offered by ADL [37]. SCORM is adopted by Blackboard, WebCT, Moodle and several others.

3.1.1 CAM: Content Aggregation Model The SCORM Content Aggregation Model (CAM) describes components used in a learning experience, how to package those components for exchange from system to system, how to describe those components to enable search and discovery. The CAM promotes consistent storage, labelling, packaging, exchange and discovery of content. It consists of the following parts.

3.1.1.1 Content Packaging (IMS) A Content Package bundles content objects with a content organization that is described in a manifest. A SCORM Content Package may represent a course, lesson, module, or may simply be a collection of related content objects. Content Packages can be nested in a tree-like structure. SCORM Content Packages may include additional information that describes how an LMS is intended to process the Content Package and its contents.

WebALT Deliverable D1.1

11

EDC-22253-WEBALT

Figure 1. Conceptual Content Package Instructional content often needs to be collected and packaged in some electronic form to enable efficient aggregation, distribution, management, and deployment. Most e-learning platforms have their own internal representation for course content. Using SCORM it is possible for example to create a course authoring tool independent of the e-learning system and import the course created as a Content Package to any other e-learning system.

3.1.1.2 Meta-data (IEEE LOM) The SCORM Meta-data Profiles represents a mapping and recommended usage of the IEEE Learning Technology Standards Committee (LTSC) Learning Object Metadata (LOM) elements for each of the SCORM Content Model Components. SCORM meta-data describes the different components of the SCORM Content Model. The LOM Information Model is broken up into nine categories. These categories are based on the definitions found in the LOM Information Model. The nine categories of meta-data elements are: 1. The General category can be used to describe general information about the SCORM Content Model Component as a whole. 2. The Life Cycle category can be used to describe features related to the history and current state of the SCORM Content Model Component and those who have affected the component during its evolution.

WebALT Deliverable D1.1

12

EDC-22253-WEBALT

3. The Meta-metadata category can be used to describe information about the meta-data record itself (rather than the SCORM Content Model Component that the record describes). 4. The Technical category can be used to describe technical requirements and characteristics of the SCORM Content Model Components. 5. The Educational category can be used to describe the educational and pedagogic characteristics of the SCORM Content Model Component. 6. The Rights category can be used to describe the intellectual property rights and conditions of use for the SCORM Content Model Component. 7. The Relation category can be used to describe features that define the relationship between this SCORM Content Model Component and other targeted components. 8. The Annotation category can be used to provide comments on the educational use of the SCORM Content Model Component and information on when and by whom the comments were created. 9. The Classification category can be used to describe where the SCORM Content Model Component falls within a particular classification system.

3.1.2 RTE: Runtime Environment The purpose of the SCORM RTE is to provide a means for interoperability between SCOs (Sharable Content Objects) and e-learning systems. SCORM provides a means for learning content to be interoperable across multiple elearning systems regardless of the tools used to create the content. For this to be possible, there must be a common way to launch content, a common way for content to communicate with an e-learning system, and predefined data elements that are exchanged between an e-learning system and content during its execution. The three components of the SCORM RTE are defined as Launch, Application Program Interface (API), and Data Model: -

The launching process defines the common way for e-learning systems to launch content objects to the learner’s Web browser.

-

The standard describes the API for content to run-time service (RTS) communication. An RTS is defined as the software that controls the execution and delivery of learning content and that may provide services WebALT Deliverable D1.1

13

EDC-22253-WEBALT

such as resource allocation, scheduling, input-output control and data management. -

The purpose of establishing a common data model is to ensure that a defined set of information about SCOs can be tracked by different elearning environments. This set of data includes, but is not limited to, information about the learner, interactions that the learner had with the SCO, objective information, success status and completion status.

Figure 2. SCORM Conceptual Run-Time Environment

3.1.3 SN: Sequencing and Navigation Parts of the SCORM Sequencing and Navigation standards are based on the IMS Simple Sequencing (SS) Specification, which defines a method for representing the intended behaviour of an authored learning experience such that any elearning system will sequence discrete learning activities in a consistent way. More specifically, it describes the branching and flow of learning activities in WebALT Deliverable D1.1

14

EDC-22253-WEBALT

terms of an Activity Tree, based on the results of a learner’s interactions with launched content objects and an authored sequencing strategy.

Figure 3. Conceptual Activity Tree and Clusters In WebALT, there is plan of a problem tree model, in which the student navigates according to correctness of the responses to individual problems. This problem tree is not the same thing that is called as an Activity Tree in the SS. However it might be that the desired behaviour can be achieved using the SS. Given the simplicity of the problem tree structure, the Simple Sequencing standard might be too complicated way to achieve it. More work is needed by IMS to determine how information flows between QTI and sequencing.

3.2 IMS E-learning Standards The mission of the IMS Global Learning Consortium [38] is to support the adoption and use of learning technology worldwide. IMS develops and promotes the

adoption

of

open

technical

specifications

for

interoperable

learning

technology. IMS is a worldwide non-profit organization that includes more than 50 Contributing Members and affiliates. These members come from every sector of the global e-learning community. They include hardware and software vendors, educational institutions, publishers, government agencies, systems integrators, multimedia content providers, and other consortia. The material in this section is gathered from various sources from the IMS. Some of the IMS standards are already incorporated in SCORM.

WebALT Deliverable D1.1

15

EDC-22253-WEBALT

3.2.1 IMS Question and Test Interoperability (QTI) The IMS Question & Test Interoperability (QTI) specification describes a data model

for

the

representation

of

question

(assessment

item)

and

test

(assessment) data and their corresponding results reports. Therefore, the specification enables the exchange of this item, assessment and results data between authoring tools, item banks, learning systems and assessment delivery systems.

Figure 4. The Role of Assessments and Assessment Items Several question types are supported, including multiple choice, matching, ordering,

text

entry

and

graphical

interaction.

QTI

also

supports

more

complicated response processing that consists of a sequence of encoded rules that are carried, in order, by the response processor. Feedback consists of material presented to the candidate conditionally based on the result of response processing. Adaptive items are a new feature of version 2 that allows an item to be scored adaptively over a sequence of attempts. This allows the candidate to alter their answer following feedback or to be posed additional questions based WebALT Deliverable D1.1

16

EDC-22253-WEBALT

on their current answer. All parts of an adaptive item are written inside a single item and therefore these parts cannot be reused in other adaptive items. QTI supports MathML in question text but does not support mathematical evaluation of answers. The Serving Mathematics project is extending the QTI standard to MathQTI [39] to enable the exchange of questions with mathematical context between question engines and authoring tools. The correctness of the learner's answer often needs to be checked with respect to a property that it has rather than just being equal to a specific value. Support for OpenMath and other mathematic representation standards is also planned. For the WebALT project the MathQTI standard seems to be very relevant but without it the QTI standard is not sufficient because of its limited support for mathematics. Unfortunately MathQTI is still under construction and it is not known when it reaches its final state.

Figure 5 An example of a QTI question

3.2.2 IMS Learning Design (LD) The IMS Learning Design Specification is a way to provide learning content with sequencing rules that are out of the scope of the IMS Simple Sequencing. Especially it allows creating learning models including multiple participants while WebALT Deliverable D1.1

17

EDC-22253-WEBALT

Simple Sequencing only allows a single learner model without any interactions. In LD the participants of a unit of training are assigned to pre-defined roles (e.g. Student, Teacher or Assessor), in which they perform activities defined for these roles. The LD is not a part of the SCORM 2004 and it is not yet decided whether or not it will be incorporated into the future versions of it. Adopting the LD seems not to be necessary for the WebALT project at this point. However, its existence should be acknowledged in order to avoid ruling out its later use as it is a possibility that this kind of advanced methods may become popular in the future of the elearning.

3.3 Remote Question Protocol (RQP) Many e-learning systems cannot present or evaluate questions of certain types, like mathematical questions. Serving Mathematics project [39] is trying to solve this problem by introducing Remote Question Protocol that allows an e-learning system to use external services for presenting and evaluating a question. The basic idea of RQP is that an e-learning system has a question bank with question templates. These templates are questions in which random variables have not yet been generated. When presenting a question there are the following stages. 1) To determine random variables used in that instance of a question template which is called question clone. If an e-learning system cannot do this by itself it could use RQP to access cloning engine which would do it. 2) To present a question. If a question includes for example MathML or OpenMath parts that an e-learning system cannot present, it could use RQP to access rendering engine which would transform these parts to something it can present. 3) To score a question. A CAS may be needed to evaluate the answer given. An e-learning system could use RQP to access scoring engine which would evaluate a question.

WebALT Deliverable D1.1

18

EDC-22253-WEBALT

User

Client

Server

Student

Assessment Engine Attempt question

Item Bank

RQP: Questio n template

Cloning Engine

RQP: Question clone RQP: Questio n clone

Rendering Engine

RQP: e.g. html Question representation Answer RQP: Questio n clone and

answer

RQP: Feedback and score

Scoring Engine

Feedback and score

Figure 6. Usage of RQP RQP could provide WebALT with following possibilities when exercises are being used by an external e-learning system that supports RQP: -

A possibility of having a multilingual item translated to any language using language generator located at WebALT server. This would be an optional way to use cloning.

-

A possibility of using a CAS located at WebALT server for cloning and scoring an item. This is a part of the original purpose of RQP.

-

A possibility of sequencing items even if problem trees located at WebALT server are stored in a structure that the e-learning system does not support. A feature that allows a scoring engine to suggest which item should be displayed next is planned to be included in RQP.

At the moment of writing this document, the definition of RQP is still not finished and it is not yet fully clear what it will provide. It is entirely possible that RQP will not support external item banks at all, so if the multilingual item bank is wanted to be located only in the WebALT server, this problem will probably need another solution. The possibility of having a language translation supported in a cloning WebALT Deliverable D1.1

19

EDC-22253-WEBALT

operation also remains to be seen. However designers of RQP are aware of the WebALT project and its main objectives and have taken them in to consideration.

4 Markup Languages in Mathematics The representation of mathematics has moved from languages conveying typesetting information to markup languages making a distinction between presentation, content and context.

4.1 TeX and LaTeX TeX and LaTeX [40] have traditionally always been the languages of choice for publishing research work, preparing handouts, and any printed material involving lots of mathematical symbols. They are still preferred by most mathematicians since they provide excellent results with a minimal editing effort: authors need to concentrate on the content and leave all typesetting details to the TeX engine. The main fault of the mathematics written in TeX/LaTeX is the fact that often it is presentation-oriented and only few authors design their mathematical macros with a semantic content view in mind. Nevertheless conversion programs, TeX/LaTeX styles and input tools are continuously being developed to allow migration from TeX/LaTeX sources to sources in richer formats like Content MathML and OpenMath with varying degrees of success (e.g. see Hermes [41], and LaTeXML [42]). Experience shows that authors often request the option of using TeX/LaTeX as input format and therefore they should be kept in mind for WebALT development.

4.2 MathML The mathematical markup language MathML [43], [44] from the World Wide Web Consortium is described in a W3C recommendation that distinguishes markup for presentation and markup for content. This allows in practise to have the content as storage format and the presentation to be generated on the fly so that the notation or language is adapted to the locale. Presentation MathML primitives cover all the typesetting features of TeX/LaTeX and browsers (Mozilla, Amaya) are now able, either natively or via a plug-in, to render Presentation MathML encodings as glyphs. Content MathML primitives used to cover the mathematics done in K-12 maths, however with the introduction of the csymbol mechanism in version 2 of the language, MathML has now been provided the extensibility it lacked in version 1 and aligns it with OpenMath (conversion

stylesheets are

available [45]). WebALT Deliverable D1.1

20

EDC-22253-WEBALT

MathML content is only partly supported by the computational software (Maple explicitly mentions MathML content) which in general uses MathML presentation when exporting the mathematics e.g. for web pages inclusion.

In [46], the

software list for MathML is provided.

4.3 OpenMath OpenMath [47], now version 2.0, is a standard aiming at the representation and communication of mathematical objects that focuses on encoding the meaning rather than the visual representation. It is designed to allow machine-processing and

exchange

unambiguous

of

mathematical

mathematical

objects

between

communication

software

between

systems

human

and

beings.

Mathematical expressions embedded in web pages but written using the OpenMath Standard can be manipulated and computed with in a meaningful and correct way.

Its encodings in XML are designed to be machine-generable and

machine-readable, rather than written by hand. The OpenMath Standard [48], is the official reference for the OpenMath language and has been approved by the OpenMath Society. This document includes an overview of the OpenMath architecture, an abstract description of OpenMath objects and two mechanisms for producing concrete encodings of such objects. The XML encoding is intended primarily for use on the web, in documents, and for applications which want to mix OpenMath to capture the content with MathML as a presentation format. The binary format is tailored to applications where large objects require a space-efficient encoding. One key idea of OpenMath is the notion of Content Dictionaries (CDs)- the mechanism by which the meaning of a symbol in the OpenMath language is encoded, as well as an XML encoding for them. Symbols defined in CDs can now be used also in Content MathML by using their URL:

DerivedSubgroup is a Content MathML expression equivalent to the OpenMath

. To display an OpenMath encoded mathematical expression, it is often necessary to transform it to Presentation MathML, SVG or TeX/LaTeX depending on the situation. In a browser, MathML is the preferred mechanism and XSLT stylesheets WebALT Deliverable D1.1

21

EDC-22253-WEBALT

for rendering OpenMath expressions containing symbols in the most commonly used CDs (Core CDs) are distributed from the OpenMath web site.

4.4

OMDoc

OMDoc [49], [50], [51] is a markup language and data model for Open Mathematical Documents on the web that, like OpenMath, concentrates on representing the meaning of mathematical formulae instead of their appearance. OMDoc builds upon OpenMath and MathML to provide support and markup for a document structure with special emphasis on the context, or in other words, in the theory level. This allows to develop semantics-based added-value services for displaying and manipulating mathematical formulae and structured documents (converting notebooks from Mathematica to OMDoc are given in [52]). OMDoc is used in the e-learning IST project LeActiveMath as source format for the didactical material.

4.5 MathDox Mathdox [32] is an XML markup language for mathematical documents. It is closely related to both DocBook and OMDoc. The former is a fairly general standard for electronic books, the latter is a very rich, and strongly logic-oriented standard for mathematical documents. Just as OMDoc, MathDox uses OpenMath for representation of mathematics. However, the main difference being that OMDoc focuses on formalizing mathematics whereas MathDox focuses on interactivity. MathDox can be viewed as an extension of DocBook. The DocBook type grammar sees to it that there are natural scopes, like chapters and sections, within a Mathdox document, where mathematical objects `live'. MathDox extends DocBook with OpenMath to provide a meaning to mathematical objects, and with XML tags representing various programming constructs as well as queries to external mathematical services, to add interactivity. Such interactions take place within part of the context, which fixes the precise semantics of the objects involved. Constructs are available for handling the mathematical context of a document. This dynamic context behaves like the state of a CAS. It keeps track of the variables introduced, their properties, their values, and their scopes. MathDox is developed by RIACA at the Technical University of Eindhoven.

WebALT Deliverable D1.1

22

EDC-22253-WEBALT

5 State of the Art in Learning Material Students in any course need learning materials. These materials can take the form of on-line lectures, but also as background material for subjects taught in a course, like articles. In this section, the evaluation issues corresponding to Learning Material, as described in Section 2.2,

are discussed. For instance we

discuss how platforms handle the creation and distribution of learning materials, what formats platforms support, and how mathematics is supported by the platforms.

5.1 Adding learning material to a platform Some of the evaluated platforms only support the possibility to upload material to the platform. These platforms let the teacher/author create his own lectures or other material on his own computer without the aid of the platform. Platforms which are depending on their learning material in this way, often allow all format types, leaving it to the browser if that format is supported or not. Examples of used format types are Microsoft Word and PDF. Learning material created outside the platform and uploaded to the platform tends to be static and without any interaction.

This

is

mostly

because,

in

these

cases,

tools

for

creating

interactiveness are missing. To solve this problem, the teacher/author is usually guided through the process of creating interactive learning material on the platform. Examples of this kind of platforms are Clarolina and IntraLearn. However it is also possible to create interactive content in this way as proven by LeActiveMath although their editing tools are still being developed. Other platforms (Content-e, Ischolar) also offer the possibility to create learning material on the platform itself. In that case, an author is restricted to the possible formats allowed by the platform. Some of these platforms are capable of offering interactive learning material. This is

possible because those platforms

force the learning material in a special kind of format such that interactivity is possible.

5.2 Sharing learning material with other teachers/authors Reusing documents and lectures or sharing them with other teachers is no problem when those lectures were made outside the platform, as discussed in the previous section. In those cases, the original documents should still be with the original author and the teachers concerned with sharing are not restricted in any WebALT Deliverable D1.1

23

EDC-22253-WEBALT

way by the platform. Moreover, most of the learning material of this type can often just be downloaded by a student or teacher from the platform. A teacher might have a problem if he wants to share lectures that are created on the platform itself. This teacher is completely dependent on the platform's functionality. In general, teachers can share content across courses and institution boundaries. The system provides a central content repository where course content files can be stored and accessed by other teachers (Angel, LONCAPA). There are platforms for which the repositories can support IMS/SCORM (Atutor, Desire2Learn, Moodle).

5.3 Math Support in Learning material In this section we discuss the possibilities to display mathematics in learning material. Displaying mathematics is not trivial and more attention to this problem is paid in Chapter 4 and in Section 6.3. The different ways platforms display mathematics depend on the format used by authors for their learning material. For the representation of mathematics, the following formats are found. •

MS Word (Doc) MS Word documents allow the use of a formula editor which enables the authors to create formulas in their documents.



PDF PDF is a display format, it is used to translate other documents into an unchangeable and uniform display format. This implies that the presentation of mathematics in PDF is dependent on the original format before translating it into PDF.



LaTeX LaTeX has good support for presenting mathematics. The display of mathematics using LaTeX can be translated to PDF, PS, etc..



An image Mathematics can be represented with a image. Such an image is created beforehand by the teacher/author by rendering e.g. a LaTeX or MathML file into an image.



MathML Web browsers are able to display documents written in HTML, the markup language of the web, but HTML does not support mathematics directly. When writing a document (lecture or article) in HTML, an author can use MathML in WebALT Deliverable D1.1

24

EDC-22253-WEBALT

order to directly display mathematics in the document. Unfortunately MathML is not yet supported by every browser. Formats as MS Word, PDF, LaTeX and images are often used in platforms with no mathematics support at all. Such a platform, for instance, is Moodle. Blackboard and Wallis are examples of platforms which uses MathML.

6 State of the Art in Assessments When learning something new, students need practice. That holds not in the least for students learning mathematics. If an e-learning platform can offer practice, then this service can make a platform extremely valuable. In this section, we make an inventory of the possibilities concerning the Assessment component of e-learning

platforms.

This

is

done

by

discussing

the

evaluation

issues

corresponding to Assessments, as defined in Section 2.2.

6.1 Supported Question Types & Automatic Grading We describe the different question types as found in the evaluated e-learning platforms. Regarding question types, there are closed and open question types. These are as follows. Closed Question Types

Open Question Types

Multiple Choice

Fill in the Blanks

True/False

Essay

Multiple Answer Ordering Matching

The closed questions all have in common that the answers are defined beforehand (predefined) and are accurate. Therefore it is easy for a platform to validate these kinds of questions and give points accordingly to a student. However, it is harder to verify the correctness of an answer when the answer is not accurate, i.e. answers to open questions. We now describe the different question types we distinguished and how automatic grading is done. For open questions we only describe automatic grading for questions that do not involve mathematics. The automatic grading of open questions that do involve mathematics is discussed in Section 6.3. WebALT Deliverable D1.1

25

EDC-22253-WEBALT

Multiple Choice In a multiple choice question, a student has to choose his/her answer from a number of predefined answers. Between the predefined answers there always is exactly one correct answer, the others are false. This is a well known question type in the old fashioned paper exams and even better known in the electronic equivalent of those exams. This is mainly because of the ease of verifying the student’s answers. True/False True/false questions are simplified multiple choice questions. Instead of choosing between a number of predefined answers, a student chooses between the answers “True” or “False”. Either the student answers that the statement given in the question is true or that it is false. Multiple Answer Multiple answer questions are similar to multiple choice questions except that they allow/expect more than one correct answer. Between the predefined answers, more than one can be correct and the student has to indicate which ones are correct. There are platforms (Ischolar) offering multiple answer questions which can punish students with penalty points for giving a wrong answer. Another possibility is that the platform rewards each possible predefined correct answer differently; a teacher might find it important that a student finds one particular correct answer and cares less about the other correct answers (Moodle). Ordering This question type merely applies to things that can be measured. A student is given a number of objects and has to put them in the right order with respect to for example size, height or wealth. Matching This question type only applies to pairs. The student is given an even number of objects and has to retrieve the pairs that match. Points are given on basis of the amount of correctly matched pairs. Fill in the blanks This is a type of an open question. The student is presented with a text of which parts (words or other kinds of entries) are omitted. Another possibility is that the student is given a question to which no predefined answers are given. The student has to fill in the missing parts or write down the correct answer. For this question type, it is more difficult for a platform to validate the correctness of the WebALT Deliverable D1.1

26

EDC-22253-WEBALT

given answer since the student is no longer forced to choose from predefined answers. We describe the ways the evaluated platforms deal with this problem of validating. Some platforms only validate the given answer as correct if it is written exactly like the correct answer as set by the teacher, such that a spelling mistake causes incorrect validation (Claroline). There are other platforms which also offer less strict grading. These platforms, given that the correct answer is text, can for example grade the given answer case-sensitively, validate the given answer as correct if the student omits or uses extra spaces, and gives the student part of the points if he has written down only a substring of the correct answer. If the correct answer is a number, then the student’s answer can be validated as correct if the answer is within a certain range from the correct answer. A platform that has all of these options is Ischolar. Essay A student is given an assignment to which an answer should be given that consists of more than one word or sentence. This means that for an essay question, the platform itself cannot grade the given answer, such that no automatic grading is possible and the system depends on the teacher to mark the student’s answers. The platform therefore forwards the answer to the teacher, who can subsequently validate the given answer. There are platforms capable of offering all these question types (Angel, Blackboard, Desire2Learn).

6.2 Hints & Feedback A platform is of extra value when it is capable of giving hints to the student in the right direction or giving feedback concerning a specific error made by the student. There are platforms (e.g. Stack) which can, given that the student answered the question wrong, recognise the type of mistake made by the student. Depending on that type of mistake, the platform gives a certain hint or feedback. If the mistake is for example minor, the platform can make the student aware of this mistake such that he is warned for similar mistakes during following questions. In other cases, the platform gives a hint in the direction of the mistake and the student gets a second chance to do the question (for fewer points). There are also platforms (Content-e, Angel) which, after doing a question and depending on the answer given by the student, direct the student to a piece of learning material in which he can learn about his mistake. In this way, a student determines his own learning path.

WebALT Deliverable D1.1

27

EDC-22253-WEBALT

6.3 Math support In this section, we discuss how mathematics is treated by the different question types. This includes how mathematics is represented in questions and how open mathematical questions are graded. Platforms offering support for mathematics have a great surplus value. They can supply teachers with extra functionalities for creating mathematical questions. We discuss these extra functionalities and possibilities for interactive mathematics.

6.3.1 Representing & Grading Concerning the way platforms deal with mathematics, there is an important difference between closed and open questions. If a platform only offers closed questions, then to deal with mathematics in a correct way, a platforms only needs to represent mathematics in a correct way, for instance with LaTeX or with MathML. This is because with closed questions, a student has to choose between predefined answers. If these answers are represented in a mathematically correct way, then the platform is mathematically safe, i.e. it deals with mathematics in a correct way. There are platforms which barely support mathematics (Atutor, Teknical Virtual Campus, Ilias); teachers insert mathematical expressions with plain text or by including an image containing the mathematical expressions. Most platforms with math support use MathML for representation (see Section 2.3). If on the other hand, the platform also offers open questions, then besides representing mathematics properly, such a platform must also be able to read the mathematical meaning of the answers given by the student. This is because almost every mathematical expression can be written in many different ways without being incorrect. For example, if the correct answer to an open question is

Cos ( 2 x ) , then if the student’s answers is 2Cos 2 ( x ) − 1 , the platform should be able to verify that this answer is equal to the correct answer as given by the teacher. Platforms capable of doing this, interpret the meaning of mathematics by making use of a computer algebra system (CAS) like Maple [53], Maxima [54], Mathematica [55], Axiom [56], MuPAD [57], Wiris [58], Darwin [59] and Scilab [60]. Besides the problem of evaluating given answers, there is the problem of entering a mathematical answer by the student. The platforms we evaluated have revealed that there are three different solutions to this problem. There are platforms (e.g. Maple) which oblige students to write the answers to CAS-related questions in a certain CAS language (dependent on the CAS used by the WebALT Deliverable D1.1

28

EDC-22253-WEBALT

platform). Other platforms include a math editor (Blackboard) or link a math interpreter to the typing field. A math editor uses the point and click approach. This means that the user (student or teacher) points at the mathematical symbol he wants to include in his text and clicks. The symbol is placed in the text and offers the possibility to enter arguments if necessary. The math interpreter obliges the user (student or teacher) to know the standard language in which mathematics is written in plain text (like x^2 for x 2 and sqrt(x) for

x ). While

the user types the answer in this manner, the interpreter translates the written answer into the corresponding mathematical expression and displays its results. In this way, the user can preview the written answer. He is offered the possibility to correct himself if the mathematical expression as shown by the interpreter is not equal to the mathematical expression the user had in mind while writing the mathematical answer.

6.3.2 Interactive Mathematical Questions We now describe the extra functionalities for creating (mathematical) questions when a platform supports mathematics.

6.3.2.1 CAS graded Questions Not only is a CAS used to verify whether the answer given by the student is equivalent to the correct answer as predefined by the teacher. A CAS is also used by platforms (Maple TA) to calculate the correct answer itself and verify whether this correct answer is equivalent to the answer given by the student. In other words, the teacher does not calculate the correct answer, but the CAS does.

6.3.2.2 Randomized Questions There are platforms offering randomized questions. In such a question, a teacher defines random variables and uses them in the question and as a consequence also in the answers. In this way, every time a student calls the question, he or she sees a slightly different version of the question. This prevents students from cheating during an exam; students cannot simply copy each other’s answers. Randomized questions make self-assessments become more valuable; if a student does not manage to answer a question correctly, he can do the question again (which gives him a slightly changed, but different question) until he understands the mathematics behind the question well enough. We give an example of a randomized question. A teacher can define the open mathematical question “Give the derivative of x n .”, where the teacher sets n to WebALT Deliverable D1.1

29

EDC-22253-WEBALT

be a random integer between 2 and 10. Every time a student opens this question he finds a slightly changed question such that he can keep practicing differentiation. Certain platforms apply randomization to the predefined answers of closed mathematical questions (Maple TA, LON-CAPA). Given the multiple choice question “What is the derivative of x n ?”, the predefined answers are for example

nx n −1 , (n − 1) x n +1 , (n − 1) x n −1 . Some platforms, knowing the random variable n , can read the mathematical meaning of n + 1, n − 1 . For some other platforms (Maple TA) the teacher has to define new random variables n1 = n + 1 and n2 = n − 1 . The student will not notice this if the teacher sets all variables correctly.

We note

that Maple TA gave some inexplicable error reports when setting up randomized questions.

6.3.2.3 Continuing with a Previous Answer Platforms offering this kind of interactive questions are capable of creating multiple-step questions. That is, the input to a certain sub-question is equal to the output (student’s answer) of the previous sub-question; the answer to a certain sub-question is graded with respect to the answer to the preceding subquestion. This means that a student can proceed with an incorrect answer and that his answer to the following sub-question is graded with respect to the incorrect answer. In other words, a student is punished only once for every incorrect answer. There are platforms (e.g. Ischolar) capable of continuing with a previous answer in the following way. A student is asked to write down a second order polynomial

f (x ) or the platform defines a polynomial f (x ) itself. The student is then asked to write down the derivative f ′( x ) , the second derivative f ′′( x ) and the primitive

F (x ) . Only then, the answers are evaluated by using a Computer Algebra System. In such a CAS, the teacher writes a piece of code that is applied to all answers. According to the output of the CAS, a certain validation and feedback is given to the student. The CAS evaluates f ′( x ) and F (x ) with respect to f (x ) , but can evaluate f ′′( x ) with respect to f ′( x ) . This means that if D ( f ( x )) ≠ f ′( x ) but

D ( f ′( x)) = f ′′( x ) , the student is still graded some points. We note that with Ischolar, only one piece of CAS code is used for every question, therefore the question is evaluated and validated as a whole. It is not the case with Ischolar that every sub-question can be evaluated and validated separately. On the other WebALT Deliverable D1.1

30

EDC-22253-WEBALT

hand, during our evaluations, we did not find a platform that can validate every sub-question separately.

6.4 Storing & Offering Questions The purpose of creating questions is to offer the questions to students in the form of a practice session or an examination. We already discussed two aspects relating to this; the different options in automatically grading questions and entering mathematical answers by students. We now discuss how questions are stored by the platform and the possibilities concerning creating assessments to offer the questions to students. There are platforms that bind questions to courses, limiting the use of those questions only to that course (e.g. older versions of Blackboard). Other platforms store the questions in question banks or pools (Angel, Blackboard, Ischolar, Maple TA). These question banks are categorized in order to retrieve a certain question very quickly. Question banks subsequently can consist of multiple levels of Question Topics such that even more categorization is possible. Platforms using question banks offer the teacher the possibility to use random selections of questions from these banks to create an exam for a student. As a consequence, it becomes very easy for a teacher to create individual exams which helps to prevent cheating. Furthermore, questions in question banks can often be shared among teachers more easily. Many platforms offer time based exams and tests. Besides the possibility to enter a begin date after which the test becomes available to students and an end date after which the test is no longer available there is sometimes also a possibility to enter a maximum duration time (Blackboard). Most platforms can also set an upper limit for the number of times a student can do a test. There are platforms that offer the possibility for proctored exams (Maple TA, Angel, eCollege). In general, the platforms can override the automated scoring such that writing errors made by the students can be graded as correct.

6.5 Markup Languages in Online Assessment in Mathematics We need to review all systems that allow assessment in mathematics to see which internal representation is used for the encoding of the exercises. Important features for WebALT are: •

language-independent markup for mathematical content WebALT Deliverable D1.1

31

EDC-22253-WEBALT



possibility to evaluate symbolic expressions



introducing symbolic variables and algorithmic problem structure



problem trees (multi-step exercises)

6.5.1 Helsinki Learning System The Helsinki Learning System (HLS) [15] project started at September 2001 in University

of

Helsinki,

Department

of

Mathematics.

Data

structures

for

representing single problems, problem trees (called exercises), and Course Content Dictionaries (CCD) were created. Single problems could be either multiple choice or string problems, but evaluation of symbolic expressions in string problems was not implemented. Also randomly generated parameters and algorithmic problems are not implemented in HLS.

Figure 7. HLS Data structures Single problems are stored as xml elements in an xml database system using the xmldb standard. A simple xml schema representing problems was created in which mathematical notation is encoded with LaTeX or MathML.

WebALT Deliverable D1.1

32

EDC-22253-WEBALT

Exercises consist of a problem tree and feedbacks to single problems. For each problem links to next problems in case of a correct and wrong answer are given. This allows a more general graph than a problem tree to be represented though the tree structure seems to be pedagogically most relevant. Course Content Dictionary defines the topic hierarchy of a course. It is represented as a recursive xml structure so that a topic element can have a topic sub-element allowing arbitrary deep topic hierarchy. For each problem there is a link to a specific topic in the CCD which enables questions to be linked to specific sections of an on-line learning material or even to a standard text book for the course. However structuring and storing course learning materials is not currently a part of the HLS project.

6.5.2 LeActiveMath LeActiveMath [20], [61] is a three year European project running from Jan 2004 until Dec 2006. LeActiveMath aims at developing an open-source user-adaptive, interactive, language-enhanced web-based learning environment for mathematics which employs intelligent technologies. It is based on semantically encoded learning objects that are annotated by metadata (educational, administrative, etc.). LeActiveMath provides modular components such as a course generator, a learner model, a domain reasoner and a natural language dialogue generator, a knowledge base and several integrated service

systems

[62].

The

course

generator uses pedagogical rules for

assembling individual workbooks according to the learner's goals, preferences and knowledge. Learner's activities are tracked in a learner history and diagnosed by evaluators that update the learner model. The domain reasoner analyses the learner’s activities and generates the next step in the learning process. LeActiveMath's interactive exercise architecture clearly separates components for evaluation and feedback. This adds flexibility, generality and transparency to exercises and supports action tracing and diagnosis. The knowledge based component is stored using the open source XML database eXist. LeActiveMath includes tools such as a dictionary, a software eyetracker, a suggestion mechanism and notes. The dictionary facilitates searching for WebALT Deliverable D1.1

33

EDC-22253-WEBALT

concepts and browsing their dependencies. The eyetracker 'DFKeye' tracks the learner's attention. The suggestion mechanism supports the user, if he gets stuck. Notes allow writing private and public notes which are then attached to concepts.

6.5.3 Serving mathematics Serving Mathematics [39], [63], is a UK based community focussing on opensource tools for on-line assessment in mathematics education. Various projects take part in the Serving Mathematics projects, among them we find - AiM: a self-testing and assessment system based on server-side marking using the computer algebra system Maple. - METRIC: a self-testing and mathematical learning system based on client-side marking implemented in Java. - STACK: System for Teaching and Assessment using the computer algebra system Maxima. - WaLLiS: an interactive learning environment that focuses on providing adaptive and intelligent feedback. - Moodle: a user-friendly and modular Virtual Learning Environement. - APIS [64]: a rendering and scoring engine for QTI v.2 assessment items. - Assis [65]: Assessment and simple sequencing integration services - TIP [66]: Integration between Bodington VLE [67], LAMS [68] and TOIA [69]. The aim of the project is to have these various project work together. There are two ways in which the interoperability issue is being approached, by the Remote Question Protocol (RQP), and by MathQTI. Both standards are being developed within the Serving Mathametics project.

6.5.4 The MathDox system The MathDox system [32] is a web-based software system for serving interactive mathematical documents based on the MathDox language. It can be divided into three parts: the document server, the client and tools for accessing mathematical services. WebALT Deliverable D1.1

34

EDC-22253-WEBALT

The client is

a math-enabled web browser. It presents views of the serviced

documents to the user, interacts with the user, and communicates user input to the document server. The document server is the main part of the system. This server caters for presentation, communication, and context. It supports a wide range of actions ranging from handling queries to searching within documents for mathematical content and from placing (and retrieving) objects into the context, to rendering documents in different views. The document server is realized as a Java enhanced web application inside a web server. It is formed by two different parts, a document manager and a context manager. Interactive mathematical documents in MathDox format can be thought of as programs (scripts) encoding the production of a view on the document. In generating such a view, they can make use of the information contained in the static context, in the dynamic context (scopes and variables), the user input communicated along with a request, and results of computations carried on by one or more mathematical services. The document manager serves such views to the client depending on all these input parameters. The context manager is responsible for managing the input parameters in both the static and dynamic context. The final part of the MathDox system consists of various mathematical services. Such services can be very diverse: some may serve as general interfaces to CAS or to Theorem Provers. The MathDox software provides ways to access these services via standard protocols, among which those developed under the MONET project ([70]).

7 State of the Art in Communication Not all platforms we evaluated, offer a communication component with their elearning platform. If they offer it, then this component is usually very extended. The communication component offers the students and the teachers to communicate amongst each other in a very extensive, instant way. This means that a student doing e-learning can have instant help from fellow students or a teacher

at

any

moment.

Besides

a

supportive

tool

for

students,

the

communication components also serve as a supportive tool for teachers amongst each other. During

the

evaluations

of

the

platforms,

we

encountered

the

following

communication tools. WebALT Deliverable D1.1

35

EDC-22253-WEBALT

Video Services

Discussion Forum

Real-time Chat Rooms

Internal e-mail

Calendar

File Exchange

Whiteboard

Online journal/Notes

In the following, we elaborate on the function of these tools and on which platforms offer them.

7.1 Video Services Video Services enable teachers to either stream video from within the system, or else enable video conferencing. This can be either between teachers and students or between students. Video Services include tools for broadcasting video to those without a video input device. There are platforms (Angel) in which teachers can include real-time video in slide or

web presentations within the optional synchronous tools. Some platforms (Angel, eCollege) give course developers the possibility to integrate streamed real audio and video into a course. 7.2 Discussion Forum Discussion forums are online tools that capture the exchange of messages over time that form discussions about certain subjects. A student or teacher can start a discussion in the forum and because forums are organized into categories, the exchange of messages and responses are grouped together and are easy to find. All evaluated platforms that have the communication component, offer a Discussion forum. Some platforms offer extra features within the discussion forum. We illuminate some of these features. For most platforms, posts in a discussion are plaintext or formatted text (for which a formatting text editor is included) but for some platforms, posts can also be HTML (Angel, Blackboard, Desire2Learn, LON-CAPA).

Blackboard has a

special formatting text editor which can create mathematical expressions. For most platforms, instructors can associate a discussion with any course content and for any group of students (Angel, Blackboard, Cose, Desire2Learn, eCollege, Educator, Ilias, Jones e-education, WebCT). Instructors can limit discussions to specific time periods (Angel, eCollege, Jones e-education).

WebALT Deliverable D1.1

36

EDC-22253-WEBALT

7.3 Real-time Chat Rooms Real-time

chat

is

an

online

conversation

between

people

that

involves

exchanging messages back and forth at virtually the same time. Again, most platforms having a communication component, offer real-time chat rooms. We illuminate platforms with special features. A number of platforms offer unlimited simultaneous group discussions, such that more than two persons can talk about a certain subject (Blackboard, eCollege, WebCT). There are platforms where students can be suspended from the chatroom by the teacher (Blackboard, Educator). For some platforms, teachers can moderate and monitor chats (Angel, Blackboard, Educator) or schedule a chat session via the course calendar (Angel, eCollege, Moodle). Some platforms create archive logs for all chat rooms (Angel, Blackboard, Desire2Learn, eCollege, Educator, Moodle, and WebCT).

7.4 Internal e-mail Internal email is electronic mail that can be read or sent from inside an online course. These email tools enable messages to be read and sent exclusively inside the course (Atutor, Blackboard, Desire2Learn, Educator, Ilias, Jones e-education, Learnwise, LON-CAPA, WebCT) or alternatively enable links to external email addresses of students in the course such that contacting course members is facilitated. For some platforms, automated email reminder messages can be sent via the internal email (Angel, Educator). Some platforms can send messages to groups as a whole (Angel, Cose, Desire2Learn, Educator, Ilias, eCollege, Jones eeducation, Learnwise).

7.5 Calendar/Progress Review Calendar/Progress Review tools enable students to document their plans for a course and the associated assignments in a course. For most platforms, Calendar/Progress Review tools enable students to keep track of their assignments, deadlines and due dates. They can check their marks on assignments and test, as well as their progress through the course material and they can compare their marks on an assignment with the average score on that assignment, view total points earned, total points possible, etc. (Angel, Blackboard, Desire2Learn, eCollege, Moodle, WebCT). For some platforms,

WebALT Deliverable D1.1

37

EDC-22253-WEBALT

entries (events, tasks) can be posted by teachers or students to an entire class, a certain team or a specific user (Angel).

7.6 File Exchange File exchange tools allow students to upload files from their local computers to the platform and share these files with teachers or other students in an online course. Most platforms offer the possibility for students to submit assignments using drop boxes. Some platforms use virus detection technology throughout the file upload/download process (Angel, Educator).

7.7 Whiteboard A whiteboard tool is an electronic version of a dry-erase board and is used by teachers and learners in a virtual classroom. It also offers other synchronous services like application sharing, group browsing, and voice chat. Application sharing allows a software program running on one computer to be viewed, and sometimes controlled from a remote computer. For example, a teacher using this feature can demonstrate a chemistry experiment or a software utility to an online student and allow the student to use the demonstration software from their own computer. Group Web Browsing allows a teacher to guide students on a tour of web sites using a shared browser window. Voice chat allows two or more persons to communicate in real time via microphones (conference call style) over an internet connection. There are platforms for which the whiteboard can have multiple instances in the same course (Blackboard). Some can archive a recording or a snapshot of a of a whiteboard session for future viewing (Angel, Blackboard, eCollege, WebCT). There

are

whiteboards

that

support

mathematical

symbols

(Blackboard,

eCollege).

7.8 Online Journal/Notes Online Journal/Notes enable students to make notes (e.g. course experiences) in a personal or private journal/diary. These notes can be personal or private. Students can share personal notes with a teacher or other students (). They cannot share private journal entries. This tool can for example be used to facilitate writing assignments (parts are written by the teacher at different moments and later assembled into a document) but also to make personal WebALT Deliverable D1.1

38

EDC-22253-WEBALT

annotations to course pages that can later be used as a study aid. The tool can also be used to record reflections about personal learning accomplishments and how to apply this new knowledge. Some platforms edit the notes in HTML (Angel, Desire2Learn, Learnwise).

8 State of the Art in Management All platforms offering automatic graded assessments or communication need some kind of management or administration tools. We discuss the different management tools, as offered by the platforms we evaluated. The different management tools are as follows. Authentication

Course Authorization

Hosted Services

Registration Integration

Student Tracking

Curriculum Management

Customized Look & Feel Open Source Multiple Languages

8.1 Authentication By providing an appropriate user name (login) and password to a user (student or teacher), the user can authenticate himself to the platform; authentication refers to the procedure by which user names and passwords are created and maintained. Almost all platforms can set courses to be publicly accessible but can also protect individual courses with username and password. Most platforms have a password reminder system. Authentication systems can involve a single logon which is the most user friendly and most vulnerable to hacking. More complicated systems can involve layers with separate logins for each layer and secure socket layer transaction (SSL) encryption (Angel, Blackboard, Desire2Learn, WebCT). There

are

platforms

in

which

administrators

can

set

up

fail-through

authentication against a secondary source in the event that the primary source fails (Angel, Blackboard, Desire2Learn, and WebCT).

WebALT Deliverable D1.1

39

EDC-22253-WEBALT

8.2 Course Authorization Course authorization tools are used to assign specific access privileges to course content and are based on specific user roles, e.g. students, teachers, teaching assistants, authors, proctors or administrators. For example, students can view pages and authors can author pages. Students

and

teachers

typically

need

different

tools

to

complete

their

instructional responsibilities. For example, students need to be able to view their records in a grade book but instructors need to be able to view and modify the records of all students in the course. Most platforms provide a small set of predefined user roles such as instructors, students, managers and guests. In other platforms, an administrator can add and define additional user roles (Angel, Blackboard, and Desire2Learn). There are platforms in which instructors can specify access permissions for each student (Angel, Desire2Learn, Learnwise, and LON-CAPA).

8.3 Hosted Services Hosted Services means that the platform provider offers the platform on a server at the provider’s location such that the institution using the platform does not have to provide any hardware. An important aspect of Hosted Services is that the platform provider takes responsibility for all technical support and maintenance of the server, as well as the actual web service of providing online courses. Almost all platforms we evaluated offer hosted services.

8.4 Registration All platforms use registration tools to add students to and drop students from a course. Administrators and/or instructors use registration tools but students also use them when self-registration is available. Students can also be added to or dropped from a course through integration of the platform with a Student Information System (SIS) (Angel, Blackboard, Desire2Learn, eCollege, Educator, Intralearn, LON-CAPA, Moodle, Learnwise, WebCT). Integration with SIS enables the platform to work with products such as SCT Banner, Peoplesoft, or Datatel. Typically, integration allows the following types of functionality: shared common student information, ability to transfer grades between the SIS and the platform, and the ability to have common accounts. Time limited student self-registration may be available to shift the clerical burden of the process to the students (Angel). There are platforms that are compliant WebALT Deliverable D1.1

40

EDC-22253-WEBALT

with the IMS Enterprice Specification of Student Data (Angel, Blackboard, Desire2Learn, Learnwise, teknical Virtual Campus, WebCT).

8.5 Student Tracking Student Tracking is the ability to track the usage of course materials by students, and to perform additional analysis and reporting both of aggregate and individual usage. Student Tracking tools include statistical analysis of student performance (on assignments) data and progress reports for individual students in the course. The progress reports generally consist of the number of times, time and date when an activity (course content, discussion forum, assessments, etc.) occurred. All platforms evaluated are capable of student tracking. With some platforms, teachers can set a flag on individual course components to track the frequency with which students access those components (Angel, Blackboard, Moodle).

8.6 Curriculum Management Curriculum

management

provides

students

with

customized

programs

or

activities based on prerequisites, prior work, or results from testing. In platforms capable of curriculum management (Angel, Desire2Learn, WebCT), teachers can specify multiple paths through courses for different skill levels or job functions. For some platforms (Angel), a teacher can map specific learning objects to individual training needs.

8.7 Customized Look & Feel Customized Look and Feel is the ability to change the graphics and how a course looks. This also includes the branding of content with institutional logos and navigation to provide a consistent look-and-feel across the entire institutional site. It also makes possible the integration of the system with additional institutional resources such as the library. For almost all platforms, institutions can apply their own institutional images, headers and footers across all courses. A number of platforms provide over a set of default course look & feel templates (Angel, Moodle, WebCT), with some platforms institutions can also create their own look & feel (Angel, Atutor, LONCAPA, Moodle, The Learning Manager, WebCT). There are also platforms where

WebALT Deliverable D1.1

41

EDC-22253-WEBALT

administrators can change the availability, order and name of menu items (Angel, Moodle).

8.8 Open Source Open Source means that the software is delivered with the source code and that the license agreement gives the licensee-holder the right to modify and redistribute the software. A representative open source license is the GNU General Public License (Atutor, Claroline, Ilias, LON-CAPA, Moodle). The official definition of Open Source software

is

maintained

by

the

Open

Source

Initiative

(http://www.opensource.org/index.php).

8.9 Multiple Languages There are platforms for which the resources (text pages, homework problems, etc.) can have multiple languages embedded into them (LON-CAPA). According to the course preferences, different sections are rendered (e.g. the Spanish or the Dutch sections) when the resources is rendered server-side. Some platforms are simply available in multiple languages (Claroline in 28 languages), other platforms offer language translations as plug-in packs (Moodle in 58 languages, Atutor in 11 languages, WebCT in 14 languages).

9 Platform Politics In this chapter, we discuss which platforms are most commonly used and give some side information about them, for both the open source systems and the commercial systems. The information collected in this chapter is gathered from the online sites of the various software systems. Open source systems are characterized by freely available software code, and the freedom to modify and redistribute code. Many of these projects involve collaboration between institutions, and/or a commercial element to provide user support and customization. To use a commercial e-learning system on the other hand, a university serving 15.000 students could spend $75.000 annually in licensing fees. E-learning system’s providers charged less in annual licensing fees when they introduced their systems to the market, but soon realized that they had to increase pricing if they were to grow. In 2002, two major commercial systems increased the annual WebALT Deliverable D1.1

42

EDC-22253-WEBALT

fee as high as with 20 percent. The installation of such complex systems generally requires a school to spend an additional $10.000 to $20.000. Worldwide, Blackboard, WebCT and eCollege are among the leading commercial systems. Added together, the client bases of these companies represent thousands of educational enterprises. In addition, there are a mix of campus IT solutions and other projects that have morphed into commercial offerings. One of the biggest of these is ANGEL, the e-learning system of CyberLearning Labs (www.cyberlearninglabs.com). CyberLearningLabs has since spun off as a commercial venture. The vendors of these leading systems (Blackboard, WebCT, Angel, eCollege) are rapidly expanding internationalization features with a view to foreign markets. Costs have also increased substantially. Besides the mentioned commercial platforms, also FirstClass, TopClass, Lotus Learning Space, ClassFronter, LUVIT and Tutor2000 seem to be common used platforms in Europe. The most important challengers to the commercial systems are Moodle and Sakai. These open source systems are challenging on price, flexibility and alternative pedagogical models. For the most part, they currently aren’t overly challenging on issues like “enterprise suitability”, vendor support, documentation and adoption. The major commercial vendors of e-learning systems will begin to make visibly defensive moves in response to the growing threat from opensource alternatives; at the moment, there are many educational institutions migrating from a commercial system to Moodle. We now give some additional information about the previously mentioned elearning platforms. Blackboard Blackboard Inc. is a privately held company formed in 1997 and based in Washington D.C., USA. The company has risen to become one of the major global e-learning system vendors, seeing a 11,047% revenue growth from 1998 through 2002. Blackboard began as a merger of two consultants and a Cornell University student-faculty software initiative. Blackboard now offers software products and services for e-Education programs across primary and higher education as well as serving corporate and government markets. Angel (CyberLearning Labs) CyberLearning Labs, Inc.’s product is ANGEL, originally developed in the CyerLab at Purdue University School of Engineering and Technology, Indiana University. WebALT Deliverable D1.1

43

EDC-22253-WEBALT

CyberLearning Labs is the privately owned commercial extension of this product, beginning operations in 2000. Describing itself as an ‘up-and-coming’ player in the e-learning marketplace they have secured contracts with the likes of Penn State University, and have signed an agreement with a German partner to develop and distribute a German language version of ANGEL.

eCollege eCollege (ECLG: NASDAQ) is a publicly traded company, based in Colorado, U.S., and incorporated in July 1996. Their share of the e-learning market is primarily based in North America, although eCollege International was incorporated in January 2002, suggesting greater international ambitions in the future. eCollege clients

range

corporations.

from

primary

Management

schools

to

solutions

are

higher offered

education for

institutions

teaching,

to

program

administration, and technology infrastructure. eCollege uses an application service provider (ASP) model allowing customers to outsource their online content

development,

management,

training,

infrastructure,

hosting,

and

support. The eCollege product tends to appeal to small to mid range educational institutions looking to set up a distance learning program from scratch, and forprofit institutions content to outsource what are seen as non-core activities. eCollege recently declared its first profits. WebCT WebCT is a privately held company whose investors include CMGI @Ventures (an Internet operating and development company), JPMorgan Partners (a private equity organisation), SCT (a technology solutions provider for higher education), and Thomson Corporation (one of the world's largest publishers). Formed in 1995 from a University of British Columbia, Canada, initiative the company now offers one of the world’s most popular e-learning products. WebCT’s market focus is higher education, advertising clients in over 85 countries and products in 14 different languages. WebCT has yet to declare profitability. Moodle Based on a ‘social constructionist theory’, Moodle is one of the most popular open source e-learning systems. It is available in 34 different languages and according to the website there are 984 sites from 74 countries currently registered (March 2004). Moodle.com is the commercial company run by Moodle's core developers offering fully-supported Moodle hosting, remote support contracts, custom code development and consulting. The Sakai Project WebALT Deliverable D1.1

44

EDC-22253-WEBALT

The Sakai Project [71] is a ‘community source’ initiative founded by University of Michigan, Indiana University, MIT, Stanford, the uPortal Consortium, and the Open Knowledge Initiative (OKI). Based on many of the principles of open source,

‘community

source’

relies

more

explicitly

on

defined

roles,

responsibilities, and funded commitments by community members than some open source development models. The project is producing open source Collaboration and Learning Environment (CLE) software with an anticipated first release in July 2004. All software is available free. For further resources and services, such as access to technical support staff, the online Sakai Project knowledgebase, pre-release code and developer workshops institutions must join the Sakai Educational Partner’s Program (SEPP) for $10,000 annually over a three year commitment.

10 Conclusions The

WebALT

project

aims

at

using

existing

standards

for

representing

mathematics on the web as well as existing linguistic technologies to produce language-independent mathematical didactical material. WebALT solutions will be commercially developed and marketed by WebALT Inc. To be commercially interesting, WebALT products should not only be of the highest quality and use the latest technology and didactical insight, but also be made available to a large audience. For these reasons, WebALT products should be compatible with the most advanced as well as the most used e-learning platforms available. Among the most advanced mathematical e-learning platforms we find several open-source projects taking part in the Serving Mathematics initiative but also the commercial MapleTA. Ongoing initiatives like LeActiveMath, OmDOC and MathDox also deserve attention from WebALT. The most used e-learning platforms are the commercial systems Blackboard and WebCT. The open-source Moodle however, is rapidly growing. Although not used by all the players mentioned above, it seems that the standards developed by ADLNet for SCORM and the IMS Global Learning Consortium for QTI (and its extension mathQTI) should be adopted by WebALT.

Acknowledgements The authors wish to thank all persons who helped in evaluating the various elearning platforms. In particular we thank Chris Sangwin from the project Serving Mathematics, André Heck from Maple TA and all consortium members. WebALT Deliverable D1.1 45

EDC-22253-WEBALT

A.

Definitions of Actors and Abbreviations

In this appendix, definitions of used terms and abbreviations are given. First, the actors involved in an e-learning platform are defined, then the meaning of the abbreviations of concepts and terms used in this document which involve elearning platforms are given. Actors e-learning platform

Machine providing online education in the most general way. It offers effective learning components created by interaction with digitally delivered content, (learning) support and services. It facilitates and enhances learning by means of personal computers, CDROM’s, and the Internet. It means flexible learning and it makes distance learning possible. In this document we will also use the term platform, e-learning system or system instead of elearning platform.

Author

Person who creates learning material content or question content using the author account of the platform.

Editor

Person who decides whether content created by an author will be published to the public on the e-learning platform.

Guest

Person who is using a guest account of the platform to

try it out. Proctor

Person who supervises an examination such that students doing the examination using the platform cannot cheat.

Student

Person who is using a student account of the platform.

Teacher

Person who is a teacher/instructor of a certain course and is using the platform to the course.

Translator

Person or program who translates content created with the platform from one natural language to a different one.

(Server) Administrator Person who is using an administrator account of the platform e.g. to register students, control gradebooks etc.. University/School

Institution making use of the platform to support students and teachers. WebALT Deliverable D1.1

46

EDC-22253-WEBALT

Publisher

Institution making use of the platform to distribute learning material and questions in multiple languages.

Web service client

Software provided by the web and used by the e-learning platform.

Asset

Learning content represented in electronic form (e.g. Web page, text, media file).

Note that a person can behave as more than one actor at the same time. For example, a teacher can be an author and a student at the same time. Standards and Further Concepts For e-learning platforms several standards have been developed. These standards are sets of guidelines, specifications and agreements such that all platforms supporting such a standard can cooperate in a standardized way. First, we explicate the abbreviations of the most common set of standards used in elearning, then we explicate the abbreviations of further concepts we use in this document. Standards (ADL) SCORM

(Advanced Distributed Learning) Sharable Content Object Reference Model. This is a standard based on the work done by IMS, AICC, Ariadne and IEEE.

(IMS) QTI

(Information Management System) Question & Test Interoperability specification.

(IEEE) LOM

(Institute of Electronics and Electrical Engineers) Learning Object Metadata

Further Concepts XML

Extensible Markup Language

MathML

Mathematical Markup Language

NL

Natural Language

NLG

Natural Language Generation

CD

Content Dictionary

CCD

Course Content Dictionary

CAS

Computer Algebra System

SAT

Self Assessment Tool

VLE

Virtual Learning Environment

MLE

Managed Learning Environment

LMS

Learning Management System WebALT Deliverable D1.1

47

EDC-22253-WEBALT

CMS

Course Management System

LCMS

Learning Content Management System

ILS

Integrated Learning System

CBS

Computer Based Training

B.

Background Information Concerning Naming of eLearning Systems

We discuss some common used names of e-learning platforms in this section. We give explanations about the names used such that they can be placed in the right context. In Appendix A, we listed the abbreviations for the common used names of e-learning platforms. A Virtual Learning Environment (VLE) is a network-based software package designed to facilitate group as well as individual activities. They are considered to offer an integrated solution to managing on-line learning. They provide a delivery mechanism, student tracking facilities (where individual student participation in VLE activities and discussions can be logged) and (often) interactive assessment. They provide access opportunities to discuss, support and provide resources, which can be self-developed or professionally authored, stored locally or made available using the World Wide Web (www). A VLE consists of the components “Communication” and “Management” and usually also of the components “Learning material” and “Assessments”. An MLE refers to the whole range of information systems and components of a school or university (including its VLE if it has one) that contribute directly, or indirectly, to learning and the management of that learning (see Fig. 1). To clarify, a VLE focuses on learning and teaching and is usually a specific piece of software that can be a possible component of an MLE. An MLE is a conceptual term for a whole range of different software, systems and components that interrelate, share data and contribute to the management of the learner experience. By its very nature there is no one definition of an MLE; the tools, components and services bundled together depend on the institution's vision.

WebALT Deliverable D1.1

48

EDC-22253-WEBALT

VLE’s are also referred to as 'Course Management Systems' (CMS) or 'Learning Management Systems' (LMS). A Learning Content Management System (LCMS) is an LMS (or VLE) with a main focus on storing, creating and delivering personalised learning content (learning material and possible assessments). In the previous paragraphs we tried to clarify a subfield of e-learning systems. There seems to be the tendency however that the terms VLE, MLE, LMS and LCMS are often mixed up and therefore distinctions between the terms LCMS, LMS, MLE, VLE are not always clear. This means that in practice, the terms LCMS, LMS, MLE and VLE can all have the same meaning. In fact, there are many more terms representing the same thing such as Integrated Learning System (ILS) and Computer Based Learning (CBS). Because we will discuss general e-learning systems in this document, we will not use any of the confusing terms VLE, MLE, etc., but we will always talk about elearning platforms, e-learning systems or simply platforms or systems. We define a Self Assessment Tool (SAT) to be a platform that solely offers assessments to students. These platforms are used by students to practice for a certain course, or possibly to do a test set by the teacher. In that case, the SAT WebALT Deliverable D1.1

49

EDC-22253-WEBALT

is able to do automatic grading. A SAT therefore consists of the component “Assessment”, but possibly also of the component “Management”.

WebALT Deliverable D1.1

50

EDC-22253-WEBALT

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44.

Edutools, http://www.edutools.org/. Aim, http://maths.york.ac.uk/moodle/aiminfo/. Angel, http://www.angel.ac.uk/. Atutor, http://www.atutor.ca/. BlackBoard, http://www.blackboard.com/. Cable, http://www.cable.bham.ac.uk/. Calculus & Mathematica, http://www-cm.math.uiuc.edu/. Claroline, http://www.claroline.net/. Content-e, http://www.content-e.nl/. Cose, http://www.staffs.ac.uk/COSE/. Desire2Learn, http://www.desire2learn.com/welcome.html. Dmath, http://dmath.savonia-amk.fi/. eCollege, http://www.ecollege.com/indexflash.learn. Educator, http://www.ucompass.com. Helsinki Learning System, http://mark.math.helsinki.fi/HLS/. ilias, http://www.ilias.uni-koeln.de/. IntraLearn, http://www.intralearn.com/. Ischolar, http://www.ischolar.ca/. Jones e-education, http://www.jones.com/. LeActiveMath, http://www.leactivemath.org/. Learnwise, http://www.learnwise.com. Logicampus, http://www.logicampus.com/index.php/welcome. LON-CAPA, http://www.lon-capa.org/. Maple TA, http://www.maplesoft.com/. Math-kit, http://www.math-kit.de/. Metric, http://metric.ma.imperial.ac.uk/. Moodle, http://moodle.org/. Mumie, http://www.mumie.net/. Stack, http://eee595.bham.ac.uk/~stack/. Teknical Virtual Campus, http://www.teknical.com/Products/virtual_campus.htm. The Learning Manager, http://www.thelearningmanager.com/. MathDox, http://www.riaca.win.tue.nl/. Wallis, http://www.maths.ed.ac.uk/~wallis/. WebCT, http://www.webct.com/. WebTutor, http://www.webtutor.thomsonlearning.com/. Wiley's eGrade, http://www.wiley.com/. ADL, http://www.adlnet.org/. IMS, http://www.imsglobal.org/. Serving Mathematics, http://maths.york.ac.uk/serving_maths/. LaTeX, http://www.latex-project.org/. Hermes, http://www.aei.mpg.de/hermes/. LaTeXML, http://dlmf.nist.gov/LaTeXML/. MathML, http://www.w3.org/math/. Ausbrooks, R., et al., Mathematical Markup Language (MathML) Version 2.0 (second edition), D. Carlisle, et al., Editors. 2003, World Wide Web Consortium. WebALT Deliverable D1.1

51

EDC-22253-WEBALT

45. 46. 47. 48. 49. 50. 51. 52. 53. 54. 55. 56. 57. 58. 59. 60. 61.

62.

63. 64. 65. 66. 67. 68. 69. 70.

71.

Conversion Stylesheets for MathML, http://www.orcca.on.ca/MathML/software/mmlctop2_0.zip/. MathML software list, http://www.w3c.org/Math/Software/. OpenMath, http//www.openmath.org/. Buswell, S., et al., The Open Math Standard, Version 2.0. 2004, The Open Math Society. OMDoc, http://www.mathweb.org/omdoc/. Kohlhase, M. OMDoc: Towards an Internet Standard for the Administration, Distribution, and Teaching of Mathematical Knowledge. in AISC. 2000. Kohlhase, M., OMDoc: An Open Markup Format for Mathematical Documents (Version 1.2). 2005. Sutner, K., Converting Mathematica Notebooks to OMDoc. 2005. Maple, http://www.maplesoft.com/. Maxima, http://www.maxima.sourceforge.org/. Mathematica, http://www.wolfram.com/. Axiom, http://page.axiom-developer.org/. MuPAD, http://www.research.mupad.de/. Wiris, http://www.wiris.com/. Darwin, http://cbrg.inf.ethz.ch/Darwin/index.html/. Scilab, http://scilabsoft.inria.fr/. Libbrecht, P. Authoring Web Content in ActiveMath: From Developer Tools and Further. in Proceedings of the Second International Workshop on Authoring Adaptive and Adaptable Educational Hypermedia, AH-2004: Workshop Proceedings, Part II, CS-Report 04-19. 2004: Technische Universiteit Eindhoven. Libbrecht, P., et al. Integration of Mathematical Systems into the ActiveMath Learning Environment. in ISSAC-2001 Workshop on Internet Accessible Mathematical Computation. 2001. Serving Mathematics in a distributed e-learning environment. 2005. APIS, http://ford.ces.strath.ac.uk/APIS/. Assis, http://www.hull.ac.uk/esig/assis.html/. TIP, http://www.jisc.ac.uk/index.cfm?name=delettip/. Bodington.org, http://bodington.org. LAMS, http://www.lamsinternational.com/. TOIA, http://www.toia.ac.uk/. Aird, M.-L., W.B. Medina, and J. Padget. MONET: service discovery and composition for mathematical problems. in Proceedings of IEEE workshop on Agent-based Cluster and Grid Computing (at CCGrid 2003). 2003: IEEE Computer Society Press. Sakai, http://www.sakaiproject.org/cms/.

WebALT Deliverable D1.1

52