Experience with the GESTALT On-Line Learning Support System * Almerindo GrazianoIr2,Paolo Maresca', Stefan0 Russo'
' Dipartimento di Informatica e Sistemistica, Universiti di Napoli Federico 11, Via Claudio 2 1,80125 Napoli, Italy, { paomares,
[email protected] School of Computing and Management Sciences, Sheffield Hallam University, Sheffield, UK,
[email protected] Abstract
the availability of a huge amount of digital information at hand through the web. The Internet, the WWW and related technologies play a fundamental role in this emerging scenario. So called on-line learning initiatives, fostered by individual or federated institutions, are established in many countries, providing distance education opportunities. We believe that, from a technical point of view, crucial to the successful use of information technologies for distance education is the adoption of open standards and tools, and that these should interoperate to support a number of activities (ranging from digital courseware authoring to retrieval and delivery) accomplished by a variety of players involved in the educational process, including lecturers, tutors, students and staff. In this paper, we describe the on-going experience with the GESTALT learning support system at University of Naples. This work is part of the ACTS Project GESTALT (Getting Educational Systems Talking Across Leadingedge Technologies) of the EU 4* Framework. Programme [l]., Many pedagogical issues and problems are also clearly related to the use of information technologies for education. These are beyond the scope of the project and of this work in particular. The paper is structured as follows. Next section presents the GESTALT component-based architecture and its functionalities, aimed at supporting on-line discovery and delivery of multimedia educational resources on the web. Section 3 briefly describes the GESTALT metadata model underlying the discovery services, and its relationships to standards. Section 4 describes the GESTALT testbed and trials at the University of Naples, while in section 5 we comment on the experience made; finally, section 6 gives some concluding remarks.
With the spread of the Internet and the Web, higher education institutions are increasingly interested in exploiting modern standard information and communication technologies to provide learners and trainees with advanced services for tele-education. Within the European Union 4th Framework Programme, the project GESTALT has defined and implemented a flexible, component-based architecture aimed at supporting on-line discovery and delivery of multimedia educational resources on the web. In this paper, we describe the on-going experience with the GESTALT system at Universiv of Naples, and report first results of on-field trials during the first academic semester.
1. Introduction The on-going revolution leading to the so-called information society, driven by advances of computer and communication technologies, is also affecting the fields of higher education and vocational training. A number of forces push educational institutions like universities towards a shift in their business process model, from institution-centred (ie, built around the lecturer and the classroom) to learner-centred (built around the student, who is not anymore a merely passive learner). Examples of such forces are: the increasing need for permanent education and training, involving mature students, often employed, willing to gain further skills and qualifications; the need for distance learning, to reach learners that are remote, or disabled, or part-time students, and to allow asynchronous interaction between the tutor and the learners;
' This work has been supported by EU under Project AC367 GESTALT (Getting Educational Systems Talking Across Leading-edge Technologies) of the ACTS programme (Advanced Communication Technologies and Services), IV Framework Programme.
1089-6503/00 $10.00 0 2000 IEEE
86
2. The GESTALT architecture
3. The metadata model
The GESTALT functional model supports a number of educational actions, roles, and relationships in a common educational process, as, for instance, in an academic institution. Typical players in such a scenario are the educational service provider, the learners, the content provider (e.g., a digital publisher, o the educational institution itself), the course authors, the lecturers, the tutors, and the admin staff members. The GESTALT architecture thus consists of a set of modular components envisaged as capable of supporting the complete functional model. The basic components are : the web client: in the GESTALT vision, user services are delivered to the desktop using the WWW technology; the Learning Environment (LE): this is the core component that, utilising an internal database, provides on-line support for the typical actions performed by the several players involved in the educational process: curricula definition and course construction, students enrolment, delivery of learning resources to the desktop, tutor’s tracking of individual student progress; the Asset Management System: controls access to valued resources, learning objects (i.e. course/module data) which should only be accessed within a paid for programme of study, the Administration Management Information Services: supports back-of-house management of the educational/training institution. One additional component of the functional model, usually not present in distance learning systems, is the Broker. This is an on-line entity that supplies information about learning objects to users (learners, course authors, etc.) wishing to discover them. The two related innovative components provided in the GESTALT architecture are: the Resource Discovery Service (RDS): it allows Web users to explore what courses and modules are available from which institutions; the RDS implementation is based on the CORBA distributed objects technology; the User Profiles service: an LDAP-based directory service used to store user preferences for LE interaction (e.g. configuration options), RDS settings (e.g. frequently searched hosts/services), and for smart searches. Within the project, big emphasis has been put on the adoption of an open and standard data model underlying the overall architecture, the RDS and user profile services in particular. Emerging standards in the metadata field have been explored and enhanced for this purpose. This is described in the next section.
One of the problems of modern education is the overwhelming increase of information being constantly produced and made available in new forms through the WWW in various digital formats. The pace is such that information is difficult to classify and some of it will never be accessed by many potential learners. A vast amount of data on the World Wide Web doesn’t even hold classification information, which counts for difficult and often imprecise searches. A number of initiatives, see [I31 and [14] are trying to address the need for new and standardized cataloging systems, which will enable to share and access large amounts of digital resources across institutions. All the initiatives use a set metadata for the description of the resources. Some, [16], also propose enhancements to the use of metadata. Metadata are data about data. They provide a means of describing a general type of data in a structured fashion for use by many diverse applications. In the educational field, like in many others, metadata facilitate searchability, extensibility, re-usability, and scalability. In an open scenario, essential is the possibility of querying and exchanging metadata sets between heterogeneous systems. Work in GESTALT has lead to identifying three key areas where the development of a standard metadata model could allow achieving the above goals. The complete GESTALT data model comprises the following components: - Courseware Content (GEMSTONES); - Student Profiling/Tracking (PAPVEPAPI); - Curriculum Management (UOM). The design of the three data models has drawn upon the work of a number of standardization activities in the field of metadata for education. Attention has focused on the work done under the IEEE LTSC [3], the IMS project [4], the ARIADNE project [51 and the Dublin Core [ 6 ] . Although not an educational initiative, the Dublin Core aims at defining a set of cross-domain metadata to enhance the searchability of electronic resources, and as such it has been the foundation onto which other domainspecific initiatives have been developed. The Dublin Core has recently formed the DC Education work-ing group and the first working draft is expected by end of February 2000.The final GESTALT data model is based on the IEEE LOM v2.5 adopted by both IMS and ARIADNE. XML’s DTD has been adopted for the binding of all three data models and XML has then been used for the creation of the metadata. The first data model is GEMSTONES (Gestalt Extensions to Metadata STandards for ON-line Education Systems). It extends the LOM model, which uses just
87
’
eight categories to group the various metadata fields, by adding three additional categories (shown in italics): - General - Lifecycle - Meta-metadata - Technical - Educational - Rights Management - Relation - Annotation - Assessment - QoS - Mappings Figures l a and Ib show the GEMSTONES DTD as implemented with the Microstar’s Near & Far Designer.
GEMSTONE
L GEMSTONE
Fig. 1 b: The GESTALT metadata model (cont.)
The Assessment category was introduced to describe assessment-type learning resources; it is based on the initial IMS model and it has been modified to bring it in line with the LOM model. The IMS has recently formed a Question and Test team (Q&T) with the scope of developing a thorough Assessment data model [2]. The Quality of Service (QoS) category (bottom of Fig. Ib) was introduced to map a learning resource requirements to the capabilities of the networking technologies andor services necessary for its delivery. Work has been undertaken at University of Naples to define this category based on the IETF Integrated Services/ Differentiated Services models, and on the basis of the experiments of QoS delivery of audiolvideo learning resources (described in section 4). An important part of the GESTALT project has been to demonstrate the effectiveness of “smart” types of Internet searches when combining the student profile with the metadata used for the description of the resources. When using ordinary Internet search engines, a search
GEMSTONE
Fig. la: The GESTALT metadata model.
88
regards a more dynamic type of information necessary for tracking student progression in hisher learning experience. Typical information includes the program of study undertaken by the student, the organization she is currently studying at, marks scored etc. The EPAPI data model is use by the Learning Environment for tracking student progress and enrolment information. The third and last data model (Fig. 2) is the Unit Object Model (UOM), used between the Learning Environment and the Management Information System (MIS) for curriculum management. Typical information in the UOM includes name of the unit, schedule for the unit, whether it is compulsory or not, etc. Figure 3 shows the relationships between all the above data models and the components of the GESTALT architecture.
with a given keyword, say C++, will return thousands of hits. Typically, only a small subset of the hits will be relevant for the end student who, for instance, only speaks English, is looking for an on-line C++ course and has certain preferences over the media format the course is offered with, e.g. just text based material. By combining the relevant keywords with the additional information stored in the student profile, the search results will be more accurate and also less numerous; this allows actual manageability. However, equivalent information within the learning resources and the student profile will probably be represented by different element names. The Mapping category (bottom of Fig. lb) is then necessary to map between different element names across the two data models. In the GESTALT data model we only mapped the student profile’s preferences (PAPI based) onto the resources metadata elements’ name. The mapping category can be similarly extended to allow searchability over different data models such as the Dublin Core (DC), provided that the DC-equivalent element is found in the GEMSTONES structure. The Student Profile data model draws upon the PAPI work by Farance and Schoening [7], who aims at facilitating the “interchange and interoperability of student records, i.e. portable student records”. The GESTALT data model extends this work by envisaging two sub-models; the first, based on the PAPI work, allows storing more static information about the student. It includes learner information (Personal category), learner accomplishments (Portfolio category), learner history (Performance category) and learning styles (Preference category). In the GESTALT architecture these information are stored in an LDAP directory server and
3 ld-
L
Unit
-f
+%Scheduleschema
>Schedule
f+
EPAPI
Client
Content
Fig. 3: The GESTALT architecture and data models. Similar work to GESTALT, with regard to intelligent searches of educational resources combined with user definable profiles, has been undertaken under the Courseware Description (CDL) initiative[ 111. Some
0
7 startoate He] 7 E n a D a w
7 Id
- I1
+* Occurrence t
Fig. 2: The Unit Object model metadata category are accessed by the RDS for the “smart” searches. The second sub-model, called EPAPI (Extended PAPI)
other projects have also worked extensively to develop educational applications of metadata. The vast majority of
89
data models and applications are mainly based on the DC data model. Examples include the Gateway to Educational Materials (GEM) [IO], the Education Network Australia (EdNA) [ 181, the Virtual European School (VES) [ 191, and the European Schoolnet (EUN) [20]. Numerous are finally the number of metadatastandardization related initiatives and projects currently running [ 151. As for GEMSTONES, the main choice for implementing the data models has been the XML DTD. A major work is currently being done by the XML Schema Working Group [I71 within the W3C to develop alternative and more powerful binding mechanisms, which will allow for data typing and other important features, see [12]. The EASEL project [21], on which the authors are working at the moment, and which builds upon the results of the GESTALT project, is considering adopting an XML SCHEMA in place of the DTD.
U
U
Fig. 4: The GESTALT trials platform at UoN The first class of trial scenarios concerns the search activities traditionally performed by students or course authors, seeking a particular educational resource, generally through a “standard” web search. In a normal scenario, a user, be (s)he a student or a professional navigating the web looking for an educational resource, or a course author looking for a resource for building his own course’s set of resources, performs the search by means of currently available Internet information retrieval tools, getting a long list of often un-related results. With the GESTALT brokerage services, based on the RDS, the search is performed on a distributed database of metadata descriptions of educational resources, provided by the resource authors. Students and lecturers at UoN experiment the effectiveness of the search looking for learning material related to the topics of the F12 course; the educational materials are mainly borrowed from the previous EU RENAISSANCE project, with additional multimedia material made available to the project for a richer set of trials. The metadata accompanying the resources have been developed with the technologies (XML) and in the format (GEMSTONES) that were defined in the previous phases of the project. A variation of this search scenario is based on the user’s profile stored on the LDAP server, used to refine the search. The second category of trial scenarios concerns activities in the course delivery phase. They are centred around the use of the LE and MIS. A tutor uses it to define the structure of the course F12 and link educational resources, available in a variety of electronic formats, and also to track students’ progress; a selected number of students of the F12 course use the LE to access courseware material interactively and interact with the tutor asynchronously and remotely. A member of staff cooperates with the tutor for trialling processing of students applications and their enrolment. Fig. 5 (left) shows a snapshot of the LE web page with a hypertextual course on C++ programming delivered on-line.
4. The GESTALT user trials The GESTALT user trials take place in the current semester at University of Naples Federico I1 (UoN), and they involve both members of staff and students, representative of the different roles played by the users of the GESTALT architecture: lecturers (professors and assistant professors), acting either as .tutors, or =course authors (or both); 0 a cohort of students; 0 a staff member, acting as administrator for student applications and enrolment. The trials platform (Fig. 4) consists of a number of client computers and two server computers, installed in one of the laboratories of the Department of Computer Science (DIS), running the various software components providing the GESTALT architecture services: Resource Discovery Service (RDS), Learning Environment (LE), Management Information Services (MIS), Directory Service (LDAP), etc. These servers are accessed by the students from client computers in a different remote laboratory, and by the tutors from their offices. The connections are provided by the campus network. User access through the Internet is supported too at the addresses http://servgest.grid.unina.it. The trials are centred around the courses of Fondamenti di Informatica I (Fundamentals of Computer Science I, FII), and chiefly of Fondamenti di Informatica I1 (FI2), belonging to the curriculum “Computer Engineering” at the Faculty of Engineering. The latter course runs over the first semester of the academic year, from September to January.
90
educational activities centred around the LE, and the digital delivery of courseware. As for the RDS retrieval support system, the user tests performed covered mainly functional aspects. About 80 metadata descriptions were built for learning resources and inserted in the search database of the RDS. The results confirm the high potential usefulness of the service, especially in its interoperability with the LEbased delivery service. However, the tests do not allow to fully assess its effectiveness. Further tests would be required, with a larger metadata sets and in a real-world scenario, to definitely assess the benefits of the “smart” searches with respect to traditional web searches. To the aim of the trials, the tutors compared the use of the FIl and FI2 existing web sites with the LE-based approach. If on the one hand, appropriate training was required that allowed tutors to use the LE in all its functionalities, on the other hand the tutors felt somewhat lightened from their traditional cumbersome Web management duties. Although competent in the Web technologies, a tutor can often find these duties overwhelming and difficult to cope with. The functionalities offered by the LE proved to be advantageous over the traditional “web-siting” of course notes and information. Very useful was the student tracking capability. The trials with students have been followed by a system usability assessment phase, based on the methodology and tools developed and provided by project partners at the Trinity College Dublin [9]. Students and tutors had access to a web site at TCD to fill user
The third category of user trials concerns the delivery of multimedia courseware with QoS guarantee. These trials aim at showing how metadata for resources can be exploited, along with LDAP user profiles, to provide the information necessary for multimedia applications to be able to negotiate session parameters on networks with QoS capabilities. The multimedia resources used in these trials are prerecorded videoclips of lectures from a distance learning course (provided by the italian NETTUNO distance learning consortium [SI) covering some of the topics of the F12 course. The lectures (initially in VHS format) have been digitalized and produced in standard digital audiohideo formats so as to be delivered on-line through the Web; a characterization of these videoclips has also been produced, in terms of the network resources (e.g. bandwidth, peak rate) required for quality on-line delivery. The DiVA video-on-demand prototype developed at DIS is used for these trials on a controlled QoS network testbed in the DIS laboratory. Fig. 5 (bottom) is a snapshot of the client desktop with the LE window and DiVA client window playing a recorded lecture.
5. Results The user trials involve one professor and one assistant professor (co-authors of this paper), also acting as tutors for the trials, one admin staff member, and a subset of about twenty students of the two selected institutional academic courses. The preliminary results of the
Fig. 5: snapshot of the student’s view o f the LE with a hypertextual (left) and a multimedia (right) learning resource experience made allow to draw comments on three key questionnaires and then analyse results, respectively. aspects of the overall business process envisaged in Questions aim at assessing usability aspects, such as GESTALT: the learning resource search service, the friendliness and layout consistency of human interfaces of electronic instructional materials.
91
network (QoS) protocols, can be integrated to provide flexible support for educational activities in an open scenario. The project succeeded in this respect, and it was judged just one point off the maximum possible score by the auditors. The trials and the results presented in this paper have confirmed this approach and encourage to further explore these issues.
More specifically, 5 criteria are considered by the assessment methodology: A) naturalness, B) user support, C) consistency, D) non-redundancy, E) flexibility. A sample question for criterion D is: “Is there an adequate amount of instructional. material on each screen?”. Preliminary results based on the analysis of the questionnaires on the courseware material delivered through the LE are summarised in Table I.
Table I: Results of the usability assessment
Acknowledgements The work presented in this paper is part of the ECfunded GESTALT project, which has seen the participation of research groups in several academic and industrial partners, namely: Fretwell-Downing Education (UK, group leader K. Riley, also manager of the project), British Telecom (UK, P. Foster), Kyros (GR, D. Maroulis and R. Smith), Trinity College Dublin (IRL, V. Wade), University of the Agean (GR, J. Dartzentas), University of Naples (I, S. Russo and G. Ventre) and Waterford Institute of Technology (IRL, W. Donnelly).
Agree
68,7
1 E
01
0
16,7
83,3
0
Finally, as for the QoS guaranteed delivery of digital resources, the trials on the UoN laboratory testbed demonstrated the effectiveness of the approach of integrating metadata and user profiles technologies with multimedia applications and network protocols. The cost for this is that of characterising the multimedia resource in terms of (a limited set of) parameters expressing the requirements on the network (and possibly on the client computer), to be negotiated with QoS-aware network equipment. The more precise the media characterisation, the more effective the usage of network resources can be. This characterisation is probably to be considered an authoring issue, but it is unlikely to be left up to the content authors, since it requires advanced technical skills. One further result of the QoS experiments has been the refinement of the QoS schema within the GEMSTONE metadata model. The detailed description of this work is beyond the scope of this paper; it is reported in reference
References [ I ] The GESTALT Project,htto://www.fdgrouo.co.uk/gestalt/ [2] P. Foster et al, The GESTALT project, Courseware Metadata Design (GEMSTONES), Project Deliverable D401, h ttp://www.fdgroup.co.uk/gestalt/D0401 .pdf [3] IEEE Learning Technologies Standards Commi ttee (LTSC), h ttp://grouper.ieee.org/groups/ltsc/
[4] The Instructional Management Systems (IMS) Project, http://www.imsproject.org/ [ 51 ARIADNE Educational Metadata Recommendation Summary, Working document, Version 2.0, April 1998 [6] Dublin Core Metadata for Resource Discovery. IETF RFC 2413 [7] F. Farance, J. Schoening, PAP1 - Personal And Private Information.http://edutool.com/papi/ [8] The NEITUNO Distance University Network, http://l95.62.32.250/main.html [9] V. Wade, M. Lyng, Experience Improving WWW based
PI.
Courseware through Usability studies, AACE World Conference on Educational Multimedia and Hypermedia & World Conference on Educational Telecommunications 99, Boston, 1999. [ I O ] Stuart A. Sutton, Gateway to Educational Materials (GEM): metadata for networked information discovery and retrieval, Computer Networks and ISDN Systems, Volume 30, h u e s 1-7, April 1998, pp. 691-693 [l I ] L. Jay Wantz, M. Miller, Educational metadata in use. Computer Networks and ISDN Systems, Volume 30, Issues 1-7, April 1998, pp. 701-703 [I21 D. Ioannides, XML schema languages: beyond DTD. Library Hi Tech, Volume 18, Number 1, pp 9-14.
6. Conclusions In this paper we have given an overview of the architecture and data model of the GESTALT system for supporting on-line retrieval, management and delivery of learning resources, and we have described the experience done at UoN setting up and using the system with a subset of students of two institutional courses of Computer Engineering . One of the main achievements of the project has been to show how a number of different technologies, including web, brokerage, metadata, digital authoring and
92
[I31 D. Domer, Cataloging in the 21st century - part2: digitization and information standards. Library Collections, Acquisitions, & Technical Services, Volume 24, lsue 1, Spring 2000, pp. 73-87 [ 141 J. Darzentas, Sharing metadata: enabling online information provision. OCLC Systems & Services, Volume 15, number 4,1999, pp. 172-178. [I51 J. Milstead, S . Feldman, Metadata: Cataloging by Any Other Name. ONLINE, January 1999 [I61 M. Marchiori, The limits of Web metadata, and beyond. Computer Networks and ISDN Systems, Volume 30, Issues 1-7, April 1998, pp. 1-9 [ 171XML Schema Working Group. htt~://www.w3.org [ 181Education Network Australia. htto://www.edna.edu.au [ 191Virtual European School. htto://www.ves.eu.orn/home.htm [20] European Schoolnet. htto://www.en.eun.ordfront/actual/index. html I211 Educator Access to Services in the Electronic Landscape (EASEL - IST Project 10051). htta://www.fdnrouo.co.uk/easel/
93
