ONTOLOGY-BASED LEARNING APPLICATIONS: A DEVELOPMENT METHODOLOGY Dimitris Kanellopoulos, Sotiris Kotsiantis, Panayiotis Pintelas Educational Software Development Laboratory (ESDLab) Department of Mathematics, University of Patras, Patras GR 26500, Greece {
[email protected],
[email protected],
[email protected]}
identifies key-concepts in the context; thus, automatically building a thesaurus. It can also be edited and refined as needed, allowing insertions of new themes, new relations and new terms. Ontologies are created using ontology editors, such as Protégé 2000 [6]. Protégé is a Java-based ontology editor with OWL Plugin, and thus it allows ontology implementation as an applet on the Web and permits multiple users to share the ontology. Duineveld et al. [7] describe and compare various ontologydevelopment environments. Other tools such as Chimaera [8] provide diagnostic tools for analyzing ontologies. For example, the analysis that Chimaera performs includes both a check for logical correctness of an ontology and diagnostics of common ontology-design errors.
ABSTRACT Current learning standards (e.g. IEEE LOM, IMS, SCORM, CanCore) trying to adopt Semantic Web technologies to conquer web-based learning contexts. In this paper we propose a methodology for developing learning ontologies, and discuss the role of the Semantic Web in educational or learning systems. We also provide a brief overview of learning ontology, its need, development tools and process. In addition we provide representative examples of ontology-based applications or projects in education. KEY WORDS Ontology, semantic web, e-learning
Learning-independent applications, problem-solving methods, and software agents can use learning ontologies and knowledge bases (built from these ontologies) as data. Learning ontologies are developed with the aid of ontology development languages and tools, in order to: • Share common understanding of the structure of learning information among humans or software agents. For example, if several different learning web sites share and publish the same underlying ontology of the terms they all use, then software agents can extract and aggregate information from these websites. The agents can use this aggregated information to answer user queries or as input to other educational applications; • Enable reuse of domain knowledge. For example, if one group of researchers has developed common learning ontologies, others can simply reuse it for their domains. Libraries of reusable ontologies are provided on the web (e.g. the Ontolingua ontology library: http://www/ksl.stanford.edu/software /ontolingua/, or the DAML ontology library: http://www.daml.org/ontologies/) • Make domain assumptions explicit. Explicit specifications of domain knowledge are useful for new users who must learn what terms in the domain mean. • Separating the domain knowledge from the operational knowledge. We can describe a task of configuring a learning material from its components according to a required specification and implement a
1. Introduction The Semantic Web enables better machine processing of information on the Web, by structuring web documents in such a way that they become understandable by machines [1]. The Semantic Web framework includes major components such as: ontologies, ontology languages, tools, semantic annotations, logical support, intelligent agents, and applications/services. Educational semantic web will influence the next generation of e-learning systems [2]. Henze et al. [3] proposed a framework for personalized e-learning and showed how the semantic web resource description formats can be utilized for automatic generation of hypermedia structures. They investigate a logic-based approach to educational hypermedia using TRIPLE (a rule-based query language for the Semantic Web). Specific learning ontologies can be used in order to develop intelligent learning information systems. A learning ontology contains knowledge for developing intelligent learning information systems. It is an explicit formal specification of how to represent the learning objects, learning concepts (classes) and other entities, and the relationships among them [4]. The properties of each concept describing various features and attributes are called slots, while the restrictions on slots are called facets. An ontology together with a set of individual instances of classes constitutes a knowledge base [5]. A learning ontology is a domain ontology that describes the learning terms and the relationships between them. It provides a clear definition of each term used and 518-812
27
•
refers to representing the ontology in a formal language. First, in the ontology the main classes are entered as concepts, followed by their attributes and slots. Classes describe learning concepts. For example, a class of books represents all books. Specific books are instances of this class. A class can have subclasses that represent concepts that are more specific than the superclass. For example, we can divide the class of all books into Mathematics, Physics,… and Medicine books. It is remarkable that if a class A is a superclass of class B, then every instance of B is also an instance of A. It is advisable to build an initial small ontology of classes and slots. Suitable tools to query the ontology are also required. Ontology development is an iterative process; it involves developing a preliminary ontology that is refined with time. In practical terms, coding an ontology includes: a) Defining classes, b) Arranging the classes in a taxonomic hierarchy (subclass-superclass). Particularly, there are three possible approaches in developing a class hierarchy: 1) a top-down development process where we start with the definition of the most general concepts and subsequent specialization of the concepts. 2) a bottomup development process, in which we start with the definition of the most specific classes (the leaves of the hierarchy) and subsequent grouping of these classes into more general concepts. 3) a combination development process in which we have a combination of the top-down and bottom-up approaches. c) Defining slots and described allowed values for these slots, and d) Filling in the values for slots for instances.
program that does this configuration independent of the learning material and components themselves. Analyse domain knowledge. Formal analysis of learning terms is valuable, when both attempting to reuse existing ontologies and extending them.
This paper is organized as follows. Section 2 proposes a methodology for developing ontologies for learning systems. Section 3 presents ontology-based applications in education and the last section presents some thoughts for future research.
2. The methodology for developing ontologies for learning systems An interesting guide to create your first ontology is given in [9]. There is no one methodology to model learning contexts – there are always viable alternatives. Hereafter, we offer one possible process for developing a learning ontology that we found useful in our ontologydevelopment experience. The proposed methodology for developing learning ontology includes six steps: 1) identifying the purpose, 2) ontology capture, 3) coding, 4) refinement, 5) testing and 6) maintenance. •
Identify the purpose to develop the ontology. At this step, we answer three questions: i) why is the ontology being built? ii) what is its intended use? iii) who are its users? Additionally we plan for the application and expected uses of the knowledge base. This usually means working with domain experts that have a set of problems that could be solved with knowledge-base technology.
•
Ontology capture mechanism: It consists of three different stages: i) determining the scope of the ontology; ii) selecting a method to capture the ontology and iii) defining the concepts in the ontology. Determining the scope involves identifying all the key concepts and relationships. This can be achieved by sketching a list of questions (named competency questions) that a knowledge base based on the ontology should be able to answer. The proposed method used for ontology capture is similar to that used in Object Oriented (OO) Analysis and Design. However, OO programming centers primarily around methods on classes (viz. a programmer makes design decisions based on the operational properties of a class). On the contrary, an ontology designer makes these decisions based on the structural properties of a class. The process of defining concepts in ontology is also called categorization, which involves taking closely related terms and grouping them as concepts or categories.
•
Then, we can create a knowledge base by defining individual instances of these classes filling in specific slot value information and additional slot restrictions. When the ontology has been built, we can use forms to enter instances into the ontology. For example, Protégé 2000 automatically generates forms in its role as a knowledge-acquisition tool generator. •
Coding the ontology: A suitable ontology editor is selected based on the requirements of the domain and the functionality of the learning ontology. Coding
28
Refinement: It consists of two phases: a) intra-coding refinement, and b) extra-coding refinement. Intracoding refinement involves the refinement done during the coding phase. As the code is being developed, if either some errors are discovered or new requirements come up, the code is refined to correct the errors or fulfil the new requirements. Extra-coding refinement refers to the changes done to overcome the errors that are uncovered during testing, and enhancements carried out during maintenance. Forms can be customized to form a refined knowledge-acquisition tool; while doing this,
•
•
further design problems in the original ontology may surface. Testing: It uncovers defects in functional logic and implementation, and is carried out at all stages of development. Once the knowledge base has been created, end-user tests should be carried out to uncover defects in the ontology and/or the knowledge acquisition tool. Depending upon the problems encountered, appropriate changes need to be carried out to the ontology – and, at times, to the knowledge acquisition process and tools – to overcome any shortcomings. In addition we test the full application with end-users. This step can lead to further revisions to the ontology and the knowledge-acquisition forms.
These maps include concepts and relations among concepts. Conzilla is a concept browser that allows the user to navigate through a space of context maps to access associated content [13]. While the context maps are not referred to as ontologies by the authors of Conzilla, they may be regarded as equivalent. Conzilla is also being developed as part of the PADLR project as a means of accessing and annotating learning objects. Edutella (http://edutella.jxta.org): This project provides an infrastructure for Peer-to-Peer (P2P) systems for exchanging educational resources. Edutella uses metadata based on standards such as IEEE LOM to describe resources. Edutella provides a query service and a range of services for dealing with different meta-data sources and vocabularies. Querying in Edutella is via the Conzilla query interface. Semantics is captured using a mixture of RDFS, DAML+OIL and Datalog rules [14].
Maintenance: It can be corrective, adaptive or perfective [10]. Corrective maintenance involves considering the problems faced by the users while querying the ontology and correcting the ontology to overcome these problems. Adaptive maintenance involves modifying the ontology to fulfil new requirements in the future. Perfective maintenance involves improving the ontology, to further refine it.
Elena (http://www.elena-project.org): This project aims to create Smart Spaces for Learning, which are defined as “educational service mediators” that allow the consumption of heterogeneous learning services via assessment tools, learning management systems, educational (meta) repositories and live delivery systems. The project focuses on personalization and provides the infrastructure that matches learner needs with available resources and services. Elena’s educational web services are primarily learning services, which range from assessment to short lectures, courses and degree programmes. They also include supplementary services such as brokering services, learner assessment services, service evaluation services and learning service provider reputation services. Elena comprises the Edutella P2P infrastructure, a Learning Management Network (which includes the learning service providers and reputation services) and a Smart Space for Learning which includes a number of Personal Learning Assistants. These assistants use learner profiles to search for, select and negotiate with suitable learning services [15].
3. Ontology-based education applications In this section the role of ontologies and/or the Semantic Web in educational or learning systems is discussed. The following applications are explicitly based on ontologies and standards that have an important role in the representation of learning objects and repositories. CIPHER (http://www.cipherweb.org): This project supports the exploration of national and regional heritage resources. This is accomplished by supporting online Cultural Heritage Forums (CHFs) where a community (focussed on a specific theme or interest) can browse or construct narratives relating to the theme or interest [11]. Tools associated with this project include the Apollo Knowledge Editor (http://apollo.open.ac.uk) and the RAT (Resource Annotation Tool: http://rat.open.ac.uk).
EML (Educational Modelling Language) (http://eml.ou.nl/introduction/explanation.htm): It is a notational system developed at the Open University of the Netherlands as a means of representing: (a) the content of a unit of study (e.g., texts, tasks, assignments) and (b) “the roles, relations, interactions and activities of students and teachers”. Therefore, EML goes beyond standards such as IEEE LOM to model the social context of education. It is intended to capture any type of pedagogic strategy including, for example, competence based approaches and problem based learning. It now forms the basis for the IMS Learning Design Specification. As with many XML based approaches ontologies are not mentioned. However, the learning, unit of study, domain, and learning theory models which form the pedagogic meta-model can be construed as a set of ontologies [16].
Connexions (http://cnx.rice.edu): It is an open source project that provides modules (equivalent to learning objects), a repository, a markup language, and a set of tools for authoring, composing modules into courses and navigating through these courses. Special emphasis is placed on the community of authors [12]. The markup language (cnxML) captures essential aspects of modules such as type (e.g. example, proof, problem, solution) and metadata (such as author, maintainer, abstract, objectives). XML Crosswalks (or mappings) will be provided to provide access to material marked up with other standard schemes such as MathML, IMS, SCORM. Conzilla (http://www.conzilla.org/): KTH group is working towards the creation of the Conceptual Web (viz. a layer above the Semantic Web intended to make it more accessible to humans using graphical context maps). 29
climate models as well as to the rich literature on climate modelling and climate change [20].
Garden of Knowledge: Developed at the Royal Institute for Technology (KTH) in Sweden, the Garden of Knowledge is a learning environment “for keeping track of the interrelated structure of ideas, designed to support the expression of their relations to other ideas as well as their evolution over time and culture”. Underlying the project is the notion of a Knowledge Manifold or collection of personal idea spaces (Knowledge Patches) through which individuals experience the world. Learners can add content in the form of knowledge components (units conforming to international standards such as IMS) which can be downloaded into knowledge patches. The conceptual structures (context maps) of knowledge in these patches/components are represented in ULM (Unified Language Modelling, a notation based on the Unified Modelling Language) [17].
PADLR: The Personalized Access to Distributed Learning Repositories (PADLR) project aims to produce a distributed learning web infrastructure. This involves Peer-to-Peer learning resource networks (Edutella is used here), learning environments and personalized courselets. Apart from Edutella, another module in the project is the Courseware Watchdog that crawls the web for resources. Courseware Watchdog is also able to cluster resources and provides browsing and visualization tools for the resources it finds and clusters it creates. Finally, it has a mechanism for updating its ontology as new resources become available ([21], [22]). POOL: The Portal for Online Objects in Learning (POOL) Project is a Canadian project using the CanCore metadata scheme to produce learning object repositories [23]. SPLASH (http://www.edusplash.net) is an application for creating metadata for learning objects, as well as storing and accessing these. PONDS are larger repositories associated with communities. An important aspect of this project is its recognition that there should be two levels of metadata: one of these is generic but with fewer fields than standards such as IEEE LOM, the other is constructed by communities to meet community needs.
GESTALT (http://www.fdgroup.co.uk/gestalt/): This project provides a Resource Discovery Service (RDS) that provides access for learners to educational resources. This takes places as a three-step process: (1) the broker accepts a query; (2) it searches for resources using metadata and optionally learner profiles; and (3) it delivers these to the user [18]. KTH Metadata Tools: SCAM (http://kmr.nada.kth.se/scam/) is a content archive management system that uses RDF for its metadata that supports standards such as Dublin Core, IEEE LOM and IMS. SHAME is a framework for building metadata editors. There are demonstration editors for LOM and Dublin Core. It can also be used to build query systems for RDF metadata user [19].
RichODL (http://rich-odl.open.ac.uk): It is a learning web-based environment developed at the Knowledge Media Institute (KMI) at the UK’s Open University user It is intended for the training of students and practitioners in the modelling and simulation of dynamic systems [24]. RichODL makes use of KMI knowledge technologies to provide a means of indexing and searching for solved examples with accompanying text. It incorporates a discussion space system (D3E). Ontologies are used to describe the physical domain of the modelled systems.
Magpie (http://kmi.open.ac.uk/projects/magpie/): Magpie is a generic semantic web browser, but it is originated as a means of assisting in sense making for participants in the Climateprediction.net experiment. This experiment (like the Seti@home project) makes use of the distributed computing resources of thousands of home computers, in this case, to run different versions of a climate model. Magpie provides access (via a contextual menu) to complementary sources of knowledge, which can be used in contextualizing and interpreting the knowledge in a Web page. This is done by automatically associating a semantic layer to a web page. This layer depends on one of a number of ontologies, which the user can select. When an ontology is selected, the user can also decide which classes are to be highlighted on the web page. Clicking on a highlighted item (i.e., an instance of a class from the selected ontology) gives access to a number of semantic services. For instance, the ontology could contain the class ‘Project’. Clicking on an instance of this class would provide access to Project details, Research Areas, Publications, Resulting Technologies, Members, Shared Research Areas and project Web Page. In the Climateprediction.net project access is to material which will help to make sense of statistical analyses of complex
ScholOnto (Scholarly Ontologies Project) (http://kmi.open.ac.uk/projects/scholonto): To support disagreement and conflicting perspectives in academic research fields, we need tools that support the user in making sense of the relations among documents. The ScholOnto project is developing an ontology-based digital library server to support scholarly interpretation and discourse [25]. Researchers can articulate their view of where a document fits in the ongoing academic conversation thus creating a semantic network o scholarly discourse. ClaiMaker (a tool) has been developed to model the rhetorical relations (proves, refutes, is consistent with, is analogous to, and so on) among claims in research papers, publish these on a server and make queries about the relations and the documents containing them. While not intended primarily as a learning tool, both the access to a web of inter-related scholarly papers and the opportunity to add further annotations (i.e., extend the research paper semantic web) have educational and well as research applications.
30
SCULPTEUR (http://www.sculpteurweb.org/): It provides the tools for creating, managing, visualizing and learning from cultural heritage collections. It uses semantic web technologies to query and navigate cultural objects by creating a semantic layer that integrates metadata. Cultural Learning Objects comprising 3D models of cultural objects, and other learning material, can be created and displayed using Web browsers and stored using LOM and SCORM in a repository. The project is considering the use of the CIDOC Conceptual Reference Model (CRM) as its main ontology [26].
4. Conclusion Learning systems are facing rapid changes with the advent of semantic web technologies, and intelligent learning applications become possible with the development of ontologies. There is now a need for developing an infrastructure to manage the learning information and deliver to e-learners what they want. Semantic web and ontology-based intelligent learning information systems is one of the solutions. Ontologies are becoming popular largely because of what they promise: a shared and common understanding that reaches across people and application systems. Undoubtedly, there is no single methodology for developing a learning ontology. Ontology design is a creative process and no two learning ontologies, designed by different people, would be the same. Learning ontology design choices are affected by the potential applications of the ontology and the designer’s understanding and view of the domain. In this paper we proposed a methodology for developing learning ontologies and presented ideas that we found useful in our ontology-development experience. In addition, we shortly presented a number of ontology-based intelligent learning information systems found in the literature. These projects will give a novel potential to current e-learning research.
Stojanovic, Staab, and Studer’s E-learning and the Semantic Web Scenario Stojanovic et al. [27] illustrate how the Semantic web could be used to implement an E-learning scenario. Ontologies can be use to describe: (1) the content of learning materials, (2) the pedagogical context (such as introduction, analysis, discussion), and (3) the structure (the overall set of relations among parts of a course such as previous, next, is -part-of, references and so on). This three-fold ontology can be used to personalize access to learning materials. Trellis: It is an environment that allows end-user to record/annotate relationships among document fragments. For example, the statement “The unemployment percentage in Athens is 23%” may have been taken from a complete web document. Statements can also be conclusions, observations, summaries or hypotheses created by the user [28]. Units are several statements related by constructs. Therefore, a unit could be the argument that “Migrants may not come to Athens as the unemployment percentage is 23% and a maximum percentage of 20% is acceptable”. All of these statements may be associated with URIs. Therefore, Trellis provides a means of adding user-oriented structure to the Web. The vocabulary for annotation represents a statement or discourse ontology, which can be extended by the user. While Trellis is not intended for learning, it is described as a knowledge acquisition tool, which allows users to add knowledge (the structures) as they analyze something. Browsing these analyses would be useful for learners, as would the construction of units.
References [1] T. Berners-Lee, J. Hendler, & O. Lassila, The Semantic Web, Scientific American, 279(5), 2001, 34-43. [2] L. Aroyo, & D. Dicheva, The New Challenges for Elearning: The Educational Semantic Web, Educational Technology & Society, 7(4), 2004, 59-69. [3] N. Henze, P. Dolog, & W. Nejdl, Reasoning and Ontologies for Personalized E-Learning in the Semantic Web, Educational Technology & Society, 7(4), 2004, 8297. [4] M. Uschold & M. Gruninger, Ontologies: Principles, Methods and Applications, Knowledge Engineering Review, 11(2), 1996. [5] R. Mizoguchi, Ontology Engineering Environments. In S. Staab, R. Studer (Eds.) Handbook on Ontologies, Berlin: Springer, 275-298, 2004. [6] The Protégé Project (2003), Protégé 2000 user guide, UK: http://protege.stanford.edu/doc/users_guide/index.html [7] A.J. Duineveld, R. Stoter, M.R. Weiden, B. Kenepa & VR. Benjamins, WonderTools? A comparative study of ontological engineering tools, International Journal of Human-Computer Studies, 52(6), 2000, 1111-1133. [8] Chimaera Ontology Environment: http://www.ksl.stanford.edu/software/chimaera [9] N.F. Noy & D.L. McGuinness, Ontology Development 101: A Guide to Creating Your First Ontology http://www.ksl.stanford.edu/people/dlm/ papers/ontologytutorial-noy-mcguinness.pdf
(Authoring for) WBES: Aroyo and Mizoguchi [29] propose the use of an Authoring Task Ontology as a means of simplifying the authoring of material for Webbased Educational Systems (their examples of WBESs are SmartTrainer and AIMS). Their ATO defines the main authoring activities and sub-activities as well as stages and goals. It includes nouns (or objects to be manipulated) such as course, student, text; verbs (or authoring activities) such as modify, assign, select; adjectives (the modifications of the objects) such as finished, in use, updated.
31
[23] M. Hatala, & G. Richards, Making A Splash: A Heteregeneous Peer-To-Peer Learning Object Repository, Proc. of 12th International Conference of The World Wide Web Consortium (WWW2003), Budapest, May 2024, 2003. [24] Z. Zdrahal, P. Mulholland, J. Domingue, & M. Hatala, Sharing engineering design knowledge in a distributed environment, Behaviour and Information Technology, 19(3), 2000, 189-200. [25] S. Buckingham Shum, E. Motta, & J. Domingue, ScholOnto: An Ontology-Based Digital Library Server for Research Documents and Discourse, International Journal on Digital Libraries, 3(3), 2000, 237-248. [26] F. Giorgini & F. Cardinali, From Cultural Learning Objects To Virtual Learning Environments For Cultural Heritage Education: The Importance Of Using Standards, Digicult Thematic Issue 4: Learning Objects from Cultural and Scientific Heritage Resources., 2003, http://www.digicult.info/pages/index.php [27] L. Stojanovic, S. Staab, & R. Studer, ELearning based on the Semantic Web, Proc. WebNet2001 - World Conference on the WWW and Internet, Orlando, Florida, USA, 2001. [28] Y. Gil, & V. Ratnakar, Trusting Information Sources One Citizen at a Time, Proc. of the First International Semantic Web Conference (ISWC), Sardinia, Italy, 2002. http://trellis.semanticweb.org/expect/web/semanticweb/is wc02_trellis.pdf [29] L. Aroyo, & R. Mizoguchi, Process-aware Authoring of Web-based Educational Systems, First International Workshop on Semantic Web for Web-based Learning (SW-WL’03), Workshop at CAISE'03, Klagenfurt/Velden, Autria, June 2003. http://www.swwl03.bessag.net/images/Papiers/SWWL03-Paper%201.pdf
[10] Pressman, R.S., Software Engineering: A Practitioner's Approach, 3rd Ed., McGraw-Hill, New York, NY, 1992. [11] P. Mulholland, Z. Zdrahal, & T. Collins, CIPHER: Enabling Communities of Interest to Promote Heritage of European Regions, Cultivate Interactive, Issue 8, 2002. http://www.cultivate-int.org/issue8/cipher/ [12] G. Henry, R.G. Baraniuk, & C. Kelty. The Connexions Project: Promoting Open Sharing of Knowledge for Education, Syllabus, July 2003. [13] A. Naeve, The Concept Browser – a New Form of Knowledge Management Tool, Proc. 2nd European Webbased Learning Environments Conference (WBLE 2001), Lund, Sweden. [14] W. Nejdl, B. Wolf, C. Qu, S. Decker, M. Sintek, A. Naeve, M. Nilsson, M. Palmér, & T. Risch, EDUTELLA: A P2P networking infrastructure based on RDF, Int. World Wide Web Conf. (WWW 2002), 604-615, May, 2002, Hawaii. [15] B. Simon, Z. Miklós, W. Nejdl, M. Sintek, & J. Salvachua, Smart Space for Learning: A Mediation Infrastructure for Learning Services, Proc. of the 12th World Wide Web Conference, May, 2003. http://nm.wuwien.ac.at/research/publications/b164.pdf/ [16] R. Koper, R. (2001) Modeling Units of Study from a Pedagogical Perspective: The Pedagogical Meta-Model Behind EML, Educational Technology Expertise Center, Open University of the Netherlands. http://eml.ou.nl/introduction/docs/ped-metamodel.pdf [17] A. Naeve (1997) The Garden of Knowledge as a Knowledge Manifold - a Conceptual Framework for Computer Supported Subjective Education, CID-17, TRITA-NA-D9708, Department of Numerical Analysis and Computer Science, KTH, Stockholm. [18] L.E. Anido, M.J. Fernández, M. Caeiro, J.M. Santos, J.S. Rodríguez, & M. Llamas, Educational metadata and brokerage for learning resources, Computers & Education, 38(4), 2002, 351-374. [19] F. Paulsson, & A. Naeve, Standardized Content Archive Management–SCAM,IEEE Learning Technology Newsletter (ISSN 1438-0625), 5(1), 2003, 40-42, http://lttf.ieee.org/learn_tech/issues/january2003/learn_tec h_january2003.pdf [20] M. Dzbor, E. Motta, & A. Stutt, Achieving higherlevel learning through adaptable Semantic Web applications, Int. J. Knowledge and Learning, 1(1/2), 2005, 25-43. [21] W. Nejdl, Personalized Access to Distributed Learning Resources (PADLR), 2nd year continuation proposal, http://www.kmr.nada.kth.se/papers/SemanticWeb/padlr_2 2.pdf/ [22] C. Schmitz, S. Staab, R. Studer, G. Stumme, & J. Tane, Accessing Distributed Learning Repositories through a Courseware Watchdog, Proc. of the World Conference on E-learning in Corporate Government, Health Care, & Higher Education, E-Learn 2002, Montreal, CA, October 15-19, 2002.
32
