Concept Maps and Course Ontology: a Multi-level Approach to E-learning Roberto Pirrone1 , Massimo Cossentino2 , Giovanni Pilato2 , and Riccardo Rizzo2 1
DINFO - University of Palermo Viale delle Scienze 90128 Palermo, Italy 2 ICAR - Italian National Research Council Viale delle Scienze 90128 Palermo, Italy
[email protected],{cossentino,pilato,ricrizzo}@pa.icar.cnr.it
Abstract. The work is mainly focused on the description of an original scheme for information representation in e-learning applications, where three levels are used to describe course related knowledge: content level, symbolic level, and conceptual level bridging the previous ones. Moreover, an integrated architecture design is proposed to support the presented model. The reference scenario is the development and deployment of lessons for undergraduate students using both local and internet resources. Multimedia information is managed at a content level relying on the SCORM standard. At a symbolic level, an ontology is built as a linguistic knowledge base, where all the course topics are represented together with temporal and user-related constraints. The ontology is used to generate a suitable learning path in response to the student requests, which has the form of a SCORM course description. At the intermediate level, which bridges the content and the symbolic ones, topics are represented by a concept map, implemented as a SOM network. The map is used to cluster the course materials, and to map them onto atomic concepts that can be instantiated at the ontological level. This solution allows the discovering similarity between topics, and creating new relations between them at the symbolic level. A detailed description of the model is presented, along with the outline of the architecture.
1
Introduction
Last years have been characterized by a growing interest in e-learning technologies, above all in the field of web learning. This can be defined as the integration of classical automated training systems with the WWW communication paradigm, enriched by the possibility of sharing knowledge between users by means of the semantic web infrastructure. Such systems allow students to access several remote sources of information about a certain topic, and to follow a personalized learning path among the various resources. Web-learning systems dramatically reduce the need for teacher and learner to be simultaneously present in the training phase. Finally, they allow collaborative strategies both for teaching, and studying.
If we look at the application fields of these systems, three main areas arise: professional training, scholar and continuous education. In the case of professional training the kind of knowledge one wants to transfer to the student is a procedural one, and it is related to the process of acquiring skills in very specific areas or updating previous competencies. Continuous education is mainly directed towards people willing to keep updated on emerging cultural fields. The main reference framework of the present work is inside the third scenario: here a pool of teachers has to prepare their lessons for undergraduate students. In this case, the teacher has to estimate the already acquired competencies of her/his students in order to tailor each further course topic. Besides this, according to current cognitive learning theory, a constructive process takes place when one studies a new matter. Each student builds its own conceptual schemata which are continuously restructured while new information is gained. The performance in problem solving, and answering questions about a topic depends both on prior knowledge and mental representation the student has built. As a consequence of the previous statements, there not exists a unique learning path for a lesson, and it is better to provide the learner with the possibility of consulting more information for each argument. A good learning system has to take into account all of these considerations, so it should exhibit the following properties: – generation of a course structure based on a main learning track where each lesson is enriched by a pool of information (explanations, tests, case studies, and so on); – managing a profile of the student where its preferences about ”how to learn” are coded. In recent years, several techniques have been proposed in literature to cope with the problem of generating a learning path to train a student, on the basis of her/his requests. Sacks and his collaborators [1][2] propose a system based on modified fuzzy-c-means algorithm to cluster documents; clustering is used to generate a Topic Map [3] that is accessed by a web interface. Another approach that uses SOMs has been proposed by Honkela [4] where the topology preserving property of the Self Organizing Map is claimed to underline the subjective representation of knowledge according to the constructive learning theory. Personalized learning path building is proposed in [5] where Petri nets are used to dynamically aggregate lesson contents annotated with SCORM meta-data, and to model the student behaviour. Integrated systems have been also proposed like MASEL [6] and FRED [7]. Both of them are FIPA compliant agent-based systems, and make use of ontologies to model the knowledge shared between agents in order to manage their communication. MASEL architecture uses logical inference upon a set of rules to generate suitable learning paths, while FRED relies on an ”environment ontology” which describes the training process. In this work we propose a multi-level schema to model course topics, and an integrated e-learning architecture to support it. At the lowest level, information is aggregated according to the SCORM standard [8]. In the SCORM framework, pieces of information are annotated with a set of standard meta-data in order to
provide a description of their content. A hierarchy of XML structural descriptions are generated to define data aggregation used to build single learning objects (e.g. slides) composition of learning objects (e.g. presentations) or entire courses. In our approach we will speak about documents that can be either learning objects or aggregations. Each document that is used to build a course can be either authored by the teacher or downloaded form Internet and annotated locally. The intermediate representation is achieved by a concept map that is a Self Organizing Map [9] used to cluster documents using a measure of the similarity between the terms they contain. A concept map is trained in an unsupervised way, and it is labelled with some landmark concepts that are used to bridge the gap with the symbolic level. The concept map owns an implicit representation of logical relations between concepts, and allows easy integration of new clusters standing for new concepts that can be instantiated at the symbolic level together with their relations with the nearest regions. Finally, a linguistic representation of the domain ontology is provided, where the landmark concepts play the role of atomic assertions inside the knowledge base. The use of a linguistic framework to build the ontology, instead of a XML document or RDF schema, has many advantages. Such an ontology can be used to integrate in a unique knowledge representation the course domain and the student model which regards both the definition of her/his competencies when starting the learning activity, and the student’s preferences about the way of learning a certain topic. Suitable logical predicates can be used to relate these two pieces of knowledge, and a wide collection of AI approaches can be used to derive a learning path starting from the user needs: simple logical inference, planning under constraints, graph search, reasoning, and so on. The rest of the paper is arranged as follows. In section 2 a complete description of the three levels of representation is provided. Section 3 deals with the architecture we propose to support the model. Finally, section 4 reports a brief description of the first experimental tests along with a discussion of the possible developments of the architecture.
2
The proposed model
2.1
SCORM representation of information
Our architecture is based on a representation of the course structure and contents that relies upon the SCORM specifications proposed within the Advanced Distributed Learning (ADL)3 initiative that has been funded by the U.S. Defense Department. This proposal starts from other existing attempts of standardization like IMS 4 or the IEEE Learning Technology Standards Committee (LTSC)5 . 3 4 5
Web site: http://www.adlnet.org Web site: http://www.imsproject.org/ Web site: http://ltsc.ieee.org/
SCORM is a set of specifications dealing with the course structure, the runtime environment for the course delivery (the Learning Management Server, LMS) and the data meta-description in terms of XML tags. A SCORM course is described in an XML document (called manifest file) reporting the course contents, the navigation paths among its different components that are included in the course in terms of links to their physical implementation (files). A course is composed of other courses and of Shareable Content Objects (SCO) that are the atomic self-consistent components: they can be shared among different courses and their structure is described in an XML document where their elementary parts (called Assets) are enumerated. SCOs are designed to be managed and delivered by a SCORM-compliant LMS. As already said, a SCO is composed by Assets that are multi-media resources (texts, images, videos, portions of code) described by XML meta-data. A course is obtained as a Content Aggregation of SCOs and Assets and it is delivered by an LMS following the rules described in the Manifest File. The navigation structure among different SCOs is provided by the LMS (that deduces it by the SCO Manifest File), while the navigation within the same SCO should be provided by the author. 2.2
Concept Maps
The Self Organizing Map network (SOM network) [9] is a neural network in which neurons are organized in a lattice, usually a one or two dimensional array, that is placed in the input space and is spanned over the input vectors distribution. Using a two dimensional SOM network it is possible to obtain a map of input space where closeness between units or clusters in the map represents closeness of the input vectors. In order to use self organizing networks to organize document collections is necessary to use a vector document representation to obtain the training vectors. The representation used is the Term Frequency–Inverse Document Frequency (TF–IDF) a so-called Vector Space Representation (VSR), a document encoding based on statistical considerations. Using the VSR each document in a document collection is represented by using a vector where each component corresponds to a different word. The component value depends on the frequency of occurrence of the word in the document weighted by the frequency of occurrence in the whole set of documents. The SOM to Sort Document Collections Recently in many papers the applications of self organizing neural networks to document clustering, and in particular of the Self Organizing Maps, has been emphasized. The most important application is the WEBSOM system: a SOM document map with a Web interface used to classify Usenet newsgroup articles. The papers [10], [11] report the application of SOM network to order 4600 document based on full text contents. The documents are messages from the ”comp.ai.neural-nets” newsgroup. In [12] it is claimed that the system can also be used to organize library collections and corporate full-text databases.
Document clustering is not the only performance of the SOM map, the proximity of the clusters on the map is also an important feature, but few studies investigate the proximity and the clustering hypothesis. One of them is reported in [13] in which authors analyse the proximity hypothesis, for which related topics are clustered closely on the map. In [14] the organization of information atoms in semantic clusters obtained by using the SOM was compared to the organization imposed by an hypertext author. In this study the SOM was trained by using the nodes of an hypertext and the nodes in the same unit or in units connected by the rectangular lattice were considered linked each other. This organization was compared to the link structure imposed by the hypertext author. The author finds that the 64.5% of the link in the original hypertext was ”covered” by the SOM network a result that validate the document organization obtained by using the SOM. For the purpose of explorative search the SOM lattice could be translated into an HTML table [14]. Exploiting the SOM organization features The Self Organizing Map was also used to obtain ”classic” associative links, or horizontal links, in hierarchical hypertexts. The idea was to obtain, in automatic fashion, the associative links that are missing in many educational hypertexts. Usually in these educational material there is a pre–defined path that is followed from parent node to child node with a little freedom. The information contained in these nodes can be considered the main information: a chain of concepts that constitute the backbone of the course structure. The user can navigate through this frame but usually there are few opportunity to ”explore” the information set. With horizontal links it is possible to compare new information with the main one and to explore ”lateral” path that allows the user to go in deep in the topics. These other information can be considered at the same level of the main one. This was achieved in the KnowledgeSea system presenting to the user a map of the available information labeled with some landmarks. These landmarks constitutes the lesson material: something that the student know well. In this way they are free to navigate the other material and at the same time they have some strong well–known reference points. The core of the KnowledgeSea is a two-dimensional map of educational resources where each cell is used to group together a set of web pages. In the Knowledge Sea System the SOM was used as a navigation tool, and as a tool that can rebuilt the information that are missing in the Web hypertexts. The horizontal links represent a sort of semantic proximity and this is more likely to append if the text are short, as ear as possible to the information atoms that should built the real hypertexts. As cited before this classification of the hypertext atoms gives interesting results. The SOM in the proposed system The use of the SOM in the proposed system is to build a sort of continuous concept space where many documents regarding different concepts are organized (fig. 2.2). So the concept space is build
over the SOM map exploiting the property that semantically related concepts are near each other. This map is also labelled using landmarks as in [15] but this time there are some important differences. First of all the landmarks are not lessons or slides but key concepts that where ad–hoc chosen in order to represent significant landmarks over the concept space. Second these landmarks are not supposed to be useful for the student; in fact a single lesson can cluster many concept, probably for the lessons scheduling problem. These landmarks are used to allows the ontology to deal with the concept, and to organize the concept sequencing needed to obtain lessons and courses. The map simply allows to have a continuous labelled space and separate the concept space from the document space. In this way the Ontology engine can manipulate the concepts but not the documents that are supplied by the map. This architecture allows
Fig. 1. A representation of how the concept map works
the course designer to work with the concept in order to made–up the concepts sequence and the backbone of the course. After that the map can be used to populate the course with the learning material, the documents are supplied by the map and will be found in the same cell of the interesting concepts. The map in this case work as a ”link–engine” capable to supply a set of document linked together that will constitutes a set of available information related with the main concept. Obviously the author will choose which information are more suitable for the particular course or will develop the material that is missing, but s/he should maintain available some material in order to allows the horizontal navigation already cited. So that, the structure of the course can be visualized as in fig. 2.2: a set on linked concepts, sustained by few backbone documents, that are surrounded by interesting documents that can be freely browsed. 2.3
Course Ontology
Ontologies are specifications of the conceptualization and corresponding vocabulary used to describe a domain [16]. An ontology consists of definitions of concepts related to a domain, relations and axioms about the concepts and relationships. As known ontologies are designed to provide an interface between different domains and are useful for the description of heterogeneous, distributed and semi-structured information items.
Fig. 2. A pictorial example of a course structure
For the above mentioned properties they can be helpful for the planning of a learning course. The definition of concepts, and the relation existing between them, can be also a good choice for managing the information about an e-learning course. The use of the ontology, in fact, can be very useful for finding prerequisite dependencies and the relation between the arguments present in the system for planning the most appropriate sequence of arguments of the course. The Organization of the Ontology In the last years, the Cycorp, Inc company has developed the Cyc knowledge base (KB) which has a very large ontology constituted by over one hundred thousands atomic terms axiomatized by a set of over one million assertions fixed in nth-order predicate calculus. The Cyc KB, at present, is the largest and most complete general knowledge base, equipped with a good performing inference engine [17] [18]. In the proposed approach the learning material is organized onto the SOM Concept map: each concept is mapped onto a Cyc collection that expresses the topic of the slides about the argument treated. The basic concepts (i.e. the landmarks of the SOM) are organized as monothematic collections, called Arguments. Hence we have the basic element of the e-learning ontology that is a ”concept”, which is a collection of slides, mapped in the SOM Concept map. The concepts which are strictly related as belonging to a given topic are collected into Cyc collections, named arguments, that can be themselves organized in higher-level argument collections. All the concepts belonging to a given argument are, of course, conceptually related, but also a concept Conceptk belonging to an argument Argi can be conceptually related to a concept belonging to another concept Conceptj of the argument Argl or to a whole argument Argm . The Cyc Engine provides the infrastructure for the building and enhancement of a course-oriented knowledge base and the construction of new predicates that can be introduced for explaining better the relations between concepts and/or entire arguments. Let us suppose to require a course on the concept ConceptD : from the map we see that for explaining this concept, it is necessary to know the concept ConceptK and the concept ConceptJ , and so on recursively, besides the concept
ConceptH of the argument Arg1 is a prerequisite for the concept ConceptJ . This situation can be deducted by the inferential engine of Cyc. The SOM contains the basic elements of the course, i.e. the slides. As explained before, we can say that the slides are automatically clustered by the SOM and then they are automatically labelled according to the concepts set. Each concept can be considered as a landmark. The concept is the highest level of ”abstraction” for the map and the lower level of abstraction in the ontology. The concept, therefore, constitutes the bridge between the sub-symbolic representation of the learning materials and the ontological representation of the concepts [19]. The ontology will then be used for planning the specific e-learning path. In the proposed approach, the landmarks, in fact, are declared in Cyc as ”collections of slides”, that can be organized in larger collections, here called ”arguments”. The arguments can be recursively organized in larger arguments. The arguments contain concepts (or arguments) that are strictly related on a well-defined and bounded topic, however the concepts of an argument can have relations with other concepts and/or argument belonging to other arguments than the belonging one. Besides, a whole argument can be related in different manner with other arguments, or with other specific concepts of a given argument. In fact, a course structure is strictly dependent on the semantic relations that intercourse between different concepts and/or arguments. These items of knowledge could be therefore connected each other in order to be able to build up a complete, coherent and organic learning course. The ontology engine can plan the course sequence without referring directly to the documents. The semantic basis results in a better semantic description of learning materials and better searching for useful materials according to user preferences. In the proposed solution a concept can be related to another one or to an argument and an argument can be related to another one or to a specific concept belonging to a given argument using the following binary predicates: 1. isPrerequisiteFor 2. conceptuallyRelated 3. containsInformationAbout. The first one was not present in the Cyc KB, while the other two are yet present in the knowledge base. In fig. 2.3 is reported a representation of the bridging mechanism between the concept map and the ontology, along with the ontology arrangement. With the organization of the concepts in arguments and the arguments in higher-level arguments collection, joined with the three predicates illustrated above, they are available the most important tools for design the ontological organization of the learning course items.
3
Description of the Architecture
The model for information representation already presented, has been partially implemented in an integrated architecture for web learning activities. The outline of the system is reported in fig. 4. The actors interacting with the system are the
Fig. 3. Bridging mechanism between the ontology and the concept map
Editorial Staff, the Author (s) and the User (s). The editorial staff is composed by the teacher or teachers preparing the course: they can assemble materials from local resources or from the Internet and make use of the Authoring tool to define the SCORM backbone of the course. This tool [20] allows the collaboration of multiple authors in writing a course. Each of them is responsible for the composition of one or more SCOs that he/she creates using the tool. Once a SCO is completed, it could be sent to the content management server, published as part of the course and reused in new ones. Editorial staff is also responsible for the training of the Course Map, and for defining arguments and relations in the Ontology module. Each single author feeds the system with the course resources through the authoring tool, that performs annotation using SCORM meta-data, and population of the Lessons Repository. In the repository, actual multimedia resources can be accessed directly through their SCORM indexing. Students are the users of the system: they have interaction by means of a web GUI, that is used both for lessons delivery through a SCORM compliant LMS and to ask for training about a certain topic. A Request Collector module interacts with the student requiring information about her skills, and her preferences in the learning procedures e.g. if she prefers procedural presentation of arguments, learning by examples, problem solving, and so on. The answers are translated into a suitable set of features by the Configuration tool. In order to obtain a comprehensive description of the user, observation of the dynamic behaviour of the user at the interface should be taken into account e.g. if she makes several clicks upon particular links (pictures, detailed explanations of terms, and so on) rather than the others or what is the navigation path of the mouse when she reads a document. The set of features describing the user is instantiated in a User Model space which provides compact information to the Learning Path Generator. This module, uses the ontology to create a symbolic representation of
User model
Ontology Editorial Staff
Concept map
Learning path gen
Course manager
Author
Config tool
Authoring tool for lessons and tests
Request Interpreter Lessons Repository
G U I
User
LMS
Fig. 4. The outline of the whole web learning architecture
the learning path in terms of related arguments that are connected according to the logic predicates we illustrated in section 2.3. The Course Manager identifies actual documents from the concept map, generates the SCORM description of the course for the LMS, and verifies that the repository owns all the needed resources. 3.1
Agent-based Implementation
The discussed system is quite complex and even the logical representation of its main components provided in fig. 4 shows that a rigorous software engineering approach can be very helpful in fully achieving the system goals. Several interesting challenges are related with the realization of this program. First of all it is an highly distributed system with several users working different sites at the same time (while the editorial staff is creating the outline of a new course, some authors could write new contents and some students could study parts of the existing courses). The presence of highly experimental components (inference engine and other artificial intelligence related elements) and the likelihood of introducing substantial changes with the introduction of new features suggest the opportunity of pursuing some characteristics like an easy maintenance, an high encapsulation of modules and above all the availability of a complete, detailed design of all the system. Another important issue is that the dimension of the whole project creates some realization problems to a research group as small as the involved one. In order to face these problems we decided to develop the system using the multi-agent paradigm. It offers several advantages: the involved researchers are already skilled in this field and some of them have a specific experience in the MAS (multi-agent systems) design methodologies. Agents can be considered as sub-systems with an high level of encapsulation and their natural vocation to the autonomy helps in decomposing the whole problem into a set of simpler one. They can be easily deployed into different platforms and referring to a well defined standard (FIPA [21] in our case) simplifies the realization of
communications among the different components in different hosts. In order to justify the approach to the design and implementation of the system that we will describe, let us consider each of the modules of fig. 4 as a different system. This is correct because they really are different applications. Each system is composed of several sub-systems devoted to provide services one to the other in order to cooperate in realizing the system functionalities (for example the authoring tool includes an editor that allows to the authors the composition some SCOs). If we think at this situation from a multi-agent point of view, then we have that each module can be considered as an agent society, more specifically an open society (it accepts collaborations with external agents). The composition of all these societies creates the whole MAS. Each agent is responsible for only a part of the behavior of the society it belongs to and it can be easily maintained, upgraded or substituted with a new one with the only attention of supporting all of its communication capabilities. Each agent offers some services to the community and could need the collaboration of other individuals in order to achieve its goals. In designing the solution, we will pursuit a specific goal: lowering the time and costs of its realization. In order to obtain this result, we think that a fundamental contribution could come by the automation of as many steps of the process as possible (or similarly by providing a strong automatic support to the designer). In pursuing these objectives we use a design methodology (PASSI, ”Process for Agent Societies Specification and Implementation” [22]) specifically conceived to be supported by a CASE tool (PTK-PASSI ToolKit6 ) that automatically compiles some models that are part of the process, using the inputs provided by the designer. PASSI is a step-by-step requirements-to-code methodology for developing multi-agent software that integrates design models and philosophies from both object-oriented software engineering and MAS using UML notation.
4
Conclusions and Future Work
We presented an original model to cluster information related to course topics for a web learning application, which uses three levels of representation: SCORM meta-data, concept maps, and domain ontology. An integrated architecture has been proposed to support the model. Though the development of the architecture is still in progress, early results are very encouraging. We are building a model for the courses in ”Foundations of Computer Science” taught at the University of Palermo. Besides, since verification has an important part in all the pedagogical approaches, we are also introducing a test module in the system. A test will be composed of several questions in order to verify all the educational goals of the course (or sub-course). The test author can decide how many questions will be introduced in the examination about each argument of the course. These questions will be randomly taken from the repository that should contain a sufficient number of alternatives. As a consequence, each examination will be different from the other but the number of questions for each argument (and to some extent, the difficulty) will be the same. The tool permits the creation of 6
www.csai.unipa.it/passi
questions with multiple answers (the student should indicate the correct one) or with open text answers. Obviously, in the first case, if the test author introduces the correct answers in the system, the examinations will be automatically corrected and the results immediately provided to the student and recorded in the student curriculum. Currently the inferential engine built in Cyc is used to browse the ontology, and to generate paths, but we are investigating the implementation of a planner or a reasoning system. Another interesting research field is the definition of a complete user model. A possible implementation can be obtained using a SOM, like in the concept map. Clusters in this user map should represent conceptual definitions of different users typologies to be instantiated at the ontological level.
References 1. Mendes, M., Sacks, L.: Dynamic Knowledge Representation for E-Learning Applications. In: Proc. of the 2001 BISC International Workshop on Fuzzy Logic and the Internet, FLINT 2001, University of California Berkeley (2001) 2. Mendes, M., Martinez, E., Sacks, L.: Knowledge-based Content Navigation in e-Learning Applications. (In: Proc. of London Communication Symposium LCS 2002) 3. Pepper, S., Moore, G.: XML Topic Maps (XTM) 1.0. TopicMaps.Org Authoring Group (2001) http://www.topicmaps.org/xtm/index.html. 4. Honkela, T., Leinonen, T., Lonka, K., Raike, A.: Self-Organizing Maps and Constructive Learning. In: Proc. of International Conference on Educational Uses of Communication and Information Technologies, ICEUT’2000, Beijing, China (2000) 339–343 5. Liu, X.Q., Wu, M., Chen, J.X.: Knowledge aggregation and navigation high-level Petri netsbased in e-learning. In: Proc. of International Conference on Machine Learning and Cybernetics. Volume 1. (2002) 420–425 6. Garro, A., Palopoli, L.: An XML Multi-Agent System for e-Learning and Skill Management. In: Proc. of 3rd Int.l Symposium on Multi-Agent Systems, Large Complex Systems, and EBusinesses MALCEB 2002, Erfurt, Thuringia, Germany (2002) 7. Smolle, P., Sure, Y.: FRED: Ontology-based Agents for enabling E-Coaching Support in a large Company. In: Proc.of 2nd Int.l Workshop on Ontologies in Agent Systems OAS 2002. (2002) 8. ADL SCORM Home Page, (http://www.adlnet.org/) 9. Kohonen, T.: Self-Organizing Maps. Springer, Berlin, Heidelberg (1995) 10. Kaski, S., Lagus, K., Honkela, T., Kohonen, T.: Statistical Aspect of the WEBSOM System in Organizing Document Collections. Computing Science and Statistics 29 (1998) 281–290 11. Kaski, S.: (Data exploration using self-organizing maps) Mathematics, Computing and Management in Engineering Series No. 82, Espoo 1997, 57 pp. Published by the Finnish Academy of Technology. 12. Lin, X., Soergel, D., Marchionini, G.: A Self-Organizing Semantic Map for Information Retrieval. In: Proc. of the Fourteenth Annual International ACM/SIGIR Conference on Research and Development in Information Retrieval, Chicago IL, USA (1991) 262–269 13. Lin, C., Chen, H., Nunamaker, J.: Verifying the Proximity Hypothesis for Self-Organizing Maps. In: Proc. Of the 32nd Hawaii International Conference on System Sciences. (1999) 14. Rizzo, R., Allegra, M., Fulantelli, G.: Hypertext-like Structures through a SOM Network. In: Proc. of ACM Hypertext ’99, Darmstadt, Germany (1999) 15. Brusilovsky, P., Rizzo, R.: Map-Based Horizontal Navigation in Educational Hypertext. Journal of Digital Information 3 (2002) 16. Gruber, T.: A translation approach to portable ontology specifications. Knowledge Acquisition (1993) 199–220 17. Cycorp Home Page, (http://www.opencyc.org) 18. Reed, S.L., Lenat, D.B.: Mapping Ontologies into Cyc, (http://citeseer.nj.nec.com/509238.html) 19. Chella, A., Frixione, M., Gaglio, S.: Understanding Dynamic Scenes. Artificial Intelligence 123 (2000) 89–132 20. Pirrone, R., Cossentino, M.: Uno strumento di authoring multi-agente per applicazioni di weblearning. AI*IA Notizie 16 (2003) 47–51 21. FIPA - Foundation for Intelligent Physical Agent, (http://www.fipa.org) 22. M.Cossentino, Potts, C.: A case tool supported methodology for the design of multi-agent systems, Las Vegas (NV), USA, The 2002 International Conference on Software Engineering Research and Practice, SERP’02 (2002)
