Ontology-based Assessment Technique Irina Leshcheva1, Dana Gorovaya2, Dmitry Leshchev2 1
Information Technology in Management Dept., Graduate Business School of Management, St. Petersburg State University, Dekabristov Per. 16, St. Petersburg, 199155, Russia 2 St. Petersburg State Politechnical University, Polytechnicheskaya st. 29, St. Petersburg, 195251, Russia
[email protected]
Abstract. Ontologies that describe the main concepts of the domains are used both for teaching and assessment techniques. The paper presents ontologybased approach to a student assessment. Students show their knowledge and understanding while creating ontologies. Such way of assessment gives advantages over traditional quizzes. The approach is illustrated by students’ test papers made them before and after course listening. Keywords: ontology, visual knowledge engineering, assessment technique
1
Introduction
According to one of the definitions of the concept Ontology [1], it is a hierarchically structured set of terms for describing a domain. The last years the main interest of the researchers in this field is concerned with the special tools that help knowledge capture and structuring [2]. But it is extremely useful to draw the ontology using well-known to everybody “pen and pencil” technique. Ontologies are useful structuring tools, in that they provide an organizing axis along which every student can mentally mark his vision in the information hyper-space of domain knowledge. Ontology design also may be used as an assessment procedure [3]. Students come to the university after they have successfully passed their exams. Nowadays most of the exams have а form of quizzes. To pass the exam students don’t have to show deep knowledge of a subject. It is enough to learn by heart how to answer most of questions correctly. It is possible even to guess the right answer. So using the quizzes doesn’t have objective results. Using ontology-based approach to the student assessment hasn't such disadvantages. It clearly shows the understanding of the subject. However it is impossible to guarantee impartial marks. So the idea is to use the ontologies as the assessment tool rather to understand the level of incoming students and to observe their improvements after a course.
2
Irina Leshcheva1, Dana Gorovaya2, Dmitry Leshchev2
2
Using Ontologies as an Assessment Tool
At the beginning of the course all students create their own ontologies of the domain in a way how they understand it at the moment. They receive an easy algorithm how to do it:
write down a list of the terms of the domain; combine the terms into groups and name this groups; create an hierarchical structure using groups and their names from previous step; check the created structure, add, delete or correct some items if needed.
Fortunately the students easily catch the idea how to create the ontology so explanation doesn’t take much time. From student’s ontologies the lecturer can see where gaps in their knowledge are so the course content can be adjusted to the particular group. During the course the students receive a lot of new knowledge and their domain ontologies are restructured and enriched. A lecturer can illustrate the course material by own ontologies. It helps to understand the lecturer’s view on the subject. But it is not necessary requirement for using onotologies as an assessment tool. After the course the students are asked to organize a list of predefined terms into ontology structure. The list is incomplete so they have to add missing concepts. The result ontology shows their understanding of the subject and how the knowledge of every student was expanded during the course.
3
The Ontology-based Assessment in practice
The described approach has been tested during the first part of the Information and communication technologies course. Before the beginning of the course students are asked to draw the ontology of the concept a Personal computer. In the Figure 1 the example of built “ontology” is shown. Obviously it is incomplete and ill-structured. P ersonal Co m p u te r K e y b o a rd
S o ftw a re
M ouse
M o n ito r
S y s te m u n it
M o the rboa rd
D isk d rive
C a se
C oo le r
H a rd d is k
C ab le s
Fig. 1. The example of ontology built before the course
The ontology built after the course is presented in the Figures 2. It still has a lot of mistakes. But the terms related to the Personal computer concept have already divided into two groups: software and hardware. This ontology provides the lecturer information which areas need more attention. For example, the student who created this ontology needs more knowledge about the software.
Ontology-based Assessment Technique
3
P erso nal C o m p u te r Softw a re
A p p lica tio n s
H a rdw a re
O p e ra tin g syste m s
M odem
S yste m u n it
N o te p a d
L in u x
C o o le r
M S W o rd
W in d o w s X P
P o w e r s u p p ly M o th e rb o a rd
A n tiv iru s s o ftw a re
G ra p h ic s e d ito rs
D e skto p
D ia lo g b o x
C PU
P a in t
V id e o ca rd
P h o to sh o p
S o u n d ca rd
In p u t K e y b o a rd
O u tp u t
D a ta sto ra g e d e vice s
M o n ito r
F lo p p y d is k s
Mouse
P rin te r
U S B fla s h d riv e s
M ic ro p h o n e
L o u d s p e a k e rs
C o m p a c t d is k s
Scanne r FD D D V D -R W d riv e
N o rth b rid g e S o u th b rid g e
H a rd d is k
Fig. 2. The ontology built by the same student after the first part of the course
4
Discussion
This described approach of assessment can be applied to a wide range of taught courses. Ontology design may be used as an assessment procedure for expressive as opposed to exploratory learning. For both formative and summarizing assessment purposes, students can clearly indicate the extent as well as the nature of their knowledge and understanding through creating ontology and explaining the involved processes. Acknowledgements This work is partly supported by the grants from Russia Foundation for Basic Research and Saint-Petersburg State University.
References 1. Gomez-Perez, A.: Tutorial on Ontological Engineering. In: Int. Conference on artificial Intelligence IJCAI’99. Stockholm, Sweden (1999) 2. Gavrilova, T.: Teaching via Using Ontological Engineering. In: Proceedings of XI International Conference “Powerful ICT for Teaching and Learning” PEG-2003, pp. 23--26. St. Petersburg, Russia (2003) 3. Gavrilova, T.: Ontological Engineering for Practical Knowledge Work. LNAI 4693, Proc. оf 11-th Int. Conf. "Knowledge-Based Intelligent Information and Engineering Systems KES 2007", pp. 1154--1162. Italy, Springer (2007)
