Universal Design Within the Context of e-Learning Andrina Granić and Maja Ćukušić Faculty of Natural Sciences, Mathematics and Education, University of Split Nikole Tesle 12, 21000 Split, Croatia
[email protected],
[email protected]
Abstract. When designing an e-learning system it is important to consider not only technical and resource related aspects, but user characteristics along with pedagogical and contextual issues as well. This paper proposes an approach to universal design within the e-learning environment able to address issues related to learner-centred design paradigm, context of use approach, individualized approach, pedagogical framework and guideline framework. A refinement from methodological point of view enhanced with the exemplification of approach's appliance is required. Keywords: universal design, e-learning, context of use, learner-centred design, pedagogical framework.
1 Introduction An evolution of computer from computational machine to "knowledge machine" influences a development of variety of new computer-mediated human activities, among which e-learning merits special attention. Placed on the crossroad between information and communication technology (ICT) and education, e-learning is considerably changing traditional definition of an educational system. However, despite so much publicity and activity, the progress in the field has been very slow. Related problems are mostly associated with poor design of e-learning applications [34] as well as still inadequate research base for e-learning evaluation [1]. Furthermore, although computers are being used at different levels of teaching process (as the subject of teaching as well as a tool for supporting the teaching process) and despite decades of research, their use for tutoring (as the teacher itself) in everyday teaching environment has been quite limited [16]. On the other hand, the objective of design for all or universal design is to provide accessible and usable information society technology (IST) products and services, raising so called good user-centred design paradigm to a more encompassing concept of addressing the needs of all potential users, cf. [31]. Therefore, an employment of universal design within the e-learning context promotes individualization and enduser acceptability, ensuring that usability and accessibility should be a design concern, thus avoiding the need for afterwards adaptations. Unfortunately, studies have regularly shown that the accessibility of web-based applications in general falls short of an acceptable level [25] and most of existing efforts related to accessible e-learning ones propose guidelines that primarily address technical accessibility issues [6]. C. Stephanidis (Ed.): Universal Access in HCI, Part III, HCII 2007, LNCS 4556, pp. 617–626, 2007. © Springer-Verlag Berlin Heidelberg 2007
618
A. Granić and M. Ćukušić
Additionally, when considering e-learning applications it has been claimed that usability assessment needs further consideration of learning perspective [20], [28], although some authors propose applying heuristics without further adjustment to the e-learning context, e.g. [7], [21]. Nevertheless, in e-learning we do not need user interfaces that support "doing tasks", but interfaces that support "learning while doing tasks", cf. [12]. Obviously, there should be a synergy between the learning process and a user's/learner's interaction with the application [29], hence taking into account the different ways user's learn and ensuring their natural and flexible interactions as well. Our research, focused on e-learning systems which emulate human teacher in the process of learning and teaching, brought about comparable assumptions. Although employed in the teaching process of a few real subject matter, they have been continually evaluated concerning their efficiency in the learning process and usability of their interface design. Related studies and empirical assessments however still mainly involve usability aspects, cf. [10], yet not addressing issues concerning accessibility and learning perspective as well. Applied methodology, based on the combination of behaviour and opinion founded measures, linked scenario-based user testing with a kind of "less formal" heuristic evaluation, i.e. guideline evaluation. A major strength in such an approach is the chance to supplement results from both the heuristic evaluation and the empirical user-based one, enhanced by users' feedback on their comfort while working with the system. Nevertheless, the design of accessible and easy to use system able to address the needs of all potential users requires additional considerations in the context of e-learning environments. The rest of the paper is structured as follows. Section 2 provides an overview of a number of pertinent approaches to efficient and usable e-learning design. These studies, along with our research and related findings, denote irreplaceable input for the description of the proposed "frame" for an inclusive design within the e-learning context. The approach along with its foremost aspects is elaborated in Section 3. Future research directions related to its refinement and application are discussed in Section 4. The same section concludes the paper.
2 Related Work With the aim of design e-learning systems with intuitive interfaces, ones which do not get in the way to effective learning and teaching, recent HCI research is starting to address a learning perspective as well. Brief insight into few studies along with identified main aspects describing best practice for interface design of e-learning systems is presented in the following. A holistic approach proposed by Phipps and Kelly [22] focuses on accessibility aspect and is comprised of few aspects: usability issues, making learning accessible, making courses accessible as well as adapting to individual, local political and cultural factors. The framework recognizes that accessibility of the system is primarily about people and not about technology. Hence, the design should take into account (i) accessibility, (ii) usability, (iii) learning outcomes, as pedagogic issues and aims of the e-learning resources, (iv) learner needs, (v) local factors, (vi) infrastructural and resources issues and finally (vii) quality assurance. Viewpoint outlined by De Marsico
Universal Design Within the Context of e-Learning
619
et al. [6] puts more emphasis on a joint design activity of e-learning system users and multidisciplinary experts who possess different skills related to didactical, pedagogical, technical and usability aspects, considering users with disabilities as well. In such context the roles and tasks of each participant in every successive phase of the elearning development lifecycle should be determined. Good and Robertson [9] argue that current learner-centred design frameworks do not consider the full range of issues necessary for successful design. Their learner-centred design framework named CARSS, specifically designed for child learners, embraces context, activities, roles, stakeholders and skills. Based on literature review, Zaharias [34] summarizes the most commonly identified research dimensions regarding quality and e-learning design issues, pointing out efforts in: • implementation of learner-centred design paradigms, • implementation of effective pedagogy for the design of e-learning courses and subsequent development of instructional design guidelines, along with • guidelines and frameworks for quality assurance and evaluation. Apparently common aspects appear in each study. Nevertheless, each approach to the learning and teaching system design emphasises one specific aspect in particular. Thoroughly considering important factors proposed by multidisciplinary design experts and bearing in mind findings from the related studies, we have devised approach to inclusive design within the context of e-learning.
3 An Approach to Inclusive Design in e-Learning Environment The frequently low level of usability and accessibility of e-learning systems indicates that focusing on the adoption of usability and accessibility guidelines is not sufficient for a truly design for all. As a result, simple appliance of usability heuristics, for example Shneiderman's golden rules or Nielsen's heuristics, cannot effectively be used because they fail to address the specific challenges of learner-centred design and the issue of integration of usability and learning [28]. Additionally, simple adoption of W3C guidelines in order to assure accessibility is just not enough, because the vast majority of the available accessibility guidelines are formulated either as general design principles or low level specific recommendations, not focusing on the needs of the users [33]. Evidently there is a need for further research and empirical assessment since an established set of heuristics [35] as well as a joint evaluation methodology for e-learning applications do not yet exist [2], [5]. Consequently, the application of design for all in an educational environment elevates end-user acceptability and customization of learning to individual needs, ensuring that usability and accessibility should be a concern of design. For that reason the design of accessible and easy to use e-learning system, able to address the needs of all potential users despite usability and learning perspective, requires additional considerations. The main issues regarding universal design related to e-learning systems have to embrace: • learner-centred design paradigm; the same practices followed by HCI community (employment of user-centred iterative design and repeated evaluation) must be used to ensure learnability, major issue for e-learning, according to Don Norman;
620
A. Granić and M. Ćukušić
• context of use approach; the main measure of an e-learning system is whether it goes with its context of use in terms of four components of HCI model [23]: whether the user for whom it is designed can use it with acceptable levels of usability and accessibility, for the tasks that s/he needs to do, in the local environment in which these tasks take place, using the available technologies; • individualized approach; the consideration of the users' different individual characteristics relevant to learning styles and preferences, such as social and cultural background, technical experience, interests, goals, physical/cognitive abilities and a like, fosters individualization and end-user acceptability; • pedagogical framework; covering at least three aspects: pedagogies which underpin the subject matter in e-learning environment, content and information for learners; and ways in which the subject matter is delivered to learners (delivery strategies). • guideline framework; the employment of usability and accessibility guidelines for e-learning quality assessment, ensuring that usability and accessibility should be a design concern, thus avoiding the need for later adaptations or specialized design. Identified aspects of the proposed approach along with addressed issues concerning universal design within an e-learning context are illustrated in Figure 1 and elaborated hereinafter. 3.1 Learner-Centred Design Paradigm The current research in HCI field acknowledges that understanding users' needs is the core of a successful design of IST products and services. This naturally leads to usercentred design approaches, a philosophy which places the users at the centre of design [19] and a process that focuses on cognitive factors (such as perception, memory, learning, problem-solving, etc.) as they come into play during users' interactions with application [34]. Moreover, in order to "make people more effective learners", i.e. to take into account the unique needs of users as learners, a shift from user-centred to learnercentred design is needed [27]. It can range from attempts to design with the needs of the learner at the forefront towards involving the learner at various stages of the design process [9]. Such approach entails understanding and considering who is the user, what are her/his needs, what we want her/him to learn, how is (s)he going to learn it and how are we going to support her/him in achieving the learning objectives. Accordingly, a variety of learners' types must be considered due to characteristics revealing user individual differences like personal learning styles and strategies, diverse experience in the learning domain, as well as previously acquired knowledge and abilities. Many authors have attempted to provide comprehensive lists of additional needs for specific educational domains, but [26] concisely summarize them under broad categories of universal applicability. In brief, in order to address the distinctive needs of users as learners, learnercentred design attempts to provide guidelines to augment the user-centred design framework.
Universal Design Within the Context of e-Learning
621
Fig. 1. An approach to universal design within the context of e-learning
3.2 "Context of Use" Approach An understanding of the context of use forms a useful input in order to cope with the concept of design for all, an inclusive and proactive approach seeking to accommodate diversity in the users and usage context. The notion of context of use, defined by ISO 9241 as "… the nature of the users, tasks and physical and social environments in which a product is used" [13], is already well recognized in the concept of universal design e.g. [14], [30], and closely related concept of universal usability [24]. On the other hand, it has been claimed that despite the fact that the power of modern IST artefact have been increased by a factor of thousand over the past two decades, user interface design have mostly stayed the same [3]. Regardless of advances in HCI field, few innovations had get through into new information society systems for all. It seems that such problem can only be overcome by switching from a focus upon the design of user interfaces to the design of interactions themselves and to the "wider view of interaction in the context of use" [ibid.]. Consequently context of use, as an important concept of everyday life in general, can be seen as a description of the actual conditions under which information society technologies are under assessment or will be used in a usual day by day working situation. From the educational perspective in particular, the effective use of context
622
A. Granić and M. Ćukušić
in an e-learning system goes in terms of four components of a HCI model [23]: whether (i) the user for whom it is designed can use it with acceptable levels of usability and accessibility, for (ii) the tasks that s/he needs to do, in (iii) the local educational, socio-cultural and organizational environments in which these tasks take place, using (iv) the available technologies (see Figure 1). 3.3 Individualized Approach As already emphasized, present HCI research puts the user, the individual at the centre of all developments, stressing the importance to design IST products and services for human needs. Therefore, the role of an intuitive user interface and a flexible interaction suited to different needs, preferences and interests becomes even more important for the users' success, as users with a wide variety of background, skills, interests, expertise, goals and learning styles cf. [4], [8] are using computers for quite diverse purposes. For that reason no single interface, though designed to be both easily learnable and highly efficient, will satisfy every user. Yet, the progress in the field of e-learning has been very slow and "… although technology is often touted as the great salvation of education – an easy way to customize learning to individual needs – it rarely lives up to this broad expectation" [11, p. 398]. It seems that too much of the research is being driven by technical possibilities, while paying inadequate attention to the area of application. The result is in an over-ambitious and pre-mature attempt to eliminate the teacher's role in educational environment [16]. Huge resources were spent for e.g. courseware development and not enough was left to improve the actual quality of learning [17]. Moreover, the interest received by user modelling aspect has not succeeded to address the variety and richness of the educational environment, even in the terms of user individual characteristics. These concerns have been ignored for quite long, hoping that new technology will somehow resolve the lack of real progress. The experience has proved so far that these issues cannot be wished away as they determine the type and scope of elearning systems that are likely to succeed. 3.4 Pedagogical Framework From a pedagogical point of view, an easy to use learning and teaching system is not necessarily also "properly designed". Namely, every e-learning experience begins with the pedagogy that drives it and continues with setting learning goals, designing learning activities which require the content suitable to meet the learning goals, cf. [15]. Although a consolidated pedagogical framework for e-learning does not exist thus far, it is argued that syllabus, course structure and design, navigation, assessment, collaboration and the like should be considered while defining framework components. E-learning system design based on selected pedagogical model would enable teachers to make use of the learning resources in a form which is appropriate to the learning goals and the particular learning style of the learner. UNITE's pedagogical framework [32], covering theories and practices for enhancement of the learning experience with the use of ICT, is a good example of elearning pedagogical basis design. This five-component pedagogical framework, through the combination of its constituents, produces the set of parameters that drive
Universal Design Within the Context of e-Learning
623
and guide the creation of learning scenarios. UNITE's pedagogical framework components are: • pedagogical framework context; defines areas that influence the framework itself and forms the basis for further development of UNITE's theories, • pedagogical approaches; promote particularly principles of constructivist theory, along with blended, collaborative and active learning, • assessment techniques; define and support diverse types of assessments as selfassessment, peer-assessment and so on, • teacher training; supports teachers' work and endorse them during content production as well as delivery strategies decision and finally • current pedagogical practices implemented in national curricula and national specifics. Definition of the pedagogical model to be used as well as setting up pedagogical goals represents an integral part of the e-learning design process and should be detailed out well before technology choices and initial user interface design. 3.5 Guideline Framework To facilitate design of usable interfaces and accessible systems in general, suitable guidelines and principles are essential. Usability should not only be evaluated, but "designed" from the start. Not only "the look" i.e. the visual design of the application, should be steered with proper guidelines, but also "the feel" i.e. the design of navigational and functional elements. Consequently, resulting e-learning application would be usable in view of a number of attributes: navigational fidelity, structure of hypermedia, system applicability, error handling, ease of use, consistency and standards, as well as performance evaluation, cf. [18]. Another system design issue that should be guided is accessibility. While some authors consider accessibility as one of usability's components, e.g. [34], others point out potential conflicts in including accessibility guidelines alongside usability ones, e.g. [22]. Most of the existing efforts for supporting the design, preparation and deployment of accessible e-learning propose guidelines that primarily address technical accessibility issues [6]. Although W3C has laid the foundation necessary for anyone to access and use web content [33], simple implementation of W3C guidelines does not suffice as they are not focusing on the needs of the users. There should be strict guidelines for both usability and accessibility not only at the general level, but on more subsequent ones. Because any adaptation of the system once it is in place is time and resource consuming, adherence to proposed principles is advised. However, although applying good design guidelines by themselves is a good start, there is no substitute for e-learning system assessment.
4 Conclusion and Future Research Directions In order to design e-learning systems with user interfaces which will not impede the effective learning and teaching, current HCI research is starting to address a learning perspective as well. Several recent studies to e-learning design are presented in the paper, offering a brief overview of diverse approaches to the identification of main
624
A. Granić and M. Ćukušić
issues related to the inclusive design in the context of e-learning. These findings, along with our research and experience, provided a constructive input for the proposition of the approach which addresses main issues concerning design for all in elearning environment. Such design, able to address the needs of all potential users, despite usability and learning perspective requires additional considerations, hence comprising: (i) learner-centred design paradigm, (ii) context of use approach, (iii) individualized approach, (iv) pedagogical framework and (v) guideline framework. Consequently, the main issues regarding universal design related to e-learning systems range from learner-centred design (thus taking into consideration context of use and individual user characteristics) to learning perspective or the pedagogy as a driving force of an e-learning experience as well as guidelines for good system design (related to usability and accessibility aspects). We propose addressing the distinctive needs of users as learners to augment the user-centred design framework. Special attention should be awarded to the context of e-learning usage defined as "a description of the actual conditions under which the e-learning system will be used in learning and teaching process". No less is to consider learners' individual characteristics as it is to define the pedagogical model and determine the pedagogical goals. Both learners' individual characteristics and the pedagogical model should be elaborated before making any technology choices or initial interface design. In order "to design for usability and accessibility", a set of strict guidelines on a general level and on a more subsequent one as well, has to be followed. So far, our research has been focused on e-learning systems which mimic the capabilities of human teachers in the process of learning and teaching. The systems have been repeatedly evaluated concerning their efficiency in the learning process and usability of the interface design. Our future research will address issues concerning learning perspective, individual user characteristics, context of use and accessibility as well. The work related to the proposed approach is still in the early phase – a refinement from methodological point of view is needed along with the examples of its appliance. In order to build up our five-aspect approach, future research directions will embrace: • specification of the final list of criteria per each aspect (with acceptable/recommended values and/or "best-practice" examples); the focus will be more on those aspects not yet addressed, • development of the appropriate methodology; so far the combination of behaviour and opinion founded measures (linked scenario-based user testing with a kind of "less formal" heuristic evaluation, i.e. guideline evaluation) was applied, and • an empirical study to test the efficiency of proposed approach will be fundamental. A coherent universal framework, a support for the design of e-learning systems able to address the needs of learners while performing learning activities in their learning environment, will be provided. Acknowledgments. This work has been carried out within project 177-0361994-1998 Usability and Adaptivity of Interfaces for Intelligent Authoring Shells funded by the Ministry of Science and Technology of the Republic of Croatia. Definition of Pedagogical framework presented in the paper has been carried out within the project 026964 UNITE: Unified eLearning environment for the school, partially supported by the European Community under the Information Society Technologies (IST) priority of the 6th Framework Programme for R&D.
Universal Design Within the Context of e-Learning
625
References 1. American Society of Training and Development ASTD (2001) [Available On-line] http://www.astd.org/astd 2. Ardito, C., Costabile, M.F, De Marsico, M., Lanzilotti, R., Levialdi, S., Roselli, T., Rossano, V.: An approach to usability evaluation of e-learning applications. Universal Access in Information Society 4, 20–283 (2006) 3. Beaudouin-Lafon, M.: Designing Interaction, not Interfaces. In: Proceedings of the working conference on Advanced visual interfaces, AVI 2004, Gallipoli, Italy, pp. 15–22. ACM Press, New York (2004) 4. Benyon, D., Crerar, A., Wilkinson, S.: Individual Differences and Inclusive Design. In: Stephanidis, C. (ed.) User Interfaces for All: Concepts, Methods, and Tools, pp. 21–47. LEA, Mahwah, New Jersey (2001) 5. Costabile, M., Marisco, M., Lanzilotti, R., Plantamura, V., Roselli, T.: On the usability evaluation of E-learning applications. In: Proceedings of the 38th Annual Hawaii International Conference on System Sciences (HICSS’05) – Track 1 – vol. 01, 6.2 (2005) 6. De Marsico, M., Kimani, S., Mirabella, V., Norman, K.N., Catarci, T.: A proposal toward the development of accessible e-learning content by human involvement. Universal Access in Information Society 5, 150–169 (2006) 7. Dringus, L.: An iterative usability evaluation procedure for interactive online courses. Journal of Interactive Instruction Development 7, 10–14 (1995) 8. Egan, D.: Individual Differences in Human-Computer Interaction. In: Helander, M. (ed.) Handbook of Human-Computer Interaction, pp. 543–568. Elsevier Science B.V. Publishers, North-Holland (1988) 9. Good, J., Robertson, J.: CARSS: A Framework for Learner-Centred Design with Children. International Journal of Artificial Intelligence in Education 16(4), 381–413 (2006) 10. Granić, A.: Human-centred Design in Intelligent Tutoring: a Key Role of Usability Evaluation. In: Proceedings of the 3rd Cambridge Workshop on Universal Access and Assistive Technology, CWUATT 06, Fitzwilliam College, pp. 121–129. University of Cambridge, Cambridge, United Kingdom (2006) 11. Healey, D.: Theory and research: Autonomy in language learning. In: Egbert, J., HansonSmith, E. (eds.) CALL environments: Research, practice, and critical issues. Alexandria, VA: Teachers of English to Speakers of Other Languages, Inc. pp. 391–402 (1999) 12. Hsi, S., Soloway, E.: Learner-centred design: Addressing, finally, the unique needs of learners. In: Computer Human Interaction ’98, CHI ’98, Los Angeles, USA, pp. 211–212. ACM Press, New York (1998) 13. ISO Ergonomic requirements for office work with visual display terminals (VDTs) – Part 11: Guidance on usability (ISO 9241-11:1998(E)). Geneva, Switzerland 14. Keates, S., Clarkson, J.: Countering Design Exclusion: An Introduction to Inclusive Design. Springer, London (2003) 15. Kelly, B., Phipps, L., Swift, E.: Developing A Holistic Approach for E-Learning Accessibility. Canadian Journal of Learning and Technology, vol. 30(3) (2005) 16. Kinshuk, Patel, A, Russell, D.: Intelligent and adaptive systems. In: Collis, B., Adelsberger, H., Pawlowski, J. (eds.) Handbook on Information Technologies for Education and Training, pp. 79–92. Springer, Heidelberg (2001) 17. Nielsen, J.: Jacob Nielsen on e-learning. Elearningpost. (January 16th, 2001) [Available On-line] http://www.elearningpost.com/features/archives/001015.asp 18. Nielsen, J.: Heuristic evaluation. In: Nielsen, J., Mack, R.L. (eds.) Usability Inspection Methods, pp. 25–62. John Wiley& Sons, Inc, New York (1994)
626
A. Granić and M. Ćukušić
19. Norman, D., Draper, S.W. (eds.): User centred system design. Erlbaum, Hillsdale NJ (1986) 20. Notess, M.: Usability, user experience and learner experience. eLearn Magazine. ACM Press, New York, NY, USA (2001) [Available On-line] http://www.elearnmag.org 21. Parlangeli, O., Marchigiani, E., Bagnara, S.: Multimedia systems in distance education: effects of usability on learning. Interacting with Computers 12, 37–49 (1999) 22. Phipps, L., Kelly, B.: Holistic approaches to e-learning accessibility. Association for Learning Technology 14(1), 69–78 (2006) 23. Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., Carey, T.: Human-Computer Interaction. Addison-Wesley, Wokingham, England (1994) 24. Schneiderman, B., Plaisant, C.: Designing the User Interface. Strategies for Effective Human-Computer Interaction, 4th edn. Addison-Wesley, Reading, MA (2005) 25. Sloan, D., Kelly, B., Heath, A., Petrie, H., Fraser, H., Phipps, L.: Contextual Web Accessibility - Maximizing the Benefit of Accessibility Guidelines. In: Conference Proceedings of WWW, Edinburgh, Scotland May 2006. Special Interest Tracks, Posters and Workshops (2006) [Available On-line] http://www.ukoln.ac.uk/web-focus/papers/w4a-2006/ 26. Soloway, E., Jackson, S., Klein, J., Quintana, C., Reed, J., Spitulnik, J., Stratford, S., Studer, S., Jul, S., Eng, J., Scala, N.: Learning Theory in Practice: Case Studies of Learner-Centred Design. In: Proceedings of Conference on Human Factors in Computing Systems (CHI96), Vancouver, British Columbia, pp. 189–196. ACM Press, New York (1996) 27. Soloway, E., Guzdial, M., Hay, K.E.: Learner-centred design: the challenge for HCI in the 21st Century. Interactions 1, 36–48 (1994) 28. Squires, D., Preece, J.: Predicting quality in educational software: Evaluating for learning, usability and the synergy between them. Interacting with Computers 11, 467–483 (1999) 29. Squires, D., Preece, J.: Usability and Learning: Evaluating the Potential of Educational Software. Computers Education 27(1), 15–22 (1996) 30. Stephanidis, C.: User Interfaces for all: New Perspectives into Human-Computer Interaction. In: Stephanidis, C. (ed.) User Interfaces for All: Concepts, Methods, and Tools, pp. 3–21. LEA, Mahwah, New Jersey (2001) 31. Stephanidis, C., Salvendy, G., Akoumianakis, D., Bevan, N., Brewer, J., Emiliani, P.L., Galetsas, A., Haataja, S., Iakovidis, I., Jacko, J., Jenkins, P., Karshmer, A., Korn, P., Marcus, A., Murphy, H., Stary, C., Vanderheiden, G., Weber, G., Ziegler, J.: Towards an Information Society for All: An International R&D Agenda. International Journal of HumanComputer Interaction 10(2), 107–134 (1998) 32. UNITE FP6 IST project. Contract No. 026964. Website http://www.unite-ist.org 33. Web Content Accessibility Guidelines 2.0. W3C Working Draft (April 27, 2006) [Available On-line] http://www.w3.org/TR/WCAG20/ 34. Zaharias, P.: E-learning design quality: A holistic conceptual framework. In: Howard, C., Boettcher, J., Justice, L., Schenk, K., Rogers, P.L., Berg, G.A. (eds.) Encyclopedia of Distance Learning, vol. II, Idea Publishing, New York (2005) 35. Zaharias, P., Vassilopoulou, K., Poulymenakou, A.: Designing on-line learning courses: Implications for usability. Scientific Journal on Applied Information Technology vol. 1(1) (2002)
