The Design, Development and Evaluation of Hypermedia Courseware for the World Wide Web A.D. Marshall and S. Hurley Department of Computing Science, University of Wales, Cardi , PO Box 916, Cardi . CF2 3XF, U.K. Email: dave,
[email protected] Abstract
The World Wide Web provides hypertext and multimedia based information across the Internet. Many applications have been developed on http servers. One important and novel use of the servers has been the provision of courseware facilities. This includes on-line lecture notes, exercises and their solutions as well as interactive packages suited primarily for teaching and demonstration packages. A variety of disciplines have bene tted notably C programming, X Windows, Computer Vision, Image Processing, Computer Graphics, Arti cial Intelligence and Parallel Computing. This paper will address the issues of (i) implementing a variety of computer science courses and (ii) using the packages in a class environment. It also considers how best to provide information in such a hypertext based system and how interactive image processing packages can be developed. A suite of multimedia based tools have been developed to facilitate such systems and these will be described in the paper. In particular we have developed a number of methods for running applications live over the WWW.
keywords: Courseware, Multimedia, Hypermedia, Parallel Computing, Computer Vision, Evaluation
1 Introduction The use of computers to provide an integrated environment for teaching a variety of disciplines has received much attention in recent years. Indeed many
frameworks[6, 16, 33] have been developed for such purposes. The material provided by such courseware varies greatly from the provision of lecture notes and lecture support material through to integrated and interactive tutorial packages. Until recently courseware has existed as stand alone packages[6, 33], however with the advent of the World Wide Web (WWW)[30] on the Internet and accompanying WWW (hypertext) browsers, such as Mosaic[22], Netscape[23] and HotJava[28], the provision of courseware has taken on a whole new dimension. Many subjects can bene t from the provision of such courseware. Indeed we are probably fortunate that our chosen disciplines lend themselves to such methods. For example many methods that we describe in our courses are interactive and can take on many states depending on the input data. It is dicult and/or time consuming to convey all such possibilities in a lecture (or on static media such as handouts or textbooks). Integrated courseware has an obvious advantage in presenting course material. In this paper we present our experiences of implementing courseware and an evaluation of using it in a class situation to support the teaching of six courses: C programming. X Windows. Parallel Processing. Computer Vision and Image Processing. Arti cial Intelligence. Computer Graphics. The outline of the approach to developing (and evaluating) the courseware arose, in part, from the TLTP project `Courseware in High Performance Computing'. The overall aim of this project e ort was to improve the educational e ectiveness of the materials being produced by adapting accepted instructional design models and theories to the original directives of the TLTP project. Those directives, which include the decision to use WWW browsers such as Mosaic[22] and Netscape[23], as a delivery medium with a potentially diverse group of students from disparate disciplines, result in educational demands which require careful consideration. Over the past couple of years a variety of courseware has appeared on the WWW. Many courses provide notes and lecture material in the form of HyperText textbooks | where sections of text are broken up into hypertext links that naturally re ect the layout of a book. Some have extended this work further to include links to other documents worldwide[32, 3, 16, 1]. Some level of interaction between the student and the course has also been facilitated. For a more complete review of such matters and for more details on programming issues consult the following references[8, 9, 18, 19, 20].
2
2 Developing Courseware In this section we aim to summarise our approach to developing courseware on the WWW. We begin by highlighting the advantages of using the WWW for courseware. We then broadly describe how the work has been implemented. Developing courseware is not merely a matter of preparing a series of lectures, linking them together and packaging them as a course. A comprehensive design strategy must also consider how to implement and manage a course, how to evaluate the materials that are used, and how to assess the learners. With that in mind, there was a conscious e ort to take a systematic approach to the development of course materials. According to Rowntree [26] the systems approach should incorporate four basic strands:
Identify course aims and objectives Develop necessary learning experiences Evaluate the e ectiveness of learning experiences Improve the experiences in the light of evaluation
In the remainder of the paper, each of these four strands will be addressed.
3 Identify Course Aims and Objectives
3.1 Flexibility of materials
One of the initial, intended goals of our courseware was that it can be used in a variety of courses, perhaps including undergraduate degree programmes in computer science, physics, all branches of engineering, mathematics and electronics, as well as the basis of training courses run by computer service departments. The challenge of designing learning materials for such a diverse group is to make the materials approachable for all classes of user, and yet maintain a high degree of specialism, for example, relevant to the eld of parallel computing or computer vision. Designing such materials confronts many well-established and accepted instructional design principles. The rst step in many instructional design models is to analyse the learners who will use the materials. Analysis of even a subset of potential users, however, would have proved expensive and time-consuming. Therefore a compromise was made by putting e ort into ensuring that the material would appeal to a broad audience.
3.2 The Use of Hypermedia and the World Wide Web (WWW)
The characteristics of an instructional medium which interacts with the learner are the factors that might in uence the learning process [14]. One concern might 3
be how the chosen hypermedia implementation of the courseware might a ect learning. The model that was developed re ects some of the known advantages and disadvantages of hypermedia. One of the advantages of hypermedia is the control it gives learners over the material they are learning. Jonassen [12] has suggested that the non-linear exploration of hypermedia could lead to a richer understanding of the structure of a particular knowledge domain. Laurillard [15] on the other hand has criticised the use of hypermedia as a teaching technology because it lacks feedback on the user's action (see Section 4.7). The material, therefore, needs to include sucient links to allow experienced learners the freedom to explore, but enough structure to support novice learners who need more guidance and feedback. To achieve this, a model was designed that included links to all relevant material (i.e. in the same unit) in the unit's index document. (This document also contains links to all diagrams within the unit). Each document in the unit has a link to the index, the link being grouped with the other unit traversal links, e.g. next page, home page etc.. This design was adopted because courseware has a more logical progression than do many hypermedia software packages. A bene t of this design is that a novice user can proceed safely through the work, since the actual core lecture notes are generally free from links which break the
ow of information presentation. On the other hand, the experienced user can easily access any related topics via the index page. This feature is available at all levels of the unit, thus minimising unnecessary link traversal - a problem which tends to disorient users and hinder the e ective use of the courseware [17]. There can be little doubt that the WWW has become the most successful networked multimedia hypertext based system in recent years. The HTML language[2, 30] used in WWW documents is extremely simple and yet powerful to use[8, 9, 18, 19, 20]. These factors highly in uenced our choice of hypermedia implementation systems. However several severe restrictions in the current WWW protocol mean that more advanced hypermedia systems need developing. We brie y discuss these issues and also point out their signi cance to courseware development.
3.2.1 Recent Developments on the WWW
Recent advances in WWW technology and software have seen the advent of second generation hypertext systems. Two such currently popular systems are HotJava[28] and Hyper-G[13, 29]. Both these build on on current WWW browsing techniques and aim to expand the relatively static nature of current browsers. Current WWW browsers, such as Netscape, have a lot of detailed, hardwired knowledge about the many di erent data types, protocols and behaviors necessary to navigate the Web. It is true that extensions can be added to provide some level of external functionality for new data types or protocols but this can become quite complex and cumbersome as their number increases. Also, due to the static nature of the current hypertext link system a number of limitations can arise once tasks begin to link in many kinds of media. 4
One problem is that hypertext links are explicitly addressed (the le name and WWW URL are directly referenced in the source object) so that if the indexed information is relocated or deleted the link is lost. Also limited search facilities are available that can search across boundaries of the WWW and authorisation features are lacking. These limitations are generally necessary for security reasons on a globally networked based system. Consequently researchers have looked at addressing such issues and second generation browsers are now appearing, although no one has yet uni ed all of the above mentioned problems. We will brie y mention two such browsers. HotJava[28]1 is signi cant because it makes the WWW truly interactive by incorporating applications that can be programmed, run live and distributed in a simple, safe and portable manner. HotJava also provides an extensible method to handle, internally, new data types and protocols. Basically HotJava has no initial concept of any data types or protocols. Instead it provides a generic mechanism for incorporating information about any data type or protocol by adding new internal classes de ning how such things should be handled. HotJava builds on the Internet browsing techniques established by Mosaic[22] and expands them by implementing the capability to add arbitrary behavior, which transforms static data into dynamic applications. HotJava is a Web browser that makes the Internet totally interactive and provides a new way for users to access these applications. Software transparently migrates across the network. There is no such thing as \installing" software. It just comes when you need it. Content developers for the WWW do not have to worry about whether or not some special piece of software is installed in a user's system; it just gets there automatically. This transparent acquisition of applications frees developers from the boundaries of the xed media types like images and text and lets them do whatever they would like. HotJava has these dynamic capabilities because it is written in a new language called Java[28]. Brie y, one can think of Java as a simpli ed, safe, and portable (device independent) version of C++. Another newly developed system is Hyper-G[13, 29]. Hyper-G claims to be one of the rst true hypermedia system supporting tools for structuring, maintaining and serving heterogeneous multimedia data including text, images, digital audio and video, PostScript and 3D scenes. Hyper-G was developed to primarily address issues concerned with referencing data over the WWW but also provides many interesting enhancements with regards to data and protocol handling. The Hyper-G system provides a server as well as browsers such as Harmony and Armadeus Various issues as to the applicability of these systems for our purposes, programming issues etc. will be discussed in Section 4.6.3.
How can this in uence Courseware on the WWW?
The innovations provided by the second generation WWW browsers provide many interesting possibilities with respect to developing courseware: Applications can now become truly interactive | the Java language pro1
Future versions of the Netscape browser will licence HotJava.
5
vides a ready made mechanism for this. Also the inclusion of Postscript with Hyper-G provides some scope here as programs and scripts can be executed from within Postscript. Signi cant advances in incorporating a full range of media have been made. Distributed hypertext linking over the WWW should also be improved.
4 Develop Necessary Learning Experiences The materials originally designed at Cardi were based on lecture notes from existing courses. The use of lecture materials is a logical foundation on which to build a course. On their own, however, lecture notes are insucient. User activities during learning are more important in determining what is learned than the presentation of instructional material [21]. The aim of evaluating the original lecture notes was to convert them into more e ective learning materials. To accomplish this, the initial lecture notes in HTML format were evaluated using models developed from principles of instructional theory. Evaluation was undertaken at an organisational level and instructional level. The organisational level focuses on courseware structure, by means of analysis of users and evaluation of the learning that has taken place. At the instructional level evaluation was concerned with the educational e ectiveness of unit content.
4.1 Organisational Unit Design
We initially considered several Instructional Design models and eventually adopted an established instructional design model: the ASSURE Model developed by Heinich, Molinda and Russels[7]:
A Analyse Learners S State Objectives S Select Media and Materials U Utilise Media and Materials R Require Learner Participation E Evaluate and Revise
The use of the ASSURE model for initial evaluation allows for the systematic alteration of existing course material (lecture notes, etc.) by focusing on learning issues which might not have been addressed in the original lecture notes, such as the potentially diverse characteristics and experiences of users. For example, this led in many cases to the alteration of language to suit a more general audience.
4.2 Instructional Unit Design
The model described above is based on an iterative approach whereby the courseware developed is assessed and then revised. This may need to occur 6
at the course unit level. The instructional level of evaluation is concerned with increasing the educational potential of each unit. This evaluation framework was adapted from Gagne's sequence of Instructional Events [5, 4], which are based on the hypothesised sequence of internal stages of information processing derived from studies of cognitive processes. Table I provides an overview of the events of instruction. Instructional Event
Relation to Learning Process
1. Gaining attention Reception of patterns of neural impulses 2. Informing learner of the objective Activating a process of executive control 3. Stimulating recall of prerequisite Retrieval of prior learning to working memory learning 4. Presenting the stimulus material Emphasising features for selective perception 5. Providing learning guidance Semantic encoding; cues for retrieval 6. Eliciting the performance Activating response organisation 7. Providing feedback about performance Establishing reinforcement performance correctness 8. Assessing the performance Activating retrieval, making reinforcement possible 9. Enhancing retention and transfer Providing cues and strategies for retrieval Table I. Events of Instruction and Their Relation to Processes of Learning c 1992 by Holt, Rinehart and Winston, Inc.) (From [4]. Copyright The use of Gagne`s events of instruction as an evaluation tool led to further changes in the development of a suitable model and particular changes in courseware content. For example instructional event three, stimulating recall of prerequisite learning, led to the insertion of additional references to other units. Utilisation of the ASSURE model and Gagne's Instructional Events is not intended to provide a prescriptive design model. Rather, it provides a framework based on sound instructional strategies within which it is possible for individual course designers to develop a dialogue about design strategies. The tools provide a common ground for collaboration. To illustrate the function of these tools in the development of educationally sound courseware, the implementation of each of the steps of the ASSURE Model will be discussed.
4.3 Analyse learners
As previously mentioned, it is expected that the users of the courseware will vary considerably in background and also in their purpose for using the courseware. This presents a problem in designing the courseware at the correct level of complexity, scope, etc. One factor a ecting the model design is the anticipated implementation of the courseware. Will it be used independently of any lecturing, or in conjunction with lecturing courses, or even merely as a revision tool to support full lecturing courses? These are questions that cannot be fully answered at present, though it is probable that the courseware will be used in 7
a variety of scenarios. It must be exible enough to meet the various demands of the di erent implementations. After evaluating the uses of the courseware it may be necessary to develop separate modules adapted to meet the various teaching environments, but which cover the same topics. At present, the courseware has been developed so as to be robust enough to stand as an independent teaching tool, with the aim that it will satisfy most of the demands made upon it. Lecturers may then be selective about the courseware material they use in their courses, adapting their courses as necessary. The material is designed for users with no previous knowledge or experience of the courseware content, and who lack adequate understanding of related principles. Hence, the lecture notes that were used as a foundation for the courseware content needed to be expanded thoroughly to provide a more satisfactory coverage of the topics presented, since they were originally designed to be used to teach experienced computer science students within the university. Jargon has to be eliminated or explained thoroughly where unavoidable, with relevant links to a glossary provided in each (main) document. Most concepts must be explained in simple terminology, minimising the need for a familiarity with the topics covered or any points made therein. It is our goal to restrict the ow of information as little as possible. For example, demonstrations and examples are included via linked documents, rather than being contained within the main unit notes. (This also conforms with HTML style guide policy by decreasing the length of documents.)
4.4 State Objectives
Following on the assumption that the user is unfamiliar with the subjects presented, it is important that objectives are stated explicitly at appropriate levels of the courseware. This is done because the purposes of the courseware should be explained. This explanation should be part of the courseware design, particularly if it is to be used independently of lecturing. Learners are informed speci cally of what they should learn from the courseware. This approach increases their awareness of the key points in the notes, and hence increases the potential of the courseware to deliver the education required. Although each unit is prefaced by an introduction, the stating of objectives also serves as an advanced study organiser. There is another reason for stating speci c objectives explicitly. If the courseware is to be used in any way in conjunction with lecturing courses, they inform the lecturer of precisely what they aim to present, and thus the lecturer can accurately adapt the courseware into his/her own course. According to the model speci cation, a unit contains a list of objectives which relate to the unit as a whole, yet which can still be de ned speci cally. Each sub-topic within a unit also has a list of objectives which cover that particular sub-topic. Thus the stating of objectives re ects the hierarchical design of the model. Some sample objectives are: 8
Correctly label an unlabelled diagram of the SISD model. Brie y explain the limitations of the SISD model. 4.5 Select Media and Materials
The decision to use the WWW (and subsequent WWW browsers) as the medium for delivering the courseware was made at the commencement of the project. Consequently, the selection of media and materials has been less restrictive than conventional choices, due to the diversity of media formats supported by WWW browsers. Whilst HTML provides the possibility of linking in various hypermedia features e.g. MPEG movies, audio clips, in-lined images etc., it requires a suitable browser, like Netscape[23], to handle them. However, another factor limiting the use of hypermedia is the hardware used to run the browser. For instance, a computer with no audio facilities cannot play audio clips, no matter which browser is used. Therefore, the \e ectiveness" of a piece of hypermedia software, however well designed, is ultimately determined by the browser used and the machine upon which that browser is run. It has been assumed that the implementation of the courseware will most likely occur in university settings where hardware and software capable of presenting all media employed by the courseware is readily available.
4.6 Utilise Media and Materials
The following sections contain illustrations of two implementations of hypermedia incorporated into the courseware.
4.6.1 Using MPEG Movies to Animate Algorithms
The MPEG movie format is the most popular storage format for image sequences on the WWW. Most browsers are able to support such a format. Animation of algorithms is clearly a useful learning tool[10, 19]. Illustrations can be compiled o -line and simply stored and played back on request. Our courseware has extensive use of such a facility. An example (from the Parallel Processing courseware) is shown in Figure 1. In this context various algorithms are discussed, that are used to implement message passing (data transfer) in different ways. Whilst these algorithms are not overly complicated nor lengthy, the visualisation of their execution for non-simple processor networks is quite challenging. This problem has been eliminated by the use of hypermedia, in this case by using MPEG movies.
4.6.2 Using Forms and Scripts to Achieve User Interaction
User interaction in hypermedia environments is often limited to selecting options with a mouse. In such an environment, the learner is merely presented with the information, having few opportunities to interact with the material. The 9
Figure 1: A Snapshot from an MPEG Movie HTML language however provides opportunities to develop additional types of participation. Consider the following example: Simulated annealing is a non-trivial process whereby the loads on a multiprocessor network may be minimised (usually not optimally). Observation of students has shown that involvement in the implementation of this algorithm improves their comprehension of it. The courseware implements a simulation which allows the user to execute the algorithm on a simple linear processor network. The user is able to see the results of the algorithm by means of a graph which plots the load on the network against the \work" done by the algorithm. To enhance understanding the user can adjust the various parameters which a ect the algorithm's performance. The algorithm can be rerun with di erent parameters, and the new results are plotted on the same graph with the previous results. This allows the user to compare and contrast performances, and understand how the parameters a ect the algorithm, and consequently more fully understand how the algorithm works. The demonstration of simulated annealing is possible because HTML allows links to executable programs and scripts. As long as a suitable HTML document is produced by the program(s) called, the user is unaware of all the \behindthe-scenes" operations that are taking place. The main feature of HTML which allows user interaction is the HTML form, which enables user input to be passed to the programs which produce the HTML documents. Some snapshots of the simulated annealing Interactive Example are shown in Figures 2 and 3. We have also made extensive use of forms and scripts to provide comprehensive search facilities within our courses[18, 19, 20]. This is a popular tool since it provides an easy means to access parts of the course in a similar manner to the index of a book.
10
Figure 2: A Form which allows the User to a ect Algorithm Performance 11
Figure 3: The User can compare several runs of the Algorithm 12
4.6.3 Using Java for true algorithm animation and user interaction
The recent innovations provide by the Java language and its ability to integrate runnable applications live over the WWW provide many exciting possibilities. The rst version of the courseware[18] was implemented before the advent of HotJava. All the background processing performed in the initial versions live was achieved by running (Perl) scripts and C programs with the resulting data and images mapped back to the HTML browser (Section 4.6.2). The second version of some of the courseware has the processing routines rewritten, where appropriate, in HotJava. This was not a major problem as the routines were available in C and easily modi ed for the (C++ lookalike) Java language. Examples of the courseware featuring HotJava applications (applets) for thresholding2 of an image are illustrated in Figs. 4 and 5. Whilst the initial outward appearance may not be that di erent there are many bene ts to this new approach:
Routines are more easily integrated into manageable programs (Java ap-
plets). { previously programs were usually embedded into scripts which resulted in fragmentation of the code. Java code being object oriented is easily extensible and reusable. All data is embedded into programs and managed internally. { Due to the html protocol data previously had to written to les of a particular type (e.g. GIF images) in order to be displayed. For the above reasons applications written in Java run signi cantly faster than the previously script encoded versions. The courseware is more global in that pervious routines were device, speci c meaning that some aspects of the course was only available locally. So long as you have access to a browser which understands Java code then you will have access to all of the course. Java compatible browsers are freely available for most major computer platforms. The Image processing Java courseware provides a mechanism for processing data located anywhere on the WWW.
4.7 Require Learner Participation
Hypermedia has been described as a more active learning medium than text because it forces learners to make decisions about which links to follow. This has been described by Romiszowski [25] as a feed forward system: the user has
2 Thresholding is a basic image processing function where pixels in the image are binarised according to whether their initial values are above or below a given (threshold) value. Thresholding is by de nition a very interactive process
13
Figure 4: Initial image display 14
Figure 5: Initial image display 15
control over which information nodes to view or ignore. This is a restricted form of interaction, however, which neither indicates nor ensures that the learner is processing the material at a deep cognitive level. The simulated annealing interactive example, described in the previous section, illustrates one method of achieving learner participation. However, whilst demonstrations such as this serve to increase the user's understanding of the relevant topics, they still do not guarantee the learner's correct processing of the material presented. Each unit therefore includes exercises developed speci cally to examine a range of learning objectives, from the testing of simple verbal knowledge, to more complex problem solving. Presenting a variety of question styles is simple. However, for the interaction of the user to be complete, some form of assessment and feedback needs to be implemented. This can be achieved either by user's self assessment or by external assessment. User self assessment involves providing a model answer for each question presented. The user is left to assess the correctness of submitted answers.External assessment involves the submission of the user's solutions to a tutor or instructor, who provides appropriate feedback to the user.
4.8 Evaluate and Revise
This step of the ASSURE Model is concerned with the evaluation and revision of the courseware to meet the constraints imposed by the implementation environment. The topic of how to evaluate courseware is covered in the next section and a practical assessment of our work is presented in Section 6.1.
5 Evaluate the E ectiveness of Learning One of our concerns was how to evaluate the teaching materials. A traditional approach is to look at both the product - the materials that have been produced - and the procedure, i.e. how students learn from the material. The evaluation should seek to answer how well the objectives of instruction have been met, and what e ects the use of the materials had on the learners. Such an evaluation is necessarily in uenced by the way that the materials are implemented, so any evaluation of the materials must necessarily wait for eld trials. What has been attempted is to develop general evaluation tools which will be included in each unit. They aim to address both organisational and instructional levels. One advantage of using HTML for this purpose is that it is possible to link evaluation forms to each module. The forms include Likert scales and spaces for comments. To evaluate the e ectiveness of the teaching materials, learners will be asked to rank their con dence for understanding a particular learning objective. Each learning objective for the unit will be stated, and con dence ranked accordingly. The feedback received from this data will allow designers to evaluate the e ectiveness of the material at the level of the individual unit. 16
More detailed evaluation will depend on how the material is implemented and how learners are assessed. Consider the following scenarios: Use of the modules as stand-alone self-instructional material replacing lecture courses. Use of the modules as a study reference in addition to lectures. Use of the modules to replace sections of lecture courses. Each of the scenarios has potential merits and poses particular educational and logistical problems. As a stand-alone teaching package the materials may need additional editing, to suit particular subject domains. Another issue which needs to be addressed is providing a means of managing and monitoring student learning. While there may be nancial savings by decreasing the amount of lecturing time, they will be o set by increased expenses for development of materials and administration. The second option, providing the materials as an additional resource, gives students the opportunity to study the relevant material at their own pace. This can be a boon for students who have missed lectures or need to reinforce the notes they have taken with additional material. The bene ts of such an approach are primarily to the student. Use of the materials in this matter would have little nancial impact on course delivery. The cost of designing such a resource, however, must then be absorbed wholly by the developers. Given the increasing number of students in Higher Education, however, such bene ts should not be ignored. From an educational perspective, some research has gone so far as to suggest that hypermedia should be limited to use as a resource [15, 27]. The nal implementation scenario is a mix between the former two. In this scenario, implementation would involve the integration of the hypertext material with lectures. This would allow some savings in terms of lecture time and management and administrative costs. For example half of the course might be delivered by lectures and half by hypermedia. This combination would avoid the need for course organisers to write customised learning materials as any subject-speci c input could be handled within lectures.
5.1 Improve the experiences in the light of evaluation
Improvement of the learning materials will depend partly on how they are implemented. Each of the scenarios that have been mentioned are potentially useful both educationally and nancially. Undoubtedly, the material will need to be altered and adapted to suit the needs of particular students and subject domains. The evaluation tools that will be included with each unit will provide users of the material with some of the feedback needed to improve the experiences of learners. A detailed evaluation of the material, however, is possible only once the materials are in use and detailed eld studies can be undertaken.
17
6 Overview of Current Implementations The courseware we have implemented currently integrates the following:
Course notes, reading lists, class information etc. Algorithm animations | interactive image processing over the WWW. Program listings or algorithms. Links to run programs and view program output. Exercises. Solutions to exercises. Sample images. Links to other sources of information on the WWW.
The courseware follows the basic framework laid out above in that major topics are basically treated as chapters of a hypertext book (Figure 6). Subtopics are sections and subsections of each chapter. Many examples of results are given in the form of image sequences. These are simply hypertext links to images. One other development is that live data processing can be performed on the WWW. This has been achieved by spawning programs, Java applets or scripts in manners described previously[18, 19, 20].
6.1 Analysis of Courseware
It is dicult to obtain metrics to perform a qualitative analysis of our courseware developed. Instead in the analysis presented in the following sections, we give student responses and our own personal analysis of the work to date. Also we include comments from the Internet community. All courseware has been implemented and used in classroom situations during the past two academic years. Thus we feel our courseware has had a fair amount of testing in practice.
7 Designer's/Implementer's Response In this section we assess the various issues which we as designers and implementers of the courseware feel are important. We also relate our experiences as teachers using the packages in a classroom situation.
18
Figure 6: C programming courseware front page 19
7.1 Assessing the design of our Courseware
The issues of hypermedia courseware design has been covered in more detail in Section 2 where we adopted Rowntree's four-strand model[26] for a systematic design and within this framework we also adopted Heinich, Molinda and Russels' ASSURE Model[7] (Section 4.1). In accordance with the above framework we have developed various forms of assessment which are detailed in the following sections. This we believe has been partially successful although further distinctions need to be made between the various groups now using our courseware. We presently have taught student courses, self-learning internal students and students accessing our material from the WWW using more or less the same courseware material. This is presently being investigated. We are also continuingly modifying our courseware in the light of evaluations.
7.2 Using the WWW as an implementation medium
The adoption of the WWW as a courseware provider has proved to be successful. It provides a exible way to provide a variety of presentation methods supporting a host of multimedia from hypertext, static images through to movies and even audio. Also, since (Java and C) programs and/or scripts can be run from a WWW server there is little that cannot be provided on-line. We have already mentioned several advantages in using this approach through this paper. We have been able to demonstrate that this approach is possible and furthermore does not pose any problems to implement given a good knowledge of UNIX and basic computer programming.
7.3 The WWW from a teaching perspective
The courseware we provide to our students consists of a few key components that aim to provide an integrated learning environment that can supplement lectures and laboratory teaching or exist as a stand alone self-study medium. This has several advantages:
Students can progress at their own pace and perhaps study course ma
terial in an order best suited to their abilities. The information is stored on-line and o ers virtually unlimited access for students. Information stored in hardcopy form, in libraries say, are limited to library opening hours and in numbers of copies available. Flexibility | a number of technologies (e.g. text, images, sound) and methodologies (e.g. notes .....) can be used. Students can make mistakes and not feel embarrassed as they might in a class situation. 20
Algorithm Animation { Scope exists for programs and algorithms to
be animated graphically and textually in order to explain and illustrate key programming concepts. The innovations provided by Hot Java and Hyper-G o er great potential in this area. Key concepts and good practices can be shared amongst a variety of courses. Alternative and/or comprehensive course material can be delivered.
8 Student Response Student response to the courseware has been very encouraging. This is not a surprise since some of the initial impetus for the provision of on-line information came from student requests. The courseware has been available for the whole of the past two academic years. Student response to this was good. This was evident from the use it received during the course (85% of students using the material | Table 1 below) and from student responses | both informal verbal comments and from end of course questionnaires (see below). Very little diculty was experienced in using the system. It also has proven popular as a revision aid during examination times. The layout in the form of a hypertext book appears to be to favoured format for students to browse through topics and the provision of advanced key word searches aids revision and general course work preparation. The provision of source code and algorithms for image processing routines, runnable versions delivering images as output and having both available for simultaneous study was warmly welcomed.
8.1 Student studies and questionnaires
Formal student projects and studies have also been set up during the year with the aims of gauging student response and also to evaluate our work with relation to other WWW courseware. The students themselves represented a small sample of the intended target audience. The ability to look at and run entire image processing algorithms was again well received, although students felt that a much wider selection of available example programs would help improve further. This is being developed. Over the past two years at the end of each course students are invited to comment on the courseware. A copy of the questionnaire used is listed at the end of the paper in Appendix A. Tables 1, 2 and 3 we provide a summary of their opinions. It is dicult to read any profound signi cance into the above results as the survey covered all target student classes. Also the WWW, being a very recent development, may need some time to become established as a learning tool in students' minds. This appears to already be happening though judging from 21
Question/Reply Use of WWW Use of Courseware Future use of C/W
Not at all 3.9% 15.4% 7.7%
Sometimes 31.1% 15.4% 46.2%
Frequently 49.6% 63.1% 27.3%
Table 1: General usage statistics of courseware
Question/Reply C programming Parallel Processing X Windows A.I. Computer Vision
Yes No 93.8% 6.2% 68.0% 32.0 % 62.8% 37.2% 58.3% 41.7% 57.8% 42.2%
Table 2: Speci c usage statistics of courseware
Question/Reply Useful Information Ease of use Clear Presentation Preference of On-line to Paper Documentation
Yes 90% 96.1% 85.9%
No 10% 3.9% 14.1%
57.7% 42.3%
Table 3: User opinion of courseware
22
All the time 15.4% 6.1% 18.8%
the survey there seems no doubt that this will continue as the WWW enjoys a high media presence. Most students appear to nd the courseware a useful learning tool and this appears especially so in revision periods. One factor that may lead to less frequent use is the fact that, in general, students only have access to the course on machines resident in the department. Full internet access to students using only their personal, home based machines is relatively rare at the moment at our institution. The use of particular courseware is also a ected by its integration within the current lecture course. The C programming, X windows and A.I. are intended to provide integral support to current lectures. The parallel processing and computer vision courses were not totally developed for this purpose and therefore have a looser integration in the current taught courses on these subjects. Also some courseware (C programming, X windows and parallel processing) is more mature and established. For example the A.I. courseware was only introduced in the last academic session and is still undergoing some nishing touches. However, it is clear from the results that student perception of the course is very favourable.
9 WWW Response All course notes were installed on the Cardi Information Server without any restrictions early last year (1994) after some initial student testing. It is very surprising how quickly word spread around the WWW. We made no real conscious e ort to advertise extensively. We have received many favourable comments about our work from numerous sources on the WWW who have compared it favourably with other courseware available at the moment[1, 11, 24, 31, 32]. Many have indicated that some of the courses developed o er the most complete integrated packages on the WWW in terms of comprehensive coverage of topics and the provision of executable programs and advanced search facilities. A majority of special interest newsgroups and WWW sites around the globe now provide pointers to our course notes as good examples and reference them as another good source of information. Indeed some even carry copies of the courseware as WWW access can be slow due to the vast number of users on the WWW/Internet. We are currently carrying out a study of WWW usage of our courseware.
10 Conclusions From the experience gained from using a variety of browsers and the WWW to develop a variety of courses in a number of situations we conclude the following:
The courseware packages were easy to develop. 23
Students nd the packages easy and intuitive to use, in particular: { They can work at their own pace. { The environment allows a two-way learning process.
The courseware is very popular with students and increasingly to a wider
audience over the WWW. The courseware are easily extensible and updatable. Material from the whole WWW can be easily integrated. The ability to use Unix (and other) scripts and more signi cantly to run (Java) programs directly means that little is not achievable with careful thought and planning. Our courseware has been recognised as a bene cial aid to learning. Our department has bene tted from a reduction in manpower resources in support of this and other courses available in hypertext form. Courses can be developed to suit a wide range of disciplines and abilities. In summary, we feel that global hypertext-based on-line courseware provides great exibility for future educational needs and we have extended global resources in this area.
11 Using Our Courseware Our courseware is freely accessible over the WWW (URL: http://www.cm.cf.ac.uk/Teaching/). We welcome comments and input on the courseware developed so far.
12 Acknowledgements We would like to acknowledge to assistance and wealth of WWW knowledge of Robert Hartill, Stefan Kruger and Robert Evans in our department. The courseware would not be at the level it is today without their support, encouragement and advice received from them.
References [1] C. Butts, C. Reilly, M. Speh, and Wang J. WWW and the Global Network Academy. In Proceedings of the First International on the World Wide Web, Paper no 36, CERN, Geneva, Switzerland, May 25-27 1994. World Wide Web Document URL: http://www1.cern.ch/papersWWW94/speh.ps. [2] Clarkson University, World Wide Web Document URL: http:// re.clarkson.edu/doc/html/htut.html. HTML Documents: A Mosaic Tutorial, 1994. 24
[3] E.A. Fox et al. Progress in interactive learning with a digital library in computer science. In Proceedings of ED-Media 95: World Conference on Educational Multimedia and Hypermedia, pages 7{12, Graz, Austria, June 1995. [4] R. Gagne, L. Briggs, and W. Walter. Principles of Instructional Design (4th edition). Harcourt, Brace, Jovanovich, New York, USA, 1992. [5] R.M. Gagne. The Conditions of Learning (4th Ed.). Holt, Rinehart and Winston, New York, USA, 1985. [6] W. Hall and H. Davis. Hypermedia link services and their application to multimedia information management. Journal of Information and Software Technology (Special Edition on Multimedia), 1994. [7] R. Heinich, M. Molinda, and J.D. Russell. Instructional Media. Macmillan Publishing, New York, USA, 1993. [8] S. Hurley, A.D. Marshall, S.N. McIntosh-Smith, and N.M. Stevens. Courseware for parallel computing using Mosaic and the World Wide Web. In Proceedings of the 2nd International WWW Conference '94, volume 1, pages 499{508, Chicago, USA, October 1994. [9] S. Hurley and N.M. Stephens. Courseware in high performance computing. In Proceedings International Conference on Parallel Computing for Undergraduates, Colgate, USA, June 1994. [10] B. Ibrahim. World-wide algorithm animation. In Proceedings of the First International on the World Wide Web, Paper no 32, CERN, Geneva, Switzerland, May 25-27 1994. World Wide Web Document URL: http://www1.cern.ch/papersWWW94/bertrand.ps. [11] Internet News Group. Comp.lang.c Frequently Asked Questions, 1995. [12] D. Jonassen, K. Beissner, and M. Yacci. Structural Knowledge: Techniques for Representing, Conveying, and Acquiring Structural Knowledge. Lawrence Erlbaum, New Jersey, USA, 1992. [13] F. Kappe and H. Maurer. Hyper-G { a universal hypermedia system. Journal of Educational Multimedia and Hypermedia, 2(1):39{66, 1993. [14] R. Kozma. Learning with media. Review of Educational Research, 61(2):179{211, 1991. [15] D. Laurillard. Rethinking University Teaching. Routledge, London, U.K., 1993. [16] M.G. Lavenant and J.A. Kruper. The phoenix project: Distributed hypermedia authoring. In Proceedings of the First International on the World Wide Web, Paper no 21, CERN, Geneva, Switzerland, May 25-27 1994. World Wide Web Document URL: http://www1.cern.ch/papersWWW94/j-kruper.ps. 25
[17] G. Marchionini. Hypermedia and learning: Freedom and chaos. Educational Technology, 28(11):8{12, 1988. [18] A.D. Marshall. Hypertext based computer vision teaching packages. In Proc. SPIE Conference on Machine Vision Applications, Architectures, and Systems Integration III, Photonics East 94, volume 1, pages 207{219, Boston, USA, October 1994. [19] A.D. Marshall. Developing hypertext courseware on the world wide web. In Proceedings of ED-Media 95: World Conference on Educational Multimedia and Hypermedia, pages 418{423, Graz, Austria, June 1995. [20] A.D. Marshall, S. Hurley, S.N. McIntosh-Smith, R.R. Martin, and N.M. Stevens. Novel uses of computers for teaching. AXIS: The UCISA Journal of Academic Computing and Information Systems, 1(3):30{41, 1994. [21] W. Montague and F. Knirk. What works in adult instruction: the management, design and delivery of instruction. International Journal of Educational Research, 19:327{433, 1993. [22] National Centre for SuperComputing Applications, World Wide Web Document URL: http://www.ncsa.uiuc.edu/SDG/Software/Mosaic/Docs/mosaicdocs.html. NCSA Mosaic Documentation, 1994. [23] Netscape Communications Corporation, World Wide Web Document URL: http://www.netscape.com. Netscape Home Page, 1995. [24] North Western Universty, World Wide Web Document URL:http://www.eecs.nwu.edu/unix.html. UNIX Reference Desk, 1995. [25] A.G. Romiszowski. Educational system design implications of electronic publishing. Educational Technology, 34(7):6{12, 1994. [26] D. Rowntree. Educational Technology in Curriculum Development. Paul Chapman, London, U.K., 1982. [27] A. Smeaton. Using hypertext for computer based learning. Computers in Education, 17(3):173{192, 1991. [28] Sun Microsystems Ltd., World Wide Web Document URL: http://www.sun.com/. Hot Java Home Page, 1995. [29] Technical University, Graz, Austria, World Wide Web Document URL: http://www.iicm.tu-graz.ac.at/. Hyper-G Gateway, 1995. [30] University of Hawaii, World Wide Web Document URL: http://www.hcc.hawaii.edu/guide/www.guide.html. Entering the World Wide Web: A guide to Cyberspace, 1994. 26
[31] University of Soutern Florida, World Wide Web Document URL:http://www.eecs.nwu.edu/unix.html. UNIX Programming Sites on the Web, 1995. [32] Virginia Tech University, World Wide Web Document URL: http://ei.cs.vt.edu/. Virginia Tech/Norfolk State University Home Page, 1995. [33] A.M. Zin and E. Foxley. Automatic program quality assessment system. In Proceedings of the IFIP Conference on Software Quality, S P University, Vidyanagarm India, March 1991.
27
