Int. J. Human—Computer Studies (1997) 47, 197—218
Experience with developing multimedia courseware for the World Wide Web: the need for better tools and clear pedagogy DAVID BENYON Department of Computer Studies, Napier University, 219, Colinton road, Edinburgh EH14 1LD, UK. email:
[email protected] DEBBIE STONE School of Health and Social Welfare, Open University, Milton Keynes MK7 6AA, UK. email:
[email protected] MARK WOODROFFE Department of Computer Studies, Open University, Milton Keynes MK7 6AA, UK. email:
[email protected]
The phenomenal growth of the Internet over the last few years, coupled with the development of various multimedia applications which exploit the Internet presents exciting opportunities for educators. In the context of distance education, the World Wide Web provides a unique challenge as a new delivery mechanism for course material allowing students to take a course (potentially) from anywhere in the world. In this paper, we describe our approach to the development of an Internet-based course designed for distance education. Using this experience, we provide general observations on the opportunities and constraints which the web provides and on the pedagogic issues which arise when using this delivery mechanism. We have found that the process of developing web-based courses is one area which requires careful consideration as technologies and tools for both the authoring and the delivery of courses are evolving so rapidly. We have also found that current tools are severely lacking in a number of important respects—particularly with respect to the design of pedagogically sound courseware. ( 1997 Academic Press Limited
1. Introduction The phenomenal growth of the Internet over the last few years, coupled with the development of various multimedia applications which exploit the Internet, presents exciting opportunities for educators. The opportunity for cheap, synchronous and international communications using both audio and video media (such as CUSeeMe, Share Vision, RealAudio, etc.) is one such development. The other main advance has been in the hypermedia capabilities offered by the World Wide Web. Across the world pages of notes, copies of overhead transparencies and other teaching materials have been made available to students. These materials frequently contain links which take the reader to other pages of related information. It is natural, then, that distance learning institutions should seek to exploit this technology—allowing students to study from anywhere in the world. Although a body of knowledge has accumulated concerning hypermedia and hypertext systems (e.g. Nielsen, 1995; Barker, 1993; 197 1071-5819/97/070197#22$25.00/0/hc 970126
( 1997 Academic Press Limited
198
D. BENYON E¹ A¸.
Cotton & Oliver, 1992; Bieber, Vitali, Ashman, Balasubramanian & Oinas-Kukkonen, 1997, this issue), there is still little specifically on the web as a teaching medium. In this paper, we reflect on our experiences in taking a traditional, distance learning course and transforming it into a course delivered over the World Wide Web. There are many lessons to be learned from this experience which can be applied to other forms of Internet-based education and, more generally, to the use of hypermedia in education. Even where the Internet is not used as the primary exposition of material (e.g. it could be used for tutorial, task-based teaching following a traditional lecture-based exposition) the issues concerned with integrating and structuring material for students to study on their own remain the same. Leaving aside discovery-based approaches to education, teaching is concerned with organizing and structuring material and facilitating a directed programme of study. We undertook our investigation in order to learn more about the benefits and problems of developing course material for delivery over the World Wide Web. The motivation for this development derived from three features of the web. First, students would be able to study the course from anywhere in the world, thus overcoming the restrictions imposed by the traditional delivery mechanism which requires a postal service. Second, we were interested in the possibilities afforded through hypertextualizing the materials and the greater flexibility of navigation which this would provide. Third, we recognized that a hypermedia version of the course would allow for a tighter integration of the various media employed in courses (such as text, video, interactive activities and so on) and increased interactivity. In traditional distance education, the various media are deliberately not tightly integrated so that the student has more opportunity to be flexible in when and where he or she studies. Whilst this has some benefits, it also means that the material has to be chunked into larger components. The tighter integration and increased interactivity offered by a hypermedia presentation may have important pedagogical benefits because it allows the student to go directly to any of the components for the information required. The context for our study was the Open University in the UK—a large and mature distance learning university. Established in 1969, it delivers some 200 courses across the curriculum at both undergraduate and masters level. The Open University has an annual intake of some 30 000 students and delivers materials to over 120 000 students. Although there are over 8000 associate lecturers dealing with the day-to-day delivery of material, these numbers mean that one-to-one tuition must be minimized. The philosophy of the Open University demands that courses should be open to students irrespective of their location or when they wish to study. Hence, there is very little synchronous communication embedded in our courses. However, Open University courses have always exploited multiple media in their materials. Whilst the basic media for most courses is text, this has usually been supplemented with audio, television and video materials, computer-based activities and home experiment kits as appropriate. The Open University has been quick to see the potential of the web as the basis for providing new and innovative ways of providing distance education (OU, 1996). However, whilst we have experience with developing hypermedia course materials and such experience has been gained elsewhere (e.g. Yazdani, 1993; Rada, 1995), we have yet to develop an understanding of using the web as the basis of our distance teaching. The European Commission (EC) has also recognized the urgent need to discover more about
DEVELOPING WEB-BASED COURSEWARE
199
the potential of this technology and supports the view that ‘‘. . .the mass market for educational multimedia—both products recorded on optical disks (CD-ROM and CD-i) and services which can be accessed by the telematic networks—cannot fail to grow rapidly in the mid-90s’’ (European Commission, 1996). The paper is organized as follows. In Section 2, we present the processes involved in courseware development. Section 3 describes the approach which we took to ‘‘hypertextualizing’’ a course in human—computer interaction. In Section 4, we discuss the facilities of existing software which we feel are important, and we conclude in Section 5 by looking at future developments in providing distance education on the World Wide Web.
2. The process of developing multimedia, networked courseware Developing multimedia, networked courseware requires the authors to undertake a number of related activities. Adapting the star model of Hix and Hartson (1993) to courseware development, we identify six main processes of course production: courseware specification, instructional design, multimedia design, integration, implementation and evaluation. Evaluation is central to our approach to multimedia courseware development; we accept the validity of the view that the question ‘‘is not whether to build a pilot system and throw it away. You will do that. The only question is whether to plan in advance to build a throwaway. . .’’ (Brooks, 1975). This prototyping approach to courseware development also implies that the designer may start at any point in the cycle, evaluate and continue development at any point. Courseware specification is the process of identifying the aims and objectives of the material, placing this in the context of the student population and their assumed previous knowledge and describing the detailed syllabus. Instructional design is concerned with the pedagogic approach taken to the courseware and is inevitably constrained by a number of factors such as time, money and the nature of the student population. However, it is also important to recognize that instructional design is constrained by both the delivery technology and by the authoring tools available—the technologies significantly affect the educational approaches which can be used. Multimedia development is concerned with selecting, designing and producing the multimedia components of the course. Whilst the instructional design is concerned with the approach and structuring of the course material, the media to be used to deliver the material require further consideration. The development team must consider a number of related features of the presentation of material such as the semantic content of objects, the channel used for communication, the temporal characteristics of the presentation, the granularity of the objects, the sequencing of display components and the coordination of the overall display (Kovacevic, 1992). In multimedia systems the user must be given some control over when and how information is presented (Muller, Farrell, Cebulka & Smith, 1992). The user needs to be able to state the equivalent of ‘‘slow down a minute’’ or ‘‘skip the details’’. Like a conversation, the user needs to be able to interrupt. This relationship between providing user control over the amount of detail required and the granularity of the objects is an important consideration in multimedia design (Blattner, 1992). Equally important is the overall consistency and look and feel of the whole interaction.
200
D. BENYON E¹ A¸.
The process of integration focuses on achieving a seamless and aesthetic combination of structure and presentation. It is insufficient to develop aspects of the courseware in isolation from each other. Graphic designers, video producers and software engineers need to work together with the courseware authors so that the whole course has a consistent look and feel, and hence navigation through the course material is intuitive and sensible in terms of the courseware and instructional design. Implementation concerns methods of instantiating the material on one or more platforms. Although there are now many cross-platform delivery mechanisms (such as web browsers), differences between platforms are still an important consideration for courseware designers. Several prototype systems may be implemented in order to assist with the evaluation of any aspect of the overall development. Once again, implementation imposes constraints on the use of multimedia and on the instructional design. Evaluation of all components is central. The aesthetics of the course need to be evaluated in addition to the specification, use of multimedia, implementation and instructional design. The evaluation methods chosen must be suitable for the purpose at hand (Preece, Rogers, Sharp, Benyon, Holland & Carey, 1994) and can include workshops, focus groups, questionnaires, expert evaluations and observation techniques. All components need to be evaluated with real learners where possible. Where this is not possible—either because such an evaluation is inappropriate or because it is infeasible—experienced tutors or past students of a similar course can play the role of real learners.
2.1. M867 USER INTERFACE DESIGN AND DEVELOPMENT
The course selected for delivery via the web was called ‘‘User-Interface Design and Development’’, course number M867. M867 is a post-graduate, 100 h course. Currently, the course consists of a set text (Preece et al., 1994), a study guide which directs the student through the set book and related material, a case study book and associated software simulation, another software system used to illustrate aspects of (generally poor) interface design and assessment material. The set book was developed from a previous version of the course and was written by experienced Open University lecturers. Accordingly, the set book is well structured and contains many self-assessment questions and exercises in tune with the general Open University pedagogic style which guides and supports the student in the learning process. This style has been developed through the experience of over 25 years of distance learning material. Unlike many approaches to computer-based learning it does not include automatic recording and marking of student answers. The nature of our students and the style of distance learning which we use means that such approaches are rarely useful. 1. Courseware specification. We took our courseware specification directly from the existing course. This provided clear learning objectives and sample assessment material which indicated the level of attainment which was expected from the students. 2. Instructional design. In a similar fashion, our instructional design derived from the existing course. Students were directed through the set book, case study and computer activities by a study guide which packaged the activities into eight units of approximately 12 h each (see Section 3 below).
DEVELOPING WEB-BASED COURSEWARE
201
3. Multimedia development. The multimedia aspects of the course were initially to be limited to those which already exist (text, graphics and computer-based activities). Our plans were to introduce further media over the coming years (speech, video and animation), but these were not our immediate concerns. We wished to keep the focus of our current activities restricted in order to provide opportunities for evaluation during the first presentation. 4. Integration. The close integration of the courseware components was the main focus of our work. We wanted to take the existing separate materials—the study guide, software activities, review questions, case study material and assessment material and provide and automate the links between these. We made the decision early on not to hypertextualize the whole of the set book (some 700 pages). Students would still be directed to read sections of this off-line. This decision was taken so that we could learn about the structure of the hypertext before tackling the whole of a large book. However, this is certainly one decision which we have since re-visited (see Section 3.4). 5. Implementation. Implementation was to be undertaken using the ‘‘second generation’’ web server, HyperWave (Maurer, 1996) and its associated ‘‘Hypercard’’ style system called HM-Card (Maurer, 1996). This restricted the use of the system to PC-based platforms, but we expected further platform support to be available in due course. 6. Evaluation. Various evaluations were undertaken, beginning with paper prototypes, explorations with alternative software systems and, finally, employing a focus group to critique the design. The system is currently being evaluated by real students.
3. Experience with developing a web version of M867 We began to hypertextualize the course by arranging for members of the development team, independently, to look at the existing materials and to identify a suitable structure for the course. Ginige, Lowe and Robertson (1995) suggest identifying the main nodes (or pages) which ‘‘contain an item of information that loses all usefulness if broken down any further’’. Nodes are then linked and structured into higher-level constructs. Linking can be through linear, hierarchical or network structures. They also identify three types of link: referential links which point to simple annotations of key terms, associative links which point to related nodes and structural links which provide a linear, hierarchical or networked structure. In M867, the structure of the course is provided by the study guide. This divides the course into eight ‘‘units’’ of approximately 12 h of study each. A typical unit description is shown in Figure 1. Icons are used to indicate the various activities and brief descriptions are provided of the various components. Chapter and Section references refer to Preece et al. (1994) (the set book). 3.1. CONVERTING FROM TEXT TO HYPERTEXT
The authors initially had different ideas concerning how the material should be structured. For example, should the linear nature of the study guide be maintained, or should the student be able to access the components in any sequence? The problem with such decisions is that the material will be used differently by different students and will be used differently by the same student at different times; whilst a linear structure may be
202
D. BENYON E¹ A¸.
FIGURE 1. Content of a typical unit description in the study guide.
DEVELOPING WEB-BASED COURSEWARE
203
appropriate for the student going through material for the first time, this may be inappropriate for the student revising the material. Conversely, if too much freedom is provided, then students may become bewildered and would lose the coherence of the original structure. Both Shneiderman (1997, this issue) and Smith, Newman and Parks (1997, this issue) discuss the problems of designing for users and tasks when there is such a variety of both. We explored these issues by constructing paper prototypes of possible links and structures and working through different scenarios trying to focus on the different requirements of different students and their activities. It soon became clear that the three types of link identified by Ginige et al. (1995) were too limiting. We wanted many more types of links. Moreover, these links should be strongly typed (i.e. different types of link would behave differently) and this behaviour should be signalled clearly to students. For example, we saw a great advantage could be gained from providing glossary links. Preece et al. (1994) already provide a glossary which could easily be incorporated, hypertextually, into the web version of the course. However useful glossaries may be, there is clearly a need for a fuller explanation of key terms and concepts. In the paper-based version of the course, it can be difficult for the student to identify where further explanations are to be found. For example, in teaching about HCI guidelines, we found that the word ‘‘guideline’’ is not in the course book glossary. However, according to the index, there is information about guidelines which extends over several pages and the required information is actually contained in two different paragraphs. We therefore identified the need for annotation links which would provide relevant but brief definitions taken from the course book. These links should ‘‘pop-up’’ rather than take the student to another node in the hypermedia structure. Such links need to be in-line so as not to disturb the natural flow of the text. These are what Nielsen (1996a) describes as ‘‘spring-loaded’’ modes which he defines as a mode ‘‘that persists only as long as the user explicitly does something to keep it active’’ (e.g. holding down a mouse button). The need for structural links were also apparent. The structure of the study guide provides natural anchors for links to the software activities and relevant portions of the case study and set book. (Recall that we did not intend having the whole of the set book on-line.) The review questions point naturally to the answers to review questions and these answers themselves point to smaller portions of the set book or to larger components of recommended reading. The overall course demonstrates the natural hierarchy of course to units to components of units which allowed us to identify the major entities of the course: the study guide had entities Overview, Aim, Objectives and Review Questions; the book was described in terms of the Glossary, Chapter Overview, Chapter Aims and Objectives, Example, Sections, Key Points and Interviews; and the case study had the entities Explanation and Interactive Activity. Instances of these entities provided the nodes of the course. Taking the example from Figure 1, Unit 2 Overview became node 2.1, Unit 2 Aims became node 2.2, Unit 2 Review Questions became node 2.5 and so on. Further down the hierarchy, the answer to review question 5 became node 2.5.5 and, where appropriate, portions of the set book (e.g. Section 3.2) became a lower-level node. This structure would be used for organizing the presentation of the material. A linear structure can be provided through ‘‘next’’ and ‘‘previous’’ buttons for those wishing to follow a sequential ordering and a network structure which enables browsing is provided by the structural links.
204
D. BENYON E¹ A¸.
When we turned our attention to the associative links it became clear that a whole range of links would be required. We felt that we would want to signal the existence of key concepts and relate these to examples, diagrams, discussions from various perspectives, applications, telling examples, themes, related areas of study and so on. Some of these materials would be quite closely related to the original concept, whereas others would be quite remote from the immediate material. However, they all provide useful and legitimate routes through the course materials. For example, a relational glossary (Woodman, 1995) is useful for providing nesting of key terms and links between the levels showing how concepts are developed and used by other concepts. Synonyms need to be signalled and explained. There is a need for an ‘‘assessment’’ view of the material and ‘‘revision’’ view ensuring that the major aspects can be distinguished from the more minor aspects. Figures 2 and 3 illustrate some of these different types of links which we identified. During the course, students are expected to submit three assignments. The assignments require students to analyse and criticize a computer-based system from an HCI perspective, suggest a re-design which will improve the human—computer interaction and then undertake an evaluation of their re-designed system. Many components of the course need to be brought together when preparing an assignment as illustrated in Figure 2. Before the final course examination, students will want to review all the material. Different approaches will be taken by different students; some students will start by looking over the aims and objectives of the course; other students will re-read their marked assignments, and concentrate on the comments that have been made by their tutor which may point to areas of weakness that they may feel they want to review. In this case, they could re-access the materials using the assessment view shown in Figure 2. Still other students will have their own personal revision strategy; e.g. if short of time for revision some students may only re-read the ‘‘Key points’’ summaries for the required chapters, and then spend some time looking at the case studies and their own system to
FIGURE 2. An assessment route through the course material.
DEVELOPING WEB-BASED COURSEWARE
205
FIGURE 3. Another possible assessment route through the material.
help them decide which information they feel they already know and what information they need to revise further. This type of approach is shown in Figure 3. It is important to recognize that the need for these different types of link and different routes through the material arose when we considered the design of a hypertextualized version of the course. The new medium provides new teaching methods and consideration of the characteristics of the hypermedia version of the course provided important pedagogical insights. Our experience as educators told us that the existing material allowed for flexibility in learning strategies. Any hypertextualized version of the course should facilitate and extend the facilities of the text-based course and should not restrict the students’ views of the material. By hypertextualizing a course, authors are imposing a particular structure (or several structures) on the course material. Text provides a flexible structure in which students can easily annotate and mark up alternative structures. Courseware developers need to ensure that their hypertext structures are just as malleable. In looking to extend the ‘‘functionality’’ of the existing course, we identified the need for students to build and develop their own conceptual maps (McAleese, 1994; Jonassen & Marra, 1994) of the material—effectively developing their own hypertextualization of the material. Various layering of help and advice was needed from the coarse-grained to the fine-grained. Of course, many of these issues are true of any educational hypermedia design. We found the process of hypertextualizing our existing course relatively straight-forward—due largely to the fact that the course was originally well-structured and had been developed with distance teaching in mind. Agreement on the initial conceptual design of the course (identifying the main entities, nodes and main links required) took 3 person days working in parallel, a day to meet and paper-prototype our ideas and another
206
D. BENYON E¹ A¸.
2 days to refine and document them. Having identified our requirements we turned to producing a first implementation of the course.
3.2. THE FIRST PROTOTYPE
The study guide translated easily to HTML using an automatic word-processor to HTML converter. However, the case study could not be easily implemented in HTML. The original case study book had been produced in full colour using a quality publishing package and contained over 20 full colour figures and some photographs. The colour coding of many of the tables in the original case study book would not easily convert into HTML. Accordingly, we had to find a system which maintained the original page layout and colour and decided to convert the case study book into Adobe Acrobat. Thus, we quickly found a problem with the existing World Wide Web tools. HTML on its own could not support the features which we required, in this case the carefully and professionally designed layout of the case study book. Although the Acrobat reader can be launched from a web browser, the student has another interface and another set of functions to understand which distracts from the fundamental teaching objectives. The ‘‘plug-in’’ approach to providing facilities not otherwise provided by a web browser results in increased complexity and a lack of consistency at the interface with the resulting usability problems. It also requires particular versions of the browser to be used which is not something that cannot be relied upon. Nielsen (1997) analysed the different browsers which accessed the www.useit.com site during December 1996 and found that 37 versions of Netscape, 12 versions of Internet Explorer and 19 other browsers had been used. The well-structured nature of the original course made identification of the nodes of the courseware relatively straight-forward. Each of the sections of the unit description in the study guide (see Figure 1) mapped onto a single node: the Introduction, Aims, Objectives, Reading, directions for doing the interactive activities, Review Questions and so on. However, identifying a sensible grouping of the nodes proved much more difficult. Because of the poor facilities in HTML to deal with different types of links, we found that in a number of places a link took users to a page with just a few lines on it. For example, the answer pointers to review questions often had only small amounts of text associated with them and on their own, out of context, these pieces of text meant very little. Figures 4 and 5 show an early prototype of the web pages for a review question and the answer pointer provided. Instead of jumping to a new page for the answer pointer, we wanted the answer pointer to be associated with the question. Although some control for this sort of design can be provided by Java, it is still necessary to launch and control a new window to provide the answer pointer. Frame-based web pages also offer a solution. However, frames can only be viewed using certain web browsers and have their own usability problems (Nielsen, 1996b). In particular, bookmarking of specific frames is not possible, so a student would not be able to save a link to a specific answer to a review question. The behaviour of browsers with frames creates usability problems. For example, the back button does not work in Netscape 2.0 (Nielsen, 1996b) and although the ‘‘Document Done’’ message is displayed at the bottom of Netscape 3.0 (Gold) it may still be loading one of the frames which makes it appear as if the system has crashed. Incorporating Java or frames into the courseware means taking the decision to develop
DEVELOPING WEB-BASED COURSEWARE
207
FIGURE 4. Example of initial implementation of review questions and link to answer pointer.
at the leading edge of high technology. This will, to some degree ‘‘future-proof ’’ the application, but will bar access for less technologically equipped students (Murray, Stone & Close, 1996). Thus, we found an important conflict between a ‘‘natural’’, or intuitive, hypertext design—one page equals one node—and the demands of good pedagogy. The traditional page design illustrated in Figures 4 and 5 is inadequate. The benefits of a hypertext structure which we wanted to exploit were that students could go directly to the details of any of the components without having to page through a lot of material. However, if the
208
D. BENYON E¹ A¸.
FIGURE 5. Example of initial implementation the answer pointer associated with the review question in Figure 4.
chunks which they go to are too small, important contextual information is lost. Although we recognize that using frames may help to alleviate this problem, the lesson for educational hypermedia developers is important. It is only once the conceptual course structure is realized in an implementation that the usability issues become apparent, and the web lags behind other hypermedia systems in many respects.
DEVELOPING WEB-BASED COURSEWARE
209
Shneiderman (1997, this issue) comments that web design is still at the ‘‘Model T’’ stage of development. In our context, we would argue that educational hypermedia is the model T; the web has some way to go before it reaches this stage.
3.3. THE SECOND PROTOTYPE
The experience of developing the first prototype demonstrated that ‘‘traditional’’ web technology was too much in conflict with the teaching requirements which we had. First, it could not provide typed links. Second, we could not control the size and placement of some of the nodes, e.g. everything had to be in a single page since pop-up, spring-loaded windows were not facilitated. Third, we could not provide the level of formatting which we wanted in terms of colour and screen layout. Accordingly, we decided to investigate a different technology—the ‘‘second generation’’ web solution provided by HyperWave (Maurer, 1996). HyperWave [Maurer (1996); also discussed under its old name of Hyper-G in Nielsen (1995)] is a web server which, amongst other things, helps to control different types of link and which can be used with a hypermedia authoring and presentation tool, HM-Card. HyperWave breaks away from the idea of having one-way single type links embedded in web pages; instead, links are maintained in an external link database. [Bieber et al. (1997, this issue) discuss these linkbases in more detail.] This means that links are bi-directional, can have various attributes (and hence can be typed) and can be automatically maintained. For example, if a cluster of web pages are defined as being in particular order, adding a new page will automatically put the page in its correct place in the sequence. Links can also come from read-only documents and from multimedia such as video. In the context of courseware, maintaining consistency is of particular concern as it is all too easy for links to become out of date or out of synchronization with each other. For example, if the course contains a video of someone speaking, with details of their affiliation, phone number and current job title, any change in circumstances must be consistently updated throughout the multiple links; maintenance of courseware is a serious issue. For the second prototype, the course was translated into HM-Card, a ‘‘Hypercard’’ style system which can be used as a stand-alone authoring/presentation tool or can be embedded within a HyperWave application. HM-Card provided greater control than the traditional web version and allowed much more freedom with respect to layout, choice of colour and so on. It also allowed us to design screens which included several nodes. Figure 6 shows a Unit header in the HM-Card version, and Figure 7 shows the answer pointer appearing underneath the relevant review question. HM-Card encourages modular development of hypertext systems and enables the author to control how nodes are accessed. By authoring the courseware in HM-Card, the course material can be integrated with the web—either at the level of a single page or at the level of a complete lesson—rather than the flat page, one-directional and single type of link which is available in HTML. HM-Card also provides a more sophisticated data model than HTML, allowing designers to control the appearance of screens and the navigational ability of users. For example, users can be forced into following a set sequence when desirable and can be allowed to navigate freely when desirable. Once again, although such facilities are now being provided by plug-ins to standard browsers
210
D. BENYON E¹ A¸.
FIGURE 6. A Unit header implemented in HM-Card.
(e.g. the HyperStudio plug-in for Netscape), we perceived the standardized interface provided by an integrated package to be important. Although our original plans were to put the HM-Card version of the course onto a HyperWave server, the problems which we have experienced in using HM-Card have rendered this aspect of the implementation unnecessary. The most significant aspect of using HM-Card was that the sequential control of the material could be more rigorously implemented. However, the software imposed some other restrictions, particularly in the types of link provided. HM-Card requires the programmer to group cards in a module, and then specify a navigational paradigm for the module. While control structures such as ‘‘next’’ and ‘‘previous’’ were provided, they only permitted navigation within a set of cards. (The version of HM-Card available did not permit hyperlinks which connect cards in different modules.) So if a module consisted of three cards, on starting from card 1 (displayed as 1 of 3 on the screen) pressing ‘‘next’’ moves to card 2, pressing ‘‘next’’ a second time moves to card 3, but pressing ‘‘next’’ a third time moves the user not on to the next module, as anticipated, but back to card 1 in the set. The design of the screens took much longer than expected (some 3 person months) and a number of the expected benefits were not realized. Notwithstanding these problems, a complete implementation was possible and the electronic version of the course is currently being studied by 10 students in nine different countries.
DEVELOPING WEB-BASED COURSEWARE
211
FIGURE 7. A review question and answer pointer implemented in HM-Card.
3.4. EVALUATION
A focus group evaluation of this version of the courseware was undertaken. The focus group consisted of four experienced tutors and two people, unfamiliar with the material, playing the role of student. It lasted just under 2 h. The evaluation was task-based, and participants were asked to familiarize themselves with the courseware by undertaking a set of exercises within 2]1 h sessions prior in preparation for the focus group meeting. The focus group discussions were centred around issues such as the usability and attractiveness (aesthetics) of the courseware; whether the integration of the various components was found to be helpful and whether the courseware was felt to be conducive to learning. In addition, the participants were also asked for their opinions and suggestions as to how the courseware could be improved. A variety of issues became apparent during this session. These varied in detail, ranging from features of individual screens to general principles. Whilst the courseware was found to be easy to use, it was not optimized for usefulness: it often did not allow users to do what they wanted to do in an intuitive and efficient manner. For example, the circular structure of the sections was found to be confusing. There was a lack of consistency, both in the look and feel across components and within components. Some common user-interface standards (e.g. the greying out of buttons) were not supported. There was a lack of a full and coherent overview of the
212
D. BENYON E¹ A¸.
material, a trail/or history mechanism, keyword searching, personalizable bookmarks and an ‘‘electronic highlight pen’’. Although these sorts of facilities are usually available in hypertext browsers such as MS Help, they were not easily implementable in HM-Card. Other problems concerned being able to display multiple windows and being able to see computer simulations at the same time as seeing the related text. The courseware was felt to be rather ‘‘clunky’’ and old fashioned (similar to 1970s CBT). Another poor feature was that it would help to have more guidance on where to go next at the end of each section. In a book this is self-apparent, as the next page is always immediately available and visible. The focus group was also asked whether or not they would actually use the courseware (given that there would, in the future, be an electronic, hypertextualized version of the book). Everyone said that they would use it occasionally, to answer specific questions, particularly if they were in a hurry; it provided a different way of looking at the information. It was felt that the addition of an electronic version of the book would make the courseware more useful, as long as it had a consistent interface, hyperlinked key aims and objectives, had appropriate navigation tools and was easier to use than the actual book. 3.5. SUMMARY
The purpose of this discussion is to illustrate the iterative nature of the courseware development and to highlight some of the pedagogic and aesthetic features which we found to be important in developing hypermedia courseware. Of course, many of the usability issues discovered in the second prototype can be overcome through a re-design of the screen layout. But this is not the central point. The two prototype implementations developed to date are both inadequate. Using the readily available HTML-based prototype, it is not possible to control the page sequencing and layout. Whilst the second prototype improved upon the first, it still did not provide the facilities demanded by the educational design. We are now engaged in the development of a third prototype. An evaluation of a hypertextualized version of the set book is underway (Wilkinson, Crerar & Falchikov, 1997) and we are awaiting the results of the student evaluation of the second prototype. Owing to the problems encountered with HM-Card, an early HyperWave implementation is now unlikely, although an option remains for the future. The emphasis of the third prototype will be on integrating the multimedia components, providing more guided tours of the material and exploiting the student—student and student—tutor communication aspects of courseware. Details of these developments are provided in Woodroffe, Stone and Carswell (1997).
4. The need for better tools The constraints imposed upon a course team by the lack of integrated and functionally adequate software tools—both in terms of authoring and in terms of delivery—remain a major obstacle in developing World Wide Web based courses. Furthermore, with web browsers and servers developing all the time, problems currently without solutions can soon become problems with solutions.
DEVELOPING WEB-BASED COURSEWARE
213
There are clearly ways around many of these problems. Unfortunately, trying to develop stable courses using evolving technologies is itself a major problem as different students have different facilities. In addition to the pedagogical considerations, developing at the leading edge of high technology will exclude some students, while adopting a ‘‘lowest common denominator’’ approach lacks real innovation. In this section, we outline some of the lessons which can be learned from the problems which we have encountered.
4.1. DELIVERY ISSUES
Although there are several competing hypermedia design methods and models (Bieber & Isakowitz, 1995), we have been unable to find any hypermedia design guidelines which are generally applicable as to how to organize instructional material. This is perhaps not surprising since the decisions are so dependent on the course material. For example, Rada (1995) comments that ‘‘a challenge to hypertext authors is to put just enough information on a node—not too much and not too little’’ (p. 22). We concur with this, but, in web-based implementations where students will be using different sizes of screen, controlling the screen layout is impossible. Although authors can decide on the amount of material on a node, controlling the presentation of this is outside their influence. Other difficult design issues concern the dividing up of HTML into separate files. We are not sure whether to provide the whole of the study guide (46 pages) as a single document, whether to divide it up by unit or whether to have each individual node as a file. Such design decisions have to be taken bearing in mind that many students will be studying the material from home and may wish to download large sections, whereas other students may wish to study on-line. Related issues such as the speed of on-line web usage have a major impact on the overall usability of the courseware. The effectiveness of any page design will depend on the browser, the student is using and on the size of the screen they have. Whilst many guidelines for web pages exist (e.g. Isaacs, 1996) as do guidelines for web sites (Shneiderman, 1997, this issue), it is difficult to see how general these can be. We would argue that we have followed Shneiderman’s advice and moved from the information entities to concrete implementations using user-centred evaluation; however, we have found that we cannot deliver. Given the constraint that different students have different browsers, we cannot control the colour of displays, the format of screens or many basic features of the layout of material.
4.2. AUTHORING ISSUES
Of course, we are not the only ones in recognizing the limitations of hitherto web technology. Nielsen (1996a) lists 15 features which are missing from current web browsers. Even his list would not seem to meet our needs (and these are requirements which we do not see as being in anyway unusual for web courseware). Java and the next generation of web browsers will bring animation and programmability to web pages. However, as courseware developers, we do not expect to program in order to author the facilities we require. As each new feature of the web arrives, we have to wait for appropriate authoring tools.
214
D. BENYON E¹ A¸.
The cost of authoring courseware is often quoted as 100 : 1—100 h for 1 h of student study time—and our experience of developing the HM-Card material would concur with that figure. The cost of developing and integrating multimedia is similarly expensive. Whilst it is relatively easy to prototype multimedia courseware using modern software packages, developing a fully robust, well-designed version and maintaining that courseware remains a labour-intensive activity. With web-based courseware development, the situation is exacerbated because of the lack of integration of high-quality courseware development tools with web authoring tools. Once again, we can expect this situation to get better in the next few years, but the problem of embedded links remains a longer-term problem. The issue of changing the whole paradigm of the web is addressed by Hill (1996) and by Andrews (1996). Both of these are interesting because they point a way forward. Hill describes how the Microcosm hypermedia system (Davis, Knight & Hall, 1994) deals with some of the issues of navigation and how a distributed link service can help. He comments that ‘‘Although it is possible to extend the features available [in standard web browsers] using CGI scripts. . . , HTML is not always an ideal presentation medium. . . , e.g. creating a tour through a set of arbitrary web documents cannot easily be offered. . .’’ Maintaining a suitable structure and constraining the students movement through certain aspects of the course material is exactly what educators want to do. Having external linkbases enable students to load different link sets over the same material for different tasks. In the context of the web, HyperWave is the only system to offer an opportunity to do this. It is unfortunate that the restricted facilities of the version of HM-Card proved too constraining for our purposes.
4.3. PEDAGOGICAL ISSUES
One of the principal lessons which we have learnt is that we are not yet convinced that there is any pedagogical benefit arising from simply hypertextualizing the existing course material. From Bush’s (1945) vision through the bundling of Hypercard with the 1980s Macintosh to SuperBook (Egan, Remde, Landauer, Gomez, Eberhart & Lochbaum, 1989) and modern day help systems, hypertext and hypermedia structures have been seen as something of a panacea for dealing with large quantities of information. The problem with courseware is that educators need to structure and organize the material in a way that is educationally meaningful and which provides real added value for the students. Printed text is highly flexible. Students have the ability to see the whole of the material and authors have the opportunity to signal the structure to students through chapters, units of study and so on. We believe that it is dangerous to think that there is educational benefit from simply hypertextualizing the existing material and putting it on the World Wide Web. This is not to say that there are no other benefits. For example, studying from a distance becomes easier, course materials can be quickly updated and students who move around may gain. However, for the great majority of our students, if the course is primarily text-based it may as well be presented in book form. We see the real added value of web courses coming from the multimedia and communication aspects more than from the hypertext aspects of the technology.
DEVELOPING WEB-BASED COURSEWARE
215
Hypermedia provides for a much tighter integration of media and increased interactivity, which, we believe could deliver important pedagogical benefits. Animation, video and audio are more pedagogically important than the simple hypertext structure. We also see important benefits coming from interactive use. Internet conferencing moderated by an experienced tutor is one such development. In this model, students will mail their comments and questions through on-line form-fill dialogues which a tutor will annotate with relevant links whilst the course is running. The course material therefore develops over a number of presentations with additional links and routes through the material. The AnswerGarden (Ackerman & Malone, 1990) is one such system which has been adapted to the web, as the AnswerWeb (Mayes & Neilson, 1995). AnswerWeb allows students to benefit from previous students’ questions and answers and has the potential to have a major pedagogical impact on web-based courseware. Coupled with personal workspaces for maintaining individualized guided tours, web links, etc., we see real educational benefit coming from this new communication medium. Another important insight provided by this experience is that we need to provide different and personalized versions of the course. We envisage a simply structured course aimed at the beginner and a range of more advanced features for experienced and/or enthusiastic students. The various types of links need to be consistently signalled to the students—but only if they want to see them. This leads naturally to the idea of an adaptive hypermedia system (e.g. Ho¨o¨k et al., in press), though to what extent it is user-controlled adaptivity or system-controlled remains to be investigated. For example, a direct manipulation, form fill object could be provided which would allow students to select the types of links they want to see and use. Thus, students will be able to develop their own profile of interaction with the material. As course designers, we supply a few generic profiles—basic, revision, assessment, etc.—which students can use or adapt as required. As with the aspects of control over the presentation of the material above, we expect this type of facility to be provided at the web server end rather than by the client or by the individual page. As the access to new technologies has increased, so has their consideration for use in teaching. It has been recognized that the Internet is suited to distance learning for such reasons as it is ubiquitous, flexible, timeless and interactive (Kenney, 1996). However, the bulk of the available information to do with Internet teaching concentrates not on the instructional design or teaching support aspects required for teaching and learning via the Internet but on the design of (web) pages themselves, and how to make them interactive for learning. In a summary of a workshop that discussed the role of the web in UK Higher Education, Mumford (1995) indicates that teachers will need to be helped with the pedagogic issues associated with the use of the web as a teaching and learning medium; Thomas, Carswell, Emms, Petre, Poniatowska and Price (1996) also cite the need for new teaching support structures for electronic course delivery via the Internet. In addition, Haugen and Ask (1996) have suggested that what is really needed is a new pedagogy for electronic learning environments. This view is shared by Paine and McAra (1993), who suggest that there will be no future for multimedia, other than in games software, unless the epistemological and pedagogical issues are thought through, and it can be demonstrated that deep learning can be achieved through this medium.
216
D. BENYON E¹ A¸.
5. Conclusion In line with the development model which drives our approach to the production of web courseware, it is worth reviewing how we have arrived at the conclusions above. We began with a high-quality tried and tested course. Rather naı¨ vely, perhaps, we thought that this would transfer easily to the web. When we turned our attention to the development of the instructional material and multimedia design for the web version, we realized that some of the benefits of the existing course would be lost if certain facilities were not available. We attempted to implement our course and soon realized that the existing tools do not support the demands of the course. The process of attempting to develop a web version of the course has led us to realize that simply hypertextualizing material is not enough. We want to exploit the hypermedia environment provided by the web by integrating the communications aspects, the multimedia aspects and the hypertext structure. Our experience, to date, then is a rather sobering one. There are serious usability issues concerned with the web which have yet to be tackled. The web is multilayered (unlike traditional hypermedia systems) with pages and sites, clients and servers and the telecommunications infrastructure, all contributing to the usability of the overall system. The web is slow. Students have a variety of browsers with different facilities. Students study in different environments. As educators, we need to control much of this so that we can teach, rather than letting students simply browse. However, we also want to exploit the flexibility and tailorability inherent in hypermedia in order to add real value to traditional distance learning. Collis (1996) believes that the web is the ‘‘killer application’’ (i.e. the one most likely to result in significant breakthrough and future usage) in relation to education. Although even we see great potential in the technology, we feel that we need better tools and a better understanding of the pedagogic impact which web-based courseware will have. We need high-level multimedia development tools which will enable us to develop courseware with a consistent look and feel. We need authoring and delivery tools which will allow us to control how the information is presented and we want to control the navigation between pages. We want to integrate personal and shared communication spaces in order to exploit the power of the medium: a tight integration of media, an opportunity to learn from colleagues and enhanced interactivity of the course. At present, the web does not provide us with these things. The lesson for educators using the web is that an incremental development approach, driven by sound pedagogy is vital. This work is supported by the project EONT: An Experiment in Open and distance learning using New Technologies funded by the EU’s SOCRATES programme under financial agreement TM-OP-1995-1-GR-88.
References ACKERMAN, M. & MALONE, T. (1990). Answergarden. A tool for growing organizational memory. In F. H. LOCHOVSKY, Ed. COIS-90: Proceedings of the Conference on Office Information Systems. Cambridge, MA: ACM Press. ANDREWS, K. (1996). Applying Hypermedia research to the World Wide Web [http://hyperg. iicm.tu-graz.ac.at/apphrweb/].
DEVELOPING WEB-BASED COURSEWARE
217
BARKER, P. (1993). Exploring Hypermedia. London: Kogan Page. BIEBER, M. & ISAKOWITZ, T., Eds. (1995). Designing hypermedia applications (special issue). Communications of the ACM, 38, 26—112. BIEBER, M., VITALI, F., ASHMAN, H., BALASUBRAMANIAN, V. & OINAS-KUKKONEN, H. (1997, this issue). Fourth generation hypermedia: some missing links in the World Wide Web. In S. BUCKINGHAM SHUM & C. MCKNIGHT, Eds. ‘‘¼eb ºsability’’ (special issue). International Journal of Human—Computer Studies. BLATTNER, M. (1992). Multimedia and Multimodal Interface design CHI ’92. Tutorial Notes. Cambridge, MA: ACM Press. BROOKS, F. (1975). ¹he Mythical Man Month. Reading, MA: Addison-Wesley. BUSH, V. (1945). As we may think. ¹he Atlantic Monthly, 176(1), 101—108. BYTE (1995). Hyper-G organises the ¼eb, pp. 59—64. COLLIS, B. (1996). ¹ele-learning in a Digital ¼orld: the Future of Distance ¸earning. London International Thompson Computer Press. COTTON, B. & OLIVER, R. (1992). ºnderstanding Hypermedia: From Multimedia to »irtual Reality. London: Phaidon Press Ltd. DAVIS, H., KNIGHT, S. & HALL, W. (1994). Light Hypermedia link services: a study of third party application integration. In Proceedings of the 6th ACM Conference on Hypermedia, pp. 41—50. Edinburgh, Scotland: ACM Press. EGAN, D., REMDE, J. R., LANDAUER, T. K., GOMEZ, L. M., EBERHART, J. & LOCHBAUM, C. C. (1989). Formative design-evaluation of SuperBook. ACM ¹ransactions on Office Information Systems. New York: ACM Press. EUROPEAN COMMISSION (1966). Report of the ¹ask Force ‘‘Educational Software and Multimedia’’, SEC (96) 1426. Available at [http://www2.echo.lu/mes/en/report796-toc.html]. GINIGE, A., LOWE, D. B. & ROBERTSON, J. (1995). Hypermedia authoring. IEEE Multimedia, 2. HAUGEN, H. & ASK, B. (1996). Material for IT-based open and distance learning. In ¹echnology and Communications: Catalyst for Educational Change, Vol. 2. IC¹E New Orleans ’96: Proceedings of the 13th International Conference on ¹echnology and Education, New Orleans, 17—20 March, Vol. 2, pp. 508—510. TX: International Conferences on Technology and Education, Inc. HILL, G. (1996). Extended linking facilities for the WWW. In Proceedings of the Hypermedia Research and the ¼orld ¼ide ¼eb ACM Hypertext ’96, Washington, DC. HIX, D. & HARTSON, H. R. (1993). Developing ºser Interfaces: Ensuring ºsability ¹hrough Product and Process. New York, NY: Wiley. HO®O®K, K., KERLGREN, J., WAERN, Y., DAHLBA®CK, N., JANSSON, C. -G., KARLGREN, K. & LEMAIRE, B. (in press). A glass box approach to adaptive hypermedia. ºser Modelling and ºser Adapted Interaction, Vol. 6. ISAACS, M. (1996). Guide to good practice for www authors. Advisory group on Computer Graphics, SIMA Report Series Report 19. JISC New Technologies Initiative. JONASSEN, D. H. & MARRA, R. M. (1994). Concept mapping and other formalisms of Mind tools for representing knowledge. Alternative ¸earning ¹echnologies Journal, 2, 50—56. KENNEY, J. (1996). Integrated internet services for in service training. In ¹echnology and Communications: Catalyst for Educational Change, Vol. 2. IC¹E New Orleans ’96: Proceedings of the 13th International Conference on ¹echnology and Education, New Orleans, 17—20 March, Vol. 2, pp. 452—454. TX: International Conferences on Technology and Education, Inc. KOVACEVIC, S. (1992). A compositional model of human-computer dialogues. In M. BLATTNER & R. B. DANNENBERG, Eds. Multimedia Interface Design, pp. 373—404. New York, NY: ACM Press. MCALEESE, R. (1994). A theoretical view on concept mapping. Alt-J, 2, 38—48. MAURER, H. (1996). Hyper¼ave ¹he next Generation ¼eb Solution. Reading, UK: AddisonWesley. MAYES, J. T. & NEILSON, I. (1995). Learning from other people’s dialogues: questions about computer-based answers. In B. COLLIS & G. DAVIES, Eds. Innovative Adult ¸earning with Innovative ¹echnologies (A-61). North-Holland: Elsevier.
218
D. BENYON E¹ A¸.
MULLER, M. J., FARRELL, R. F., CEBULKA, K. D. & SMITH, J. G. (1992). Issues in the usability of time-varying multimedia. In M. BLATTNER, & R. B. DANNENBERG, Eds. Multimedia Interface Design, pp. 7—38. New York, NY: ACM Press. MUMFORD, A. (1995). The World Wide Web—a strategic tool for UK higher education. In J. DARBY & J. MARTIN, Eds. ºsing the Internet for ¹eaching. Active Learning. Oxford: CTISS Publications. MURRAY, P. J., STONE, D. K. & CLOSE, J. S. (1996). ¹he impact of multimedia in health promotion: the national and international distance education perspective. A report of a research overview for the Health Education Authority (HEA). To be presented at Multimedia—Interacting for Health? an expert seminar to be held on 6 February 1997. NIELSEN, J. (1995). Multimedia and Hypertext: the Internet and Beyond. London: Academic Press Professional. NIELSEN, J. (1996a). Features missing in current Web browsers. In SºN Microsystems ¼hat’s Happening. . .Columns and Commentary. [http://www.sun.com/950701/columns/alertbox/ newfeatures.html]. NIELSEN, J. (1996b). ¼hy Frames Suck (Most of the ¹ime) [http://www.useit.com/alertbox/ 9612.html]. NIELSEN, J. (1997). Browser ºsage Statistics. [http://www.useit.com/alertbox/browsers—dec96. html]. OU (1996) [http://www.open.ac.uk/]. PAINE, N. & MCARA, P. (1993). Interactive multimedia technology: a summary of current developments. In C. LATCHEM, J. WILLIAMSON & T. HENDERSON-LANCETT, Eds. Interactive Multimedia: Practice and Promise. London: Kogan Page. PREECE, J. J., ROGERS, Y. R., SHARP, H., BENYON, D. R., HOLLAND, S. & CAREY, T. (1994). Human—Computer Interaction. Reading, MA: Addison-Wesley. RADA, R. (1995). Interactive Media. New York, NY: Springer. SHNEIDERMAN, B. (1997, this issue). Designing information-abundant websites: issues and recommendations. In S. BUCKINGHAM SHUM & C. MCKNIGHT, Eds. ‘‘¼eb ºsability’’ (special issue) International Journal of Human—Computer Studies. SMITH, P. A., NEWMAN, I. A. & PARKS, L. M. (1997, this issue). Virtual hierarchies and virtual networks: some lessons from hypermedia usability research applied to the World Wide Web. In S. BUCKINGHAM SHUM & C. MCKNIGHT, Eds. ‘‘¼eb ºsability’’ (special issue) International Journal of Human—Computer Studies. THOMAS, P., CARSWELL, L., EMMS, J., PETRE, M., PONIATOWSKA, B. & PRICE, B. (1996). Distance education over the Internet. Technical Report No. 96/02, Computing Department, The Open University, Milton Keynes. WILKINSON, S., CRERAR, A. & FALCHIKOV, N. (1997). Document structure, individual differences and the learning process. Proceedings of Hypertext ’97, April, Southampton. Also available at [http://www.dcs.napier.ac.uk/hci/simon/Research— Details.html]. WOODMAN, M. (1995). Relational glossary. . .Open University Computer Department Technical Report number. WOODROFFE, M. R., STONE, D. K. & CARSWELL, L. (1997). [http://mcs.open.ac.uk/eont]. YAZDANI, M. (1993). Multilingual Multimedia. Oxford: Intellect.
