computer support for multimedia curriculum design - CiteSeerX

COMPUTER SUPPORT FOR MULTIMEDIA CURRICULUM DESIGN

Qiyun Wang

CIP-GEGEVENS KONINKLIJKE BIBLIOTHEEK, DEN HAAG Wang, Qiyun Computer support for multimedia curriculum design Thesis University of Twente, Enschede - With refs. - With summary in English and Dutch. ISBN 90-365-1670-0 Lay-out: Sandra Schele Press: PrintPartners Ipskamp - Enschede © Copyright, Qiyun Wang, 2001 All rights reserved. No part of this book may be produced in any form: by print, photoprint, microfilm, or any other means without written permission of the author.

COMPUTER SUPPORT FOR MULTIMEDIA CURRICULUM DESIGN

PROEFSCHRIFT

ter verkrijging van de graad van doctor aan de Universiteit Twente, op gezag van de rector magnificus, prof.dr. F.A. van Vught, volgens besluit van het College voor Promoties in het openbaar te verdedigen op woensdag 24 oktober 2001 om 15.00 uur

door Qiyun Wang geboren op 5 februari 1970 te Shandong, China

Promotor

: Prof.dr. J.J.H. van den Akker

Assistent-promotor : Dr. N.M. Nieveen

i

Table of Contents

PREFACE 1

INTRODUCING THE CASCASE-MUCH STUDY

1.1 Context of the study 1.1.1 Curriculum development in China 1.1.2 Curriculum development in Shanghai 1.2 Origins of the study 1.2.1 The project of multimedia curriculum for Biology (MCB) 1.2.2 The CASCADE study 1.3 Overview of the study 1.3.1 Intended target users 1.3.2 Aims of the program 1.3.3 Methodology: Focus on development research 1.3.4 Approach of the prototyping stage 1.3.5 Approach of the assessment stage 1.4 Preview of the dissertation

2

MULTIMEDIA CURRICULUM AND COMPUTER SUPPORT SYSTEMS

2.1 Multimedia 2.1.1 Definitions of media and multimedia 2.1.2 Media and learning 2.1.3 Multimedia and learning 2.2 Curriculum development 2.2.1 Concepts of curriculum 2.2.2 Curriculum development models 2.3 Multimedia curriculum development 2.3.1 The definition of multimedia curriculum 2.3.2 Multimedia curriculum modules 2.3.3 Multimedia curriculum development model

vii

1 1 1 4 9 9 11 12 12 13 16 17 19 20

21 21 21 22 24 26 27 30 32 32 34 36

ii

2.4 EPSS 2.4.1 The concept of EPSS 2.4.2 The potentials of EPSS 2.4.3 The design of EPSS 2.5 Existing computer support systems for curriculum development 2.5.1 CASCADE 2.5.2 CASCADE-SEA 2.5.3 Other support systems 2.5.4 Comparison of the support systems and implications on the design of the study 2.6 Conclusions

47 51

3

53

PROTOTYPE DEVELOPMENT

3.1 Preliminary choices 3.2 Overview of the prototyping process and the prototypes 3.2.1 Prototyping process 3.2.2 Evolution of the structure and the four components 3.3 The first prototype 3.3.1 Content 3.3.2 Support 3.3.3 Interface 3.3.4 Expert appraisal and micro evaluation 3.4 The second prototype 3.4.1 Content 3.4.2 Support 3.4.3 Interface 3.4.4 Scenario 3.4.5 Expert appraisal and micro evaluation 3.5 The third prototype 3.5.1 Content 3.5.2 Support 3.5.3 Interface 3.5.4 Micro evaluation 3.5.5 Expert appraisal at the UT 3.6 The fourth prototype 3.6.1 Content and support 3.6.2 Expert appraisal at the ECNU

37 37 39 41 43 43 44 46

53 55 55 56 64 64 68 69 71 76 76 79 79 81 82 88 88 90 92 94 101 105 105 106

iii

4

DESCRIPTION OF THE FINAL VERSION

113

4.1 Overview of the program 4.2 Content 4.2.1 Goal and usage analysis 4.2.2 Learner analysis 4.2.3 Content selection 4.2.4 Content representation 4.2.5 Content organization 4.2.6 Interface design 4.3 Support 4.3.1 Information 4.3.2 Advice 4.3.3 Tools 4.3.4 Training 4.4 Interface 4.4.1 General characteristics 4.4.2 Screen design 4.5 Scenario 4.5.1 General characteristics 4.5.2 Components

113 115 116 117 119 121 122 125 126 127 128 132 133 134 134 134 139 140 140

5

143

ASSESSING THE PRACTICALITY OF THE PROTOTYPE

5.1 Introduction 5.2 Design of the assessment studies 5.2.1 Participants 5.2.2 Procedures and activities 5.2.3 Data collection and analysis 5.3 Results with primary target group users 5.3.1 Perceived practicality of the four components 5.3.2 Actual use of the program 5.3.3 Comments and suggestions 5.4 Results of study with other users 5.4.1 Perceived practicality of the four components 5.4.2 Actual use of the program 5.4.3 Expected extension to other subjects 5.4.4 Other comments and suggestions 5.5 Conclusions

143 144 144 146 148 150 150 154 159 161 161 164 166 168 170

iv

6

DISCUSSION

171

6.1 Introduction 6.2 Discussion of the main findings 6.2.1 Content 6.2.2 Support 6.2.3 Interface 6.2.4 Scenario 6.2.5 Functionality for various user contexts 6.3 Discussion of the development research approach 6.3.1 Prototyping 6.3.2 Formative evaluation 6.3.3 Design principles 6.4 Recommendations 6.4.1 Web support 6.4.2 A follow-up study 6.4.3 Implementation 6.5 Closing remarks

171 172 172 175 177 179 181 184 184 186 188 190 190 191 192 193

REFERENCES

195

ENGLISH SUMMARY

205

NEDERLANDSE SAMENVATTING

213

APPENDICES A-1 A-2 A-3 B C D E F G H

Screen dumps of Main Frame (English and Chinese version) Screen dumps of Designer's Aid (English and Chinese version) Screen dumps of Edit Panel (English and Chinese version) Example of an instructional scenario Interview topic list (used during the second round of prototyping) Instruments for the micro evaluation (used during the third round of prototyping) Instruments for the expert appraisal at the UT (used during the third round of prototyping) Questionnaire for the assessment studies Example of interface styles Original data collected from the assessment studies

221 227 245 249 255 263 267 275 281 283

v

LIST OF FIGURES 1.1 1.2 1.3 1.4 2.1 2.2 3.1 3.2 3.3 3.4 4.1 4.2 4.3 4.4 4.5 5.1 5.2 5.3 5.4 5.5 6.1 6.2

The administrative framework of curriculum innovation in Shanghai 6 Line of reasoning for this study 11 Multimedia curriculum development process 14 Multimedia curriculum design process with CASCADE-MUCH 15 Difference between a multimedia curriculum (left) and a conventional curriculum (right) 33 Multimedia curriculum development model 36 The prototyping process 55 A screen dump of Designer's Aid 70 A screen dump of Designer's Aid in the first prototype 80 A screen dump of Edit Panel 93 Overall structure of the CASCADE-MUCH program 114 Overview of Designer's Aid 115 Linear content organization 123 Non-linear content organization 124 Integrated content organization 124 Means of answers given by the novice and experienced designers 154 Examples of walking routes 155 Time spent on each screen 157 Illustration of means 164 Average time spent on each screen 165 Possible extension of the program 181 Assessment of the effectiveness of the program 192

LIST OF TABLES 2.1 2.2 2.3 3.1 3.2 3.3 3.4 4.1 4.2 4.3 4.4 5.1 5.2

An example of matrix media-selection model (Allen, 1967) Curriculum components proposed by various authors The comparison of the compute support systems Overview of the prototypes The relationship among the components, quality and participants Modules of a multimedia curriculum Characteristics of the subject teachers (n=7) The interrelationship between the analysis and the design elements The explanation of the menu Presentation forms and screen elements for content representation The factors used for describing each knowledge unit General characteristics of the participants in study 1 (n=6) General characteristics of the participants in study 2 (n=13)

24 28 49 59 83 90 95 131 136 138 142 145 146

vi

5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10

Overview of instruments and data analysis Perceived practicality of the content (n=6) Perceived practicality of the support Perceived practicality of the interface (n=6) Perceived practicality of the scenario (n=6) Support tools utilized by the participants Perceived practicality of the four components by other users Support tools utilized by other users

148 150 151 152 153 158 162 166

vii

Preface

I still clearly remember when I studied at the primary school of a small village in China, one of our teachers said "In about 20 years, the new millenium will come. At that time you will be around 30 years old and become talented persons." I could not imagine where I would be and what I would be doing in the year 2000, but I was filled with beautiful hopes. Now, with the year 2000 just passed, a hope will soon become a reality. Through more than four years of struggling, I have successfully finished the Ph.D. study and written the book in my second language -- English. Looking backward, carrying out the Ph.D. study was a real challenge for me. Without the help of others, it would have been impossible for me to finish it. Here I would like to express my sincere thanks to those people who gave me support during the study. First of all, I would like to thank the two promoters: Prof. dr. Jan van den Akker and Dr. Nienke Nieveen. They supported me during the whole process of conceptualization, data collection and analysis, and writing the thesis. Their valuable comments and suggestions always helped me go forward in the right direction. Furthermore, their support and encouragement made me feel confident to pursue the study. In particular, Nienke Nieveen gave up much pleasant time with her lovely young daughter, but spent it in reading and making comments and suggestions on this thesis, so as to make it like it is. I also would like to thank Prof. dr. Tjeerd Plomp and Prof. dr. Zhiting Zhu. It is they who provided me with this opportunity to be able to study at the University of Twente as a Master student and then as a Ph.D. candidate. During the process of the Ph.D. study, they often stimulated me and gave me support. Particularly, Prof. Zhiting Zhu helped me organize an expert appraisal workshop in Shanghai while he was fully engaged.

viii

Preface

I also want to thank Prof. dr. Pløn Verhagen. His encouragement and humor stimulated me to treasure the opportunity of the study. Several years ago, he spent his valuable time in making comments and suggestions on the initial ideas for my thesis during his visit to Shanghai. A few months ago, he also made some constructive comments and suggestions on the final version of my thesis. I really appreciate his help. Furthermore, I would like to thank all participants who made contributions to the formative evaluation activities both in Shanghai and in the University of Twente, the Netherlands. Also, thanks are going to Kevin McKenney for his effort to polish and improve my English writing; and Sandra Schele for her delicate formatting of the book. In addition, I want to thank my parents who brought me into the world fortunately (I am the seventh and also their last child), and educated me to be diligent. Their love, encouragement, support and understanding always inspire me to push forward. Finally, wholehearted thanks are given to my wife Qingxia Yan and my son Kevin Wang for their love. My wife always took care of my personal life and helped me arrange everything in order so that I had enough time to work on the study. Although my son's arrival in the world during my Ph.D. study brought us extra work and a new challenge, she did and also does an excellent job, taking care of our son and making him grow healthily. Cheerfully, my son's birth makes me feel happy and proud, and fills me with energy and pleasure. Playing with him after a lengthy time of working became a great delight in my life. I hope that the completion of the Ph.D. study is not only a finishing act of the former academic carrier, but also a starting point of a new life in the future.

Enschede, September 2001

1

Chapter 1 Introducing the CASCADE-MUCH study

T

his chapter introduces the study on "Computer ASsisted Curriculum Analysis Design and Evaluation - MUltimedia curriculum design in CHina" (Acronym: CASCADE-MUCH). The contextual information of curriculum development in China/Shanghai is introduced in Section 1.1. Section 1.2 provides the origins of the study. An overview of the study and a preview of the dissertation are given in sections 1.3 and 1.4 respectively.

1.1 Context of the study The CASCADE-MUCH study took place in the context of a major curriculum innovation in Shanghai, China. This section first presents the overall context of curriculum development in China, and then focuses in particular on the curriculum development within the Shanghai region. 1.1.1 Curriculum development in China

Educational system China is a socialist country, which was founded in 1949 by the Chinese Communist Party. Geographically, China is the third largest country and her population of 1.2 billion is the world largest. China is a multi-ethnic country. In addition to Han Chinese, who constitute about 90 percent of the population, there are fifty-six minority groups, each with its own language and culture. Since the economic reform and open-door policies established in 1978, an ongoing educational reform --mainly referring to the educational structure-has been taking place in China. Nowadays, China is popularizing a schooling system following a 6-3-3 structure. Learners study at primary schools, junior secondary schools and senior secondary schools for six, three and three years respectively. The first nine years in primary and junior secondary schools are a

2

Chapter 1

compulsory educational period. Learners typically start their primary schooling at six years of age; before that they usually have received four years of preprimary education. After the nine-year compulsory education, learners can choose to continue their studies in senior secondary schools or in vocational schools. If they choose senior secondary school studies and go on to pass the university entrance examinations, they may enter universities (or colleges) for another four years (or three years) of studies. The university system has a 4-33 structure, composed of four-year undergraduate, three-year master and three-year Ph.D. studies. In the last twenty years, Chinese education has made great progress in terms of the number of learners who actually go to school. According to Chen (1999), minister of the Ministry of Education (MOE), by 1997, the regions making up 65% of the total population have popularized the nine-year compulsory education. The rate of entering primary schools has reached 99%; the rate of entering junior secondary schools has reached 87%; and the illiteracy rate of adults has dropped below 6% (about a 4% drop from 1992). In addition, in 1997 there were 8.5 million senior secondary school students. The rate of entering senior secondary schools has risen from 26% in 1990 to 41% in 1999. In 1997, there were about 6.1 million university students including 180 thousand graduate students. The numbers are 2.2 and 9.6 times greater than those in 1979 respectively.

Administrative framework Curriculum development in China used to be a highly centralized task carried out solely by the State Educational Commission (SEC), a ministry of the State Council. Within the SEC, the Department of Secondary and Primary Education was directly responsible for finalizing the General Teaching Outline (the national curriculum), specifying curriculum organization and the timetable arrangement for all school subjects. Once the General Teaching Outline was finalized, it was People's Educational Press (PEP) that developed and published the ready-to-use school curricula. The PEP was a specialized publishing house directly under the leadership of the SEC. The main task of the PEP was to carry out curriculum studies, as well as to develop, publish and distribute lesson materials --including both teaching and learning materials.

Introducing the CASCADE-MUCH study

3

In 1998, with the organizational reform of the State Council, the SEC changed its name to MOE. According to the Educational Act, the MOE is responsible for: i) planning and managing educational matters all over China; and ii) making overall arrangements for the development of educational undertakings. The Department of Basic Education within the MOE is specifically in charge of curriculum reform planning for primary and secondary schools, as well as the evaluation of the nation-wide curricula. The PEP is still responsible for the development, publishing and distribution of lesson materials. Furthermore, each province or district has a department of education, which is mainly responsible for local management of educational matters. Sometimes they may also develop some complementary lesson materials for their local use.

Main features and existing problems According to Goodlad (1994) and van den Akker (1998), curriculum development activities often take place at three different levels: societal or system level ('macro'), institutional or school level ('meso'), and classroom level ('micro'). However, in China before 1988, the curriculum development was usually undertaken only at the societal or national level (You, 1998; Wang, 1992a). The general teaching outline was determined by the SEC, and the concrete lesson materials were developed by the PEP, both at the national level. Although the curricula were allowed to have local alterations, curriculum development activities at school and classroom levels were less common (cf. Yat-ming, 1991). Since 1988, the SEC has attempted to change this situation by encouraging curriculum development at different levels. Four categories of curricula have been planned to be developed oriented to: 1. nation-wide regions and schools with normal conditions; 2. economically more developed regions and schools with better conditions; 3. economically less developed regions and schools with poorer conditions; and 4. minority regions. The first category of curricula is developed at the national level, while the other three categories are developed by local institutes (cf. You, 1998). Despite the attempts of the SEC, it appears that the overall curriculum development process has not changed much. Most activities of curriculum development are still carried out at the national level, and most primary and secondary schools all over the country are still using the nation-wide curricula.

4

Chapter 1

However, the nation-wide curricula developed by the PEP are often criticized by local teachers or educators for having the following shortcomings (cf. You, 1998; Wang, 1992b). First, they are oriented to the average conditions all over the country; it is hard for them to cover specific needs of various regions. The imbalance of economic development between coastal and inland regions implies that these 'one-size-fits-all' curricula cannot work well. Second, almost all curricula, even starting from primary and junior secondary schools, are designed for those learners who aim to enter higher grades. The overemphasis on entering higher grades too early is not good for learners' full development. Third, the nation-wide curricula are considered to be overly catering to testdriven education. The content and instructional strategies are mostly oriented to passing tests and entering higher grades. They overemphasize (Wang, 1992b): theory but ignore application; cognitive knowledge but ignore skills; test scores but ignore creativity development; and science but ignore arts. The development of learners' full qualities, such as physical education, morality and creative thinking skills, is not sufficiently supported in the nation-wide curricula. Generally speaking, Chinese education has made great progress in the last twenty years, particularly in the structural reform and the number of learners who can go to schools. But less progress of reform on content and instructional strategies has been achieved. Although the nation-wide curriculum can ensure academic quality in general, it has been found to have some shortcomings for specific situations in various regions as well as for quality development of learners. 1.1.2 Curriculum development in Shanghai

Introduction Shanghai is the largest city in China. According to the latest census finished in 2001, Shanghai has more than sixteen million citizens allocated in sixteen districts and four counties. It is also one of the more advanced regions both economically and educationally. In 1988, Shanghai was mandated to make a trial curriculum innovation by the former SEC. During the trial curriculum innovation process, Shanghai was responsible for development of new curriculum materials for economically more developed regions (Category 2), including Shanghai rural and suburb districts as well as some other coastal


5

regions such as in Jiangsu and Zhejiang provinces. By 1997, Shanghai had finished its first round of curriculum innovation and started proceeding with the second round. So far, the Shanghai curriculum innovation has achieved several fruitful outcomes. The newly developed curricula, curriculum standards, and learning and teaching materials for all subjects and for all grades are being fully implemented in the whole city (Zhang, 1999). However, the other economically more developed regions have not yet adopted the newly developed Shanghai curricula. One of the main reasons might be the fact that Shanghai has had the power to develop its own university entrance examinations from the beginning of the innovation. In this way, they are able to make the university entrance examinations consistent with the intentions of the new curriculum. Although the other regions are encouraged to implement the new curriculum materials, they are not allowed to adapt their own university entrance examinations. In China, university entrance examination has always been considered to be a 'baton of orchestra', playing a very important role for teaching and learning. In the regions outside Shanghai, schoolteachers are reluctant to use the newly developed curriculum materials because the students have to pass the national examinations, which are still made based on the nation-wide curricula.

Administrative framework The administrative framework of curriculum innovation in Shanghai is illustrated in Figure 1.1. The Shanghai Educational Committee is in charge of the curriculum innovation activities as well as other educational matters. It is headed by the Shanghai local government and the MOE. The curriculum innovation activities are mainly carried out by the Curriculum Innovation Committee, which is a department of the Shanghai Educational Committee. Two departments are involved in the Curriculum Innovation Committee: an office and a commission for development and evaluation of lesson materials. The office is mainly responsible for organizational work, and the commission is responsible for the development and evaluation of specific curriculum materials. For each subject, there is a development and an evaluation group. In general, the development groups consist of subject matter experts and experienced teachers, whereas the evaluation groups are mainly composed of subject matter experts and administrative leaders. In addition, within the Shanghai Educational Committee there is a test department, which is responsible for making sure that tests are consistent with the newly developed curriculum materials.

6

Chapter 1

In Shanghai, the general curriculum development approach is that the Curriculum Innovation Committee develops a general curriculum standard for all subjects. The commission for development and evaluation of lesson materials makes a specific and detailed curriculum standard for each subject based on the general curriculum standard. The development group of each subject develops lesson materials based on the specific curriculum standard for that subject. The evaluation group is responsible for evaluating the quality of the developed materials. Shanghai local government

MOE (Ministry of Education)

Shanghai Educational Committee Curriculum Innovation Committee Office

Test department

Commission for development and evaluation of curriculum materials Development group for each subject

Evaluation group for each subject

Figure 1.1: The administrative framework of curriculum innovation in Shanghai

The first round of curriculum innovation The first round of curriculum innovation started in 1988 and was completed in 1997. The main aim was to bring 'two changes and three breakthroughs' (Yuan, 1990, p. 60). The two changes referred to: a shift in the focus of the curriculum content from entering higher grades towards quality development of learners; and a shift in instructional strategies from being inert towards being more (inter)active. The three breakthroughs were to result in: lessening learners' burden, improving learners' skills, and improving learners' full qualities and personality.


7

According to Wang (1992a), director of the Curriculum Innovation Committee at that time, the basic tasks of the first round of the curriculum innovation were to develop: a general curriculum standard for all subjects and a specific curriculum standard for each subject; new lesson materials for primary and secondary schools; and some complementary lesson materials including computer based learning (CBL) software. In the first two years following 1988, the Curriculum Innovation Committee worked on the analysis and planning for the curriculum innovation. At the beginning of 1990, a general curriculum standard for all subjects and a specific curriculum standard for each subject were formulated. From then on, lesson materials for each subject have been successively developed and implemented. The new curricula were composed of three modules: required courses, optional courses and active courses. In order to lessen learners' burden, the new curricula reduced learning time for the required courses, and added more learning time for the optional and the active courses. In the beginning, only some trial primary schools used the new curricula. Some subjects, such as Chinese language, used two different textbook versions developed by two different development teams. Gradually, the new curricula were used in an increasing number of primary and secondary schools with the students entering higher grades, and each subject finally settled on a single version of textbook. By 1997, all primary and secondary schools in Shanghai were using the new curricula. The last task of the curriculum innovation was to develop and use computer based learning software in order to make the instructional strategies more active and/or interactive. It was expected that all district educational colleges, secondary schools, as well as universities, could make contributions to the development of CBL software.

The second round of curriculum innovation The second round of curriculum innovation has been going on since 1997. In the first two years after 1997, a plan of primary and secondary curricula for the new century (2000-2010) was formulated, based on the guideline that the second round of curriculum innovation would focus on improving learners' creative thinking skills and inquiry skills (SSCRCO & SECIRO, 1999). The plan included a general blueprint for the second round of curriculum innovation and specific curriculum innovation guidelines for each subject. For the sake of

8

Chapter 1

brevity, this plan will henceforth be called the 'second curriculum plan'. According to this second curriculum plan, the innovation should have the following characteristics. First, the new curricula would be classified into three new modules: basic courses, extended courses and research courses. The basic courses refer to the courses, which can provide learners with basic knowledge and skills, including language, mathematics and science. The extended courses refer to the courses, which elaborate knowledge and skills in depth in order that learners can enhance or improve the knowledge and skills they prefer. The research courses intend to improve learners' creative thinking and inquiry skills by providing an authentic learning environment and research topics. Second, the curriculum development activities would be carried out on three different levels: system level, district level and school level. The system level refers to the city level; the curricula at this level would be used by all schools in Shanghai. The basic courses and some extended courses would be developed at the system level. The economic development and educational status in different districts/counties are not equal. Some extended courses would be developed at the district level. At the school level, different schools could develop their own curricula based on their specific needs. Some extended courses and research courses would be developed at the school level. According to the second curriculum plan, especially curriculum development at the school level would be enhanced during the second round of curriculum innovation. In addition, the Curriculum Innovation Committee decided that in the second round of curriculum innovation, more information and communication technology (ICT) should be integrated into the new curricula. According to Zhang (1999), the associate director of the Curriculum Innovation Committee, the second round of curriculum innovation will 'change the single representation form of curriculum in written text towards integrated lesson materials in both written text and multimedia, in order to improve the quality and efficiency of classroom teaching and learning' (p. 2).


9

1.2 Origins of the study The CASCADE-MUCH study derived from two sources: the multimedia curriculum project for Biology undertaken in Shanghai and the CASCADE study performed by Nieveen (1997). In this section, these two origins will be briefly introduced. 1.2.1 The project of multimedia curriculum for Biology (MCB) One expected outcome of the curriculum innovation in Shanghai was to produce new curricula for the primary and secondary schools in economically more developed regions. During the same period of time, ICT has been growing tremendously, and educators realized that ICT would play an important role in the new curricula. Actually, in both of the two rounds of curriculum innovation --especially in the second curriculum plan-- multimedia technology has been expected to be integrated into the curricula. In 1997, a project called 'Bainian shuren' (fostering personnel in a long term) was initiated. It was sponsored by the Central Audio and Video Bureau of China. One of its sub-projects was to design multimedia lesson materials for the subject of Biology for senior secondary schools. Three experienced Biology teachers developed an instructional scenario for the multimedia lesson materials. The instructional scenario included a detailed instructional plan depicting: the content to be covered by the multimedia lesson materials; the organization of the content; and the way both teachers and learners would use the materials. Based on the instructional scenario, computer programmers created the multimedia lesson materials. During this process, several problems were found. First, the instructional scenario did not fully exploit the advantages of CBL. Compared to traditional classroom teaching, CBL usually has two advantages: i) interaction; and ii) individualization. Computers can act as a tutor, presenting teaching materials and asking questions. After learners give their responses to the questions, computers can give immediate feedback. In some applications, learners can also ask questions, and the computer gives answers. In traditional classroom teaching, because of a large number of learners in one class, this kind of interaction between teachers and each individual student rarely happens. In addition, learners can also control their pace of learning when they are learning

10

Chapter 1

with a computer; they can go as fast or as slowly as they want. They can repeat activities several times until they have grasped the material. Whereas in traditional classroom teaching, one pace for all learners is popularly adopted. Nowadays, with the development of ICT, computers can also help learners with collaborative learning in addition to individual learning. Unfortunately, in the produced instructional scenario, such advantages were not sufficiently utilized. There were only limited interactive activities, and learners had very little control over their own individual learning. Lesson materials were usually organized in a linear way, not demonstrating sufficient flexibility for learners to browse. In short, the instructional strategies in the instructional scenario were similar to those used in traditional classroom teaching. Second, the produced instructional scenario was not described in sufficient detail. It focused on content description with various media, but paid less attention to interface design and interconnections between various components. Due to a lack of experience, the subject teachers did not provide enough information on what a screen should look like, what elements (such as buttons, list and textbox) should be included, and what would happen after clicking a button or a menu item. This information is usually essential for computer programmers in developing a multimedia program. Furthermore, the instructional scenario developers (the experienced teachers) also complained that they lacked support tools when they were making the instructional scenario. The teachers did have a great deal of experience in teaching the subject of Biology, but they did not have enough knowledge and skills in designing multimedia materials. That was probably the main reason for the existence of the two problems mentioned above. Therefore, a support tool seemed to be necessary to help the teachers in making usable instructional scenarios. This study was to investigate in what way computers can help teachers in developing instructional scenarios. The line of reasoning for this study is shown in Figure 1.2.


11

The curriculum innovation in Shanghai New Curricula

ICT

Multimedia lesson materials The MCB project Problems in the instructional scenario Need for support Figure 1.2: Line of reasoning for this study

1.2.2 The CASCADE study This study can be seen as an extension of the CASCADE (Computer ASsisted Curriculum Analysis Design and Evaluation) project initiated in 1993 by the Department of Curriculum, Faculty of Educational Science and Technology, University of Twente, the Netherlands. The CASCADE project aimed to learn about how Electronic Performance Support Systems (EPSSs) could contribute to curriculum development. In particular, it focused on supporting Dutch professional curriculum developers through the often-neglected process of formative evaluation (Nieveen, 1997). A more detailed description can be found in Section 2.5.1. In 1996, two follow-up studies (CASCADE-SEA and CASCADE-MUCH) were launched to explore computer support in different contexts. The CASCADE-SEA (Science Education in Africa) study investigates support of teachers in creating exemplary science lesson materials for classroom use in southern Africa (McKenney, 1999). A more detailed description can be found in Section 2.5.2. CASCADE-MUCH, the current project, examines computer support for instructional scenario development in the process of multimedia curriculum development in China, particularly in Shanghai. Actually, before this doctoral research project started, a computer support system for the

12

Chapter 1

analysis phase of curriculum development had already been developed during the Master program study (1995-1996) at the University of Twente (Wang, 1996). Much experience with EPSS design has thus been accumulated. After the MCB project, carried out in 1997 as explained in Section 1.2.1, this study was to focus on the design phase of multimedia curriculum development. In 1999, another follow-up study CASCADE-IMEI (Innovative Mathematics Education in Indonesia) started to investigate how computer support can help teachers in designing mathematics lesson materials based on the realistic approach (Zulkardi, 1999). A more detailed description can be found in the web site: http://projects.edte.utwente.nl/cascade/imei/, or http://www.cascadeimei.com.

1.3 Overview of the study With the rapid growth of ICT, it is expected that more multimedia lesson materials will be produced in the future (cf. Glasgow, 1997). This study intended to investigate what computers can offer in the process of making multimedia lesson materials. By following a development research approach, the study aims to produce a computer support system for subject teachers making instructional scenarios, which will be used by computer programmers to create multimedia programs. In this section, the intended target users, the aims of the CASCADE-MUCH program, the research questions and the research approach will be introduced. 1.3.1 Intended target users As mentioned earlier, Shanghai has more than sixteen million inhabitants allocated in sixteen districts and four counties. On the city level, the Shanghai Educational Committee is directly responsible for educational matters throughout the city. Within each district or county, an educational bureau is responsible for local educational administration. In addition, an educational college in each college or county --headed by the educational bureau-- is responsible for teacher training and curriculum development in that district or county. In each educational college, each subject usually has one teaching researcher who is responsible for coordinating teaching activities, teaching research and curriculum development for that subject. Because these teaching researchers


13

used to be experienced teachers, they are often chief members and even administrators of curriculum development teams in their districts or counties. Instructional scenarios for multimedia curriculum development at district level are usually developed by those teaching researchers, sometimes together with other experienced teachers at local secondary schools. Although those teaching researchers are experienced teachers, they usually do not have competent knowledge of computer-based learning, nor sufficient skills for designing instructional scenarios. The intended target users of CASCADE-MUCH are teaching researchers who are curriculum development members at district educational colleges, as well as those subject teachers who might participate in instructional scenario development in Shanghai. They are commonly called teacher-designers in this study. As different subjects have different characteristics and usually need different instructional strategies, a selection of a small sample of subjects to start the exploration was made. For CASCADE-MUCH, the subjects of Biology and Geography were chosen as its experimental subjects, because these two subjects involve many natural phenomena which are either hard for learners to learn or hard for teachers to teach (cf. SSCRCO & SECIRO, 1999). 1.3.2 Aims of the program The development of multimedia lesson materials is a comprehensive and complex process that usually calls upon several kinds of expertise, such as expertise of subject teachers, instructional designers, multimedia designers, and curriculum specialists. This means that the development of multimedia lesson materials is usually a cooperative effort, especially when a project covers a whole course, for instance. In this study, multimedia lesson materials covering a whole course will be called a multimedia curriculum. The term of multimedia curriculum will be defined and elaborated in Chapter 2. The development of a multimedia curriculum usually starts with instructional scenario development by teacher-designers, and followed by programming by computer programmers. According to Walker's (1990) deliberative approach to curriculum development, at the beginning of a complex curriculum development project, curriculum developers usually need discussions with other stakeholders, including educators, educational administrators, teachers, learners and parents, to formulate needs, aims and possible solutions. Similarly, developing a multimedia curriculum often starts with consultations between teacher-designers and other experts in order to get consensus on the

14

Chapter 1

final product (cf. Tessmer, 1993). Of course in some cases an expert may possess expertise in multiple disciplines. Based on the consultations and discussions, teacher-designers can develop an instructional scenario. Computer programmers will eventually create a multimedia curriculum based on this developed instructional scenario. Figure 1.3 shows the process of multimedia curriculum development. Curriculum development expertise Instructional design expertise Teacherdesigners

Instructional scenario

Computer programmers

Multimedia curriculum

Interface design expertise Multimedia design expertise

Figure 1.3: Multimedia curriculum development process

As long as the teacher-designers get the support needed, this process works well. However, teacher-designers sometimes cannot get immediate support from the experts. The experts are usually very busy professionals, and it is impossible for them to be continuously available for the teacher-designers. That absence or delay of support may hinder the development process. In addition, if no sufficient discussions took place between computer programmers and teacher-designers, the computer programmers may have difficulties with understanding the developed instructional scenario. Furthermore, some good inputs or suggestions from the computer programmers for quality improvement of the instructional scenario may not be integrated in the instructional scenario or learned by the teacher-designers.


15

The use of CASCADE-MUCH is expected to yield the following solutions: a. It can help teacher-designers make a usable instructional scenario by providing them with just-in-time support. b. After the instructional scenario has been made, it can make discussion or information exchange between teacher-designers and computer programmers easier. c. It improves teacher-designers' professional knowledge/skills for multimedia curriculum design. With the support of CASCADE-MUCH, the process of multimedia curriculum development mainly includes three phases (see also Figure 1.4). Various expertise

CASCADE MUCH ➀ ➁ Teacherdesigners

➀ ➁

Instructional scenario

➁

➁ ➁ ➂

➂

Multimedia curriculum ➂

Computer programmers

Figure 1.4: Multimedia curriculum design process with CASCADE-MUCH

1. On the one hand, teacher-designers can learn from the CASCADE-MUCH program what content is covered, and how the content is represented or organized in a multimedia curriculum. On the other hand, they can also use the program to develop a tentative prototype of the instructional scenario. Of course, during this process, the teacher-designers may also need to consult with the experts to get more expertise and advice. This phase is labeled as ➀ in Figure 1.4. 2. The teacher-designers will work together with the computer programmers on the tentative instructional scenario, and make it ready for programming. During this process, formative evaluation activities will be carried out to collect comments and suggestions on the preliminary instructional scenario, and revision decisions will be made. A revised prototype will be developed by the teacher-designers based on these decisions. Also, it is possible for teacher-designers to consult with the experts as well. This phase is labeled as ➁ in Figure 1.4. This phase may take place several times until a ready-to-use instructional scenario has been produced.

16

Chapter 1

3. The computer programmers will create a multimedia curriculum based on the ready-to-use instructional scenario. During this process, computer programmers will create the multimedia curriculum mostly based on the instructional scenario, but sometimes they may also need face-to-face discussions with the teacher-designers. This phase is labeled as ➂ in Figure 1.4. Figure 1.4 also demonstrates how important an instructional scenario is in the process of multimedia curriculum development. On the one hand, it carries comprehensive information, which might be used for programming; on the other hand, it also provides a concrete platform for discussion between teacher-designers and experts/computer programmers. 1.3.3 Methodology: Focus on development research The traditional view of research used to be discovery of knowledge, and development was the translation of that knowledge into a useful form in practice (Richey, 1997). In reality, a disconnect often exists between research and practice. Either theory is too abstract to guide practice, or practice lacks suitable theory to follow. Through development research, this gap is expected to be bridged to some extent. According to Seels and Richey (1994), development research is 'the systematic study of designing, developing and evaluating instructional programs, processes and products that must meet the criteria of internal consistency and effectiveness' (p. 127). In development research, the research process and the development process are merged into one enterprise. During this joint process, development and research can contribute to each other. Van den Akker and Plomp (1993) give a functional definition to development research by specifying its two main purposes: a. supporting the development of prototypical products; and b. generating methodological directions for the design and evaluation of such products. This definition implies that development research mainly contributes to two aspects: product improvement and knowledge growth. Product improvement aims at making a high quality product to be valid, practical and effective. Knowledge growth is preferably reflected in 'design principles' of the following format: 'If you want to design intervention X [for the purpose/function Y in context Z] then you are best advised to give that intervention the characteristics A, B, and C [substantive emphasis] and to do that via procedures K, L, and M [procedural emphasis], because of arguments P, Q, and R' (van den Akker, 1999, p. 9).


17

Van den Akker (1999) mentions that knowledge growth and product improvement are of equal importance in development research. Furthermore, in agreement with the description of development research by Richey and Nelson (1996), van den Akker (1999) distinguishes two types of development research: Formative research. Research activities are carried out during the entire development process, aiming at optimizing the quality of the product as well as generating and testing design principles. Reconstructive studies. Research activities are conducted sometimes during but oftentimes after the development process, aiming at articulating and specifying design principles. This study followed a development research approach, aiming at: i) producing a valid and practical computer support system; and ii) generating methodological guidelines for the design and evaluation of such products. The main research question of the study is: What characteristics should a valid and practical computer support system for multimedia curriculum design have in the context of Shanghai? By following the development research approach, this study progressed through two main stages: prototyping and assessment. The overall research design of these two stages will be presented in the following two sections. 1.3.4 Approach of the prototyping stage

Research question In the prototyping stage, the main research question was: What characteristics should a valid (and potentially practical) computer support system for multimedia curriculum design have in the context of Shanghai? According to Nieveen (1997, 1999) and van den Akker (1999), an educational or training product can be assessed on three quality criteria: validity, practicality and effectiveness. Validity refers to the extent that the product is designed based on state-of-the-art knowledge ('content validity') and that the various components of the product are consistently linked to each other ('construct validity'). Practicality refers to the extent that users (and other experts) consider the product attractive and usable under 'normal' conditions. Effectiveness refers to the extent that the experiences and outcomes with the

18

Chapter 1

product are consistent with the intended aims. Undoubtedly, during the prototyping process, an educational or training product such as the computer support system should be designed and constructed towards the direction of meeting these quality criteria. During the prototyping stage, this study mainly focused on validity and practicality. The main reason was that emphasis often shifts from validity to practicality, to effectiveness during a development process (van den Akker, 1999), and usually validity and practicality are essential preconditions of effectiveness. However, during the entire study special attention was given to indicators of potential impact of the support program on the design of instructional scenarios.

Prototyping approach In software engineering, there are two popularly used approaches to software development: the waterfall approach and the prototyping approach (cf. de Hoog, de Jong, de Vries, 1994; Sommerville, 1996). The waterfall approach separates software development activities into different phases such as requirement specification, software design, implementation and testing. After each phase is defined it is 'signed-off' and development goes on to the following phases. The prototyping approach, according to Smith (1991), is a process of producing successive trial versions of software before developing a final system. This study adopted the prototyping approach. Three benefits of using the prototyping approach were to: 1. Gradually clarify the characteristics of the innovative program The CASCADE-MUCH program is a rather innovative endeavor, lacking much experience to learn from. It was expected that the prototyping approach could help gradually clarify the design specification and characteristics of the program. 2. Successively approximate an optimal computer support program During formative evaluation activities, some descriptive and prescriptive information could be collected. Based on this information, further improvement of the prototype could be made in a next round of prototyping. Through this cyclic prototyping process, the quality of the prototype could be successively improved and optimized.


19

3. Promote communication between the designers and users One of the main reasons why users often do not like computer programs is the lack of communication between designers and users during the program development process (Gray & Black, 1994). Smith (1991) claims that communication between designers and users becomes easier and more effective through prototypes than through other means. The detailed description of the prototyping stage, including its formative evaluation activities can be found in Chapter 3. 1.3.5 Approach of the assessment stage

Research question After four rounds of prototyping, the prototype was judged to be valid and perceived to be rather practical. The assessment stage aimed to assess the practicality of the prototype by its users. The research question of the assessment stage was: To what extent is the CASCADE-MUCH program practical for both primary target group users and other users in the context of Shanghai? In addition to assessing the practicality of the prototype for intended target users, the practicality for other users, such as teacher-designers of other subjects, was also expected to be probed. Therefore two types of users were invited to attend the assessment studies: primary target group users and other users.

Assessment approach Two studies were organized in Shanghai to assess the practicality of the prototype. The first study was with primary users, and the second study was held with a group of other users. The main aim of the first study was to assess whether the prototype was practical for subject teachers who are either experienced designers or novice designers for multimedia curriculum development to make instructional scenarios. In order to collect and compare the possible differences between experienced designers and novice designers, a built-in log file was added to the prototype to trace how the users really used the prototype. In addition, several instruments such as a questionnaire and observational notes were used during the assessment process.

20

Chapter 1

The main aim of the second study was to assess to what extent the prototype could be used in other subjects as well as in other contexts. The workshop was open for participants of related professionals: teachers or teacherdesigners of other subjects, CBL designers in primary or secondary schools, and CBL designers in computer companies. The same instruments were used to collect data in the second study. In particular, the focus was on discussing possible extension to other subjects and to other contexts. The detailed information of the design and results of the assessment studies can be found in Chapter 5.

1.4 Preview of the dissertation The development research activities and findings of the CASCADE-MUCH study are presented in the subsequent chapters. In Chapter 2, a conceptual framework is presented. In the first part of that chapter, three closely related domains are introduced: multimedia, curriculum development and EPSS. In the second part of the chapter, some related support systems are briefly described, and their implications on the design of the CASCADE-MUCH study are given. In Chapter 3, the prototyping approach of CASCADE-MUCH is elaborated in more detail. Four prototypes are described in terms of content, support, interface and scenario. In addition, details about the formative evaluation and revision decisions are provided. In Chapter 4, the detailed description of the final version of the CASCADE-MUCH program is presented. In Chapter 5, the assessment activities, procedures and results of the study are presented. Finally, Chapter 6 discusses the main findings of the study.

21

Chapter 2 Multimedia curriculum and computer support systems

T

his chapter presents the conceptual framework of the study, which is closely related to three domains: multimedia, curriculum development, and Electronic Performance Support Systems (EPSSs). Section 2.1 gives the definitions of media and multimedia, and elaborates on the relationship between media and learning, as well as between multimedia and learning. Section 2.2 provides the definition of curriculum and some curriculum development models. Section 2.3 explains the relationship between a multimedia curriculum and a conventional curriculum, and presents a model for multimedia curriculum development. Section 2.4 introduces the concept of EPSS. Section 2.5 presents a number of existing EPSSs for curriculum design, and discusses their implications for the design of the study. Finally, Section 2.6 ends up with conclusions concerning the main issues presented in this chapter.

2.1 Multimedia Although the term multimedia is popularly used nowadays, no precise definition has been commonly accepted (Moore, Burton & Myers, 1996). This section defines the two terms: 'media' and 'multimedia' as used in this study, and explores the relationship between media and learning as well as between multimedia and learning. 2.1.1 Definitions of media and multimedia

Media A consensus definition of media is that they are the means or pieces of equipment that transmit information from sender to receiver (Verwijs, 1998). For example, Romiszowski (1981) defines media as 'the carriers of messages, from some transmitting sources, to the receivers of the message' (p. 339). In the context of education, the term media is usually defined as instructional facilities that carry messages to learners. Very often these messages have different presentation forms (Verhagen, 1992). For instance, a description of the Chinese Great Wall

22

Chapter 2

in a book can use text and a picture, which are different presentation forms. Modern advances in technology have made the computer able to integrate and present these presentation forms. In the world of multimedia, these presentation forms are usually called media. In this study, the term media mainly refers to these presentation forms encoded and presented on a computer, including text, pictures, audio, animation and video.

Multimedia The term multimedia used to refer to the use of several media devices in a coordinated fashion such as synchronized slides with audiotape (Moore et al., 1996). However, the adopted definition of media in this study implies that the term multimedia particularly refers to a combination of various presentation forms presented on a single device: the computer. In addition to the terms of media and multimedia, another term, hypermedia, is also being widely used. A hypermedia system often refers to a multimedia system in which contents are decomposed into chunks of information (or 'nodes') composed of various presentation forms and connected with interrelated hyperlinks (cf. Ambrose, 1991; Park, 1991). Users can visit these chunks of information by following the embedded hyperlinks. In the extreme case, if all chunks of information are represented purely as text, a hypermedia system will become a hypertext system. Similarly, if all embedded links have been removed from a hypermedia system, it will become a multimedia system (cf. Borsook, 1997). This study does not intend to separate hypermedia systems from multimedia systems. Although the two terms are separated by the above distinctions, they are seen as interchangeable in this study. 2.1.2 Media and learning Whether media can improve learning is a controversial topic. In 1983, Clark published his famous disputing assertion: '… media are mere vehicles that deliver instruction but do not influence student achievement any more than the truck that deliver our groceries causes changes in our nutrition' (p. 445). This statement led to a debate in the 1980s and received a new impulse in 1994 with two special issues of the journal Educational Technology Research and Development. Two representative persons standing on these two opposite sides of the disputation are Clark and Kozma. Clark (1983, 1994) stresses that it is instructional method, not media, that affects learning. Kozma (1991, 1994)

Multimedia curriculum and computer support systems

23

strongly disagrees with Clark's assertion, and claims that media and instructional method cannot be separated from each other. For a specific instructional method, there might be some most suitable media to support it. An instructional method, when used together with the most suitable media, will get better learning results than with other media. People may continue to argue whether media can influence learning. A common agreement is that a combination of some proper media can make hardto-implement instructional methods more feasible. For example, in the subject of Biology, the process of cell division is usually hard to explain by oral explanation or still pictures, but a computer simulation with animation and sound can make the process much clearer and easier to understand. A follow-up question is 'What are the proper media?' Already several decades ago, educators have tried to find answers to this question. For example, in 1967, Allen distinguished three levels (high, medium, and low) of proficiency between learning objectives and media. Allen (1967) proposed a matrix presenting the relationship between media and learning objectives as Table 2.1. This matrix has often been used and elaborated, for example by Briggs and Wager (1981). However, to some extent many cells in this matrix need negotiation. For instance, television programs can do a fine job of presenting factual information; programmed instruction nowadays can be used efficiently to achieve many learning objectives such as teaching facts, visual identification and rules. In addition to this matrix, some other models, such as flowcharts and worksheets, can also be used for selecting appropriate media (Verwijs, 1998). A flowchart approach usually starts by asking users some questions. By answering these questions with 'yes' or 'no', the flowchart will lead the users to the most suitable media for their purpose (cf. Romiszowski, 1988). A worksheet can be described as a checklist with questions or a scheme that weights criteria selected by the users. Examples of worksheet models can be found in Reiser and Gagné (1983).

24

Chapter 2

Table 2.1: An example of matrix media-selection model (Allen, 1967)

Note: + = high;

Learning principles, concepts, rules

Learning procedures

o o o – o o – o o

+ + o + – o o – –

o + + – – o – o o

o + o – o + + o o

o = medium;

Performing skilled perceptual motor acts Developing attitudes, opinions, motivations

Learning visual identification

Media Still pictures Motion pictures Television 3-D objects Audio recordings Programmed instruction Demonstration Printed textbooks Oral presentation

Learning facts

Learning objectives

– o – – – – o – –

– o o – o o o o o

– = low

In conclusion, media selection is a complex process, which needs to take comprehensive consideration of several factors such as instructional goals, learner characteristics, practical factors, and costs (Fenrich, 1997; Romiszowski, 1981; Verwijs, 1998), until a satisfying plan can be reached. 2.1.3 Multimedia and learning When talking about multimedia and learning, the following assumptions of multimedia can be found in literature (cf. Moore et al., 1996; Zhu, 1997): It improves effectiveness of learning since human organs may sense information simultaneously. It provides a context-rich learning environment since multimedia lesson materials are presented with multiple presentation forms. Non-linear organization improves learners' higher order thinking skills because users have to analyze what they have learned and make decisions where to go. Friendly user interface may motivate learners' interest since it is attractive and interactive.


25

However, these assumptions have not been convincingly confirmed by experiments, with some experiments even showing poorer results with multimedia (Moore et al., 1996). For example, Mayer and Anderson (1991) found that coordinated presentation of narrative and animations results in better performance on tests of creative problem solving than the wordbefore-picture group. Nevertheless, Reynolds and Baker (1987) found that texts with graphs and texts without graphs did not differ in degree of learning effect. In addition, the presentation of materials using computers was shown to increase attention and learning. Lorch, Bellack and Augsbach (1987) noted that in television presentations, children's recall of content was comparable to audio only, visual only, or simultaneous across both modes. Fenrich (1997) states that some comparative research outcomes are not very convincing because the studies fail to compare excellent regular instruction with excellent multimedia instruction. In most cases where traditional instructional methods yield excellent results, teachers feel less need for multimedia applications. Nevertheless, he believes that at least some potential benefits of a multimedia-learning environment for both learners and teachers can be found. Some of those benefits for learners are that they can: learn at their own pace, control their own learning path, and review as often as they wish; study when they want to at any time of the day or night; learn from an infinitely patient tutor that can adapt instruction to individual abilities and backgrounds; and actively pursue learning and receive immediate feedback. In addition, some potential benefits offered by multimedia applications for instructors are that they: can replace ineffective or potentially dangerous learning activities with simulations, animations and games; are well suited for teaching many dull or routine topics; may add something exciting, innovative, and/or different to an instructor's routine; save time through reduced need for teaching and preparation. Time saving can, in turn, save money and increase student contact time.

26

Chapter 2

Although a multimedia-learning environment may have the above-presumed benefits, some shortcomings warrant further attention. For example, the theory of multi-channel communication reminds that (cf. Moore et al., 1996): Humans can only receive and process one channel message at one time, consequently multi-channel messages may cause human information memory overload. Multi-channel messages may negatively interfere with each other when they are unrelated or contradictory. Ragan, Boyce, Redwine, Savenye and McMichael (1993) carried out a survey of seven major reviews of research on multimedia. Their main findings are as follows: Multimedia is at least as effective as conventional methods and has substantial cost benefits and efficiency. Frequently, multimedia instruction is more effective than conventional instruction. Multimedia is more efficient in terms of learning time than is conventional instruction (30% savings). Perhaps many people do not completely agree with these findings, and even doubt how the reviewers compared the multimedia instruction with the conventional methods and how they calculated the timesavings. Nevertheless, their findings on multimedia can serve as positive evidence for development of multimedia applications. In Shanghai, teachers as well as other educators expect that on the one hand multimedia applications can make the passive instructional process more interactive; on the other hand, multimedia can present the knowledge units in a more attractive and interesting way, even though the instructional process is still passive.

2.2 Curriculum development Curriculum development is a complex task. It can take place at different levels with different representation forms. In this section, various concepts of curriculum as well as the definition of curriculum adopted in this study are presented. Curriculum components, levels and representation forms are provided. In addition, several curriculum development models as well as the model used in this study are described.


27

2.2.1 Concepts of curriculum Among the most abstract terms in educational literature, the term curriculum probably belongs to the most elusive ones (van den Akker, 1998). It has many different definitions in literature. For example, Walker (1990) lists a broad range of curriculum as follows: learning activities; content; a set of events; situations; a series of things; total efforts of schools; a sequence of potential experiences; a set of abstractions; offering of valued knowledge, skills and attitudes. This list shows that curriculum can be defined very broadly (i.e. total efforts of schools) or very narrowly (i.e. content). Different people may refer to completely different matters when they are using or talking about the term curriculum. To understand the term, it will be helpful if the intended meaning, scope and context of the term can be specified (van den Akker, 1998). In this study, the term of curriculum refers to a plan for learning as proposed by Taba (1962). The major advantage of this simple definition is that it 'allows specification for many educational levels, representations and contexts' (van den Akker, 1998, p. 421). Walker (1990) gives a more informative description about the nature of a plan for learning. He defines that 'the curriculum refers to the content and purpose of an educational program together with their organization' (Walker, 1990, p. 5).

Curriculum components When looking at the term curriculum, a number of related topics may emerge, such as aims, content, learning activities, teaching strategies, time, etc. Table 2.2 lists several components of a curriculum proposed by various authors. Tyler (1949) summarizes four basic questions (issues) for curriculum and instruction: objective, selection of learning experience, organization of learning experience, and evaluation. Taba (1962) elaborates Tyler's four elements and expands them to seven components: rationale, aims, content selection, content organization, learning experience selection, learning experience organization, and evaluation. Marsh (1991) states that a curriculum should include four basic elements: policies/plans, teaching/learning

28

Chapter 2

experiences, teachers and students. Klein (1991) proposes nine curriculum elements as goals/objectives/ purposes, content, materials, resources, activities, evaluation, grouping, time, and space. Eash (1991) summarizes five components for a curriculum: assumptions about the learners and society, aims/objectives, content, modes of transaction, and evaluation.

Tyler Taba Marsh Klein Eash

× × × × ×

× × ×

× × × ×

Grouping, time, space

Evaluation

Learners

Teachers

Instructional strategies

Learning experiences

Content

Aims/goals/ objectives

Table 2.2: Curriculum components proposed by various authors

× × × ×

×

× ×

× ×

×

Although different authors decompose a curriculum into different components, some common components emerge. For example, all above authors agree that aim/goal/objective is a basic component of a curriculum. Most of them agree that content, learning experience, and evaluation are also essential for a curriculum. Although some components are named differently such as instructional strategies (Klein, 1991) and modes of transaction (Eash, 1991), their meaning is more or less similar. In addition, some components are closely related to each other. For example, the component of learning experiences is related to learners, and instructional strategy is related to teachers and learners. After comparing the components proposed by different authors, Nieveen (1997) summarizes five substantive components of a curriculum: i) rationale; ii) aims and objectives; iii) subject matter; iv) modes of transaction; and v) evaluation. As the component of aims and objectives is closely related to the component of rationale, in this study four basic components of a curriculum are classified as follows:


29

1. Aims/Goals/Objectives In addition to regular aims, goals and objectives, assumptions about the learners and society (Eash, 1991) and rationale (Nieveen, 1997) are also included in this component. 2. Content Here content is an umbrella term which includes all subject matter materials and resources, with which learners interact as they are experiencing a curriculum. 3. Instructional strategies Instructional strategies encompass learning experiences (Marsh, 1991; Taba, 1962; Tyler, 1949), learning activities and teaching strategies (Klein, 1991), and modes of transactions (Eash, 1991). 4. Evaluation/Assessment Although evaluation and assessment sometimes have minor distinctions (Marsh, 1997), they are used interchangeably in this study. They refer to activities and procedures for determining what students are learning or have learned.

Curriculum levels According to Goodlad (1994) and van den Akker (1998), a curriculum is usually planned on various levels (macro, meso, and micro) in various educational settings. On macro (societal or system) level, a curriculum refers to a more general curricular framework. On meso (school or institute) level, a curriculum refers to an educational program for a school or institute. On micro (classroom) level, a curriculum refers to a plan for concrete instructional activities usually taking place in a classroom. These three levels (macro, meso and micro) may be manifested somewhat differently in China, referring to national level, local level and classroom level. On the national level, a curriculum refers to the general teaching outline or the curriculum standard to be used for the whole country. On the local level, a curriculum refers to an educational program for local use such as within a province, a city, or a school. On the classroom level, a curriculum refers to a concrete plan for specific instructional activities, which usually take place in a classroom. In China, multimedia curriculum development activities (the term 'multimedia curriculum' will be explained in Section 2.3) usually take place on the local level, sometimes on the national level, but seldom on the classroom level. One of the main reasons is that educational institutes on the local level want to

30

Chapter 2

develop multimedia curricula to meet their specific needs as the nation-wide curricula are usually designed for the whole country based on the average conditions. For example, the Shanghai Educational Committee wants to develop more multimedia curricula during the second round of curriculum innovation as introduced in Section 1.1.2. In addition, for teachers at the classroom level it is nearly impossible --nor is it useful-- to develop a complete multimedia curriculum covering a whole course, even though they may develop some multimedia applications covering pieces of content in their courses.

Curriculum representation forms A curriculum may have various representation forms. A typology proposed by Goodlad, Klein and Tye (1979) and adapted by van den Akker (1998) illustrates well the representation forms as follows: the ideal curriculum reflects the original assumptions, visions and intentions; the formal curriculum reflects the concrete written curriculum documents, such as learner materials (textbooks) and teacher guides; the perceived curriculum represents the curriculum as interpreted by its users (especially teachers); the operational curriculum reflects the actual instructional process as realized in a classroom; the experiential curriculum reflects the curriculum as it is experienced by the learners or students; the attained curriculum reflects the learning outcomes of learners. In this study, a multimedia curriculum refers to the intended curriculum, which is a combination of the ideal curriculum and the formal curriculum. On the one hand, a multimedia curriculum is usually designed based on an existing curriculum document, and functions as a complement and/or extension of it. On the other hand, a multimedia curriculum may also include some elements of an ideal curriculum, which are difficult to be implemented in paper-based materials. 2.2.2 Curriculum development models Visscher-Voerman (1999) summarizes four commonly used strategies or paradigms in training and education: i) instrumental; ii) communicative; iii) pragmatic; and iv) artistic. Although these four strategies refer to broad educational design and development processes, they are also useful for


31

guiding curriculum development. According to Visscher-Voerman (1999), the instrumental (or planning-byobjective) strategy usually starts with the formulation of objectives, and ends with the assessment of objective achievement. The representative advocate of this approach is Tyler, who proposes a linear and rational model composed of four basic questions for designing curriculum and instruction (Tyler, 1949): What educational goal should the school seek to attain? How can learning experiences be selected which are likely to be useful in attaining these objectives? How can learning experiences be organized for effective instruction? How can the effectiveness of learning experiences be evaluated? The communicative (or consensus) strategy suggests that curricula or other educational products are developed in close interaction and communication between the developers and others involved. A representative advocate of the communicative approach is Walker, who introduced a deliberative (or naturalistic) approach to curriculum development. Walker (1990) assumes that better curriculum planning and development will result when educators engaged in it understand the complexity of the process. Walker's deliberative model is composed of three stages: platform of ideas, deliberation and design. In the platform stage, individuals come together to discuss and argue about their beliefs and aims regarding the ideal curriculum. In the deliberative stage, the attention shifts to how the beliefs formulated are used in assessing actual states of affairs and possible courses of action. In the design stage, activities will be undertaken to achieve the beliefs and aims. The pragmatic (or prototyping) strategy suggests that curriculum developers create their products by building, testing, and revising several prototypes. This approach is particularly useful where goals are unclear, for example in cases of a new product or a new context (Nieveen, 1997). It originated in the field of computer software development, where it is usually called rapid prototyping. More recently, it has also been used in the field of electronic lesson materials development (Nieveen, 1997; Visscher-Voerman, 1999). The artistic (or connoisseurship) strategy indicates that curriculum developers behave like artists who, based on their own expertise and experiences, create their curricula in response to the specific situation in which they work. A representative advocate is Eisner, who considers that people who make decisions about curriculum development are acting in a manner similar to

32

Chapter 2

artists who choose among an almost limitless variety of ways of representing their own view of reality (Eisner, 1985). This study adopted the pragmatic strategy based on the following considerations (see also Section 1.3.4). First, multimedia curriculum development is a complex task. At the beginning of the study, it was not clear how to design a multimedia curriculum and what kinds of support would be necessary for teacher-designers. The pragmatic strategy could help the designer clarify the design specification, and successively approximate an optimal product. Second, each prototype could stimulate communication between the designer and the various participants of each formative evaluation activity.

2.3 Multimedia curriculum development In this section, the definition and modules of multimedia curriculum, and the development model to be used for multimedia curriculum are introduced. 2.3.1 The definition of multimedia curriculum This study aims to support developing a specific form of curriculum, called multimedia curriculum. Compared to the existing term of courseware, a multimedia curriculum has some additional characteristics. First, it is broader than courseware. It usually includes the entire four components (aims/goal/objectives, content, instructional strategies, and assessment), covers the complete content of a course, and integrates several instructional strategies. Whereas a courseware application may include a part of the components, or cover a part of course content; and very often it includes only a single instructional strategy. Second, the intended target users of courseware are usually learners, who use it to explore or to review lesson materials, whereas a multimedia curriculum is usually designed for both learners and instructors. Learners can use it to learn, and instructors can also use it to facilitate their teaching process. In this study, combining the definitions of multimedia (see Section 2.1.1) and curriculum (see Section 2.2.1), multimedia curriculum is defined as: a plan for learning where various presentation forms (such as text, pictures, audio and video) are integrated, encoded and presented on a computer.


33

The relationship between multimedia curriculum and conventional curriculum can be described from two perspectives: content and media. As far as the content is concerned, a multimedia curriculum and a conventional curriculum may cover the same content. However, since a multimedia curriculum usually functions as a complement of a conventional curriculum, some extended supporting materials might be included in a multimedia curriculum. Furthermore, the content of a multimedia curriculum is relatively easier to update. Although the preparation, design and construction of the content for these two kinds of curricula may have a similar degree of complexity, once the content is ready, publication and update of a multimedia curriculum are much easier. Especially with the increase in use of the Internet, updating the content of a multimedia curriculum is becoming continually more convenient. With respect to the media used, a multimedia curriculum and a conventional curriculum may in principle use the same presentation forms such as text, pictures and audio. In a multimedia curriculum, however, these presentation forms are encoded and presented on a computer, whereas in a conventional curriculum, text printed in textbooks is usually the dominant presentation form. Even though various presentation forms are used in a conventional curriculum, they usually come from different media sources. For example, text may be from textbooks, pictures from textbooks or graph books, and audio from audiotapes. Figure 2.1 shows the difference between a conventional curriculum and a multimedia curriculum on medium.

graph text

computer …

graph text textbook

audio video animation

video videotapes

…

audio audiotapes

…

Figure 2.1: Difference between a multimedia curriculum (left) and a conventional curriculum (right)

Furthermore, a multimedia curriculum often offers more possibilities than a conventional curriculum. For example, a multimedia curriculum may have more flexibility and interactivity than a conventional curriculum. Users can easily jump to another knowledge unit by clicking a hotspot. They can find some related topics by keyword searching, or they can even zoom in on or

34

Chapter 2

zoom out from a picture. In addition, a multimedia curriculum is no longer an inert information repository; it can receive, process, and give immediate feedback to learners. In this study, in the absence of further distinction, the term 'curriculum' refers to the conventional curriculum, whereas the focus of this study will be on the design of a multimedia curriculum. 2.3.2 Multimedia curriculum modules As explained in section 2.2.1, a curriculum is composed of the following four components: Aims/goals/objectives, content, instructional strategies and assessment. A multimedia curriculum may have the same components, but some minor differences may exist in each component, particularly in the component of instructional strategies. In order to highlight the differences, hereafter the components of a multimedia curriculum will be called modules. In this section, the modules of a multimedia curriculum will be reclassified based on the four components and their specific instructional strategies.

Instructional strategies An instructional strategy can be defined as the way in which teachers present lesson content or they facilitate learning (Ertmer & Newby, 1993; Moore, 1992). In a classroom or group-learning environment, the following instructional strategies are usually adopted (Frazee & Rudnitski, 1995): lecturing modeling role-playing discussion collaborative learning. However, in a multimedia-learning environment, the instructional strategies usually include the following (cf. Fenrich, 1997; Zhu, 1999): tutorial drill-and-practice simulation educational games virtual reality cognitive apprenticeship case studies testing.


35

Actually, some of the above instructional strategies are interconnecting or overlapping. For example, modeling in a classroom can be implemented by simulation or educational games on a computer. Role-playing in a classroom can also be implemented by or replaced with virtual reality. Generally speaking, classroom instructional strategies are more practiced by teachers to implement classroom teaching, whereas multimedia instructional strategies are more oriented towards individual or group learning. As a multimedia curriculum is usually designed for both classroom teaching and individual (or collaborative) learning (see also Section 4.2.1), the above instructional strategies, including those used in a classroom and used in a multimedia learning environment, should be supported in a multimedia curriculum. Next, some common elements/events, which can constitute these instructional strategies, are to be summarized.

Instructional events Gagné, Briggs, and Wager (1988) summarize nine events of instruction, which normally occur in the process of instruction. These events include: 1. gaining attention; 2. informing learner of the objective; 3. stimulating recall of prerequisite learning; 4. presenting the stimulus material; 5. providing learning guidance; 6. eliciting the performance; 7. providing feedback about performance correctness; 8. assessing the performance; 9. enhancing retention and transfer. These instructional events do not invariably occur in this exact order, although this is their most probable order. Any instructional strategy can consist of or focus on some of these events. For example, a tutorial strategy may include all these events, or merely focuses on the events 4, 5, 6 and 7; a drill-and-practice strategy may focus primarily on the events 6 and 7. These events can be grouped into four stages: preparation, elaboration, practice and testing. The preparation stage includes the first three events; the elaboration stage consists of events 4 and 5; the practice stage consists of events 6 and 7; and the testing stage consists of events 8 and 9. Together with the components of a conventional curriculum, a multimedia curriculum may include the following modules:

36

Chapter 2

aims/goals/objectives; content preparation; content elaboration; practice; assessment. This study focused on the module of content elaboration. Although all above modules can be well presented and supported in a multimedia curriculum, the added values for other modules are comparatively lower in this particular application. For instance, the module of aims/goals/objectives can be printed in textbooks with text; content preparation can be performed in oral explanation by a teacher at the beginning of a class; practice can be done by using learners' practice books, and assessment can be done via a paper-based examination after a period of learning. Relatively, the module of content elaboration needs more presentation forms to present the knowledge units of a curriculum in more detail. 2.3.3 Multimedia curriculum development model Multimedia curriculum has a dual characteristic. On the one hand, it is a curriculum. On the other hand, it is also a multimedia computer program. Similarly, the development of multimedia curriculum has a dual characteristic, too. Firstly, its development has the general characteristics of conventional curriculum development. Its development activities should include preliminary research, analysis, design, formative evaluation and summative evaluation. Secondly, as a computer program, its development should follow a computer program development approach. In this study, the adopted multimedia curriculum development model is shown in Figure 2.2.

Preliminary research

Design Formative IS evaluation

Program- Formative ming evaluation

Analysis

Note:

IS = instructional scenario;

MC = multimedia curriculum

Figure 2.2: Multimedia curriculum development model

MC

Summative evaluation


37

Compared to the curriculum development model used by Nieveen (1997), the multimedia curriculum development model adds one more cycle, which is composed of programming and formative evaluation. After the preliminary research, the conditions and constraints can be clarified, and the decision to continue or stop the project will be made. During the first circle, teacherdesigners will carry out needs analysis, subject matter analysis, and learner analysis, etc. Based on the analysis results, a tentative instructional scenario will be developed. After that, formative evaluation activities will be carried out to collect comments and suggestions on the instructional scenario from various experts. Based on the comments and suggestions, revision decisions will be made and a next round of prototyping starts. This cycle may take place several rounds until a satisfying instructional scenario has been developed. In the second cycle, computer programmers will create a multimedia curriculum based on the developed instructional scenario. Similarly, after a tentative multimedia program has been created, formative evaluation activities will be carried out, and a next round of prototyping starts based on the collected comments and suggestions. This process may continue through several rounds until an optimal multimedia curriculum has been created. Afterwards, the created multimedia curriculum may need a summative evaluation to judge its quality.

2.4 EPSS CASCADE-MUCH is a computer support system. It is related to the concept of Electronic Performance Support System (EPSS). In this section, the concept of EPSS, the advantages and disadvantages of an EPSS and the design of an EPSS are presented. 2.4.1 The concept of EPSS Today's computers can provide and integrate many kinds of performance support such as information, advice and tools (Nieveen & van den Akker, 1999; Stevens & Stevens, 1995). This is related to the concept of EPSS. Instead of providing performance support separately, an EPSS is an integrated computerized environment, in which performers (users of the EPSS) can get many kinds of support in a just-in-time fashion (Gery, 1991; Nieveen, 1997; Stevens & Stevens, 1995).

38

Chapter 2

Gery (1991) claims that an EPSS is a useful tool for (novice) performers because it can help them to solve difficulties or questions such as 'Why do this?' 'What is it?' 'How do I do it?' 'Please show me an example.' Gery (1991) believes that the number of questions or requests is finite, and the information associated with a given question or request can be reasonably defined. An EPSS can provide the performers with a powerful tool for overcoming their difficulties. The traditional methods for answering the questions or meeting the requests are usually to provide overhead support (hotlines, help desk, support personnel), paper-based support (manuals, periodicals, brochures, memos, reference cards), or ad-hoc training (supervisors, co-workers, trial-and-error). These traditional methods often work well, but they have some limitations. For example, overhead support may work efficiently for employees, but it is expensive and inefficient for employers to place an organization's most talented personnel in these reactive roles. Paper-based support may be flexible and easy to use, but these job aids require time-consuming and expensive updates. Formal training mostly needs performers to be away from their jobs. Also, employees may have diverse background, and the one-size-fits-all approach to training no longer works well (Gery, 1991; Legent Service, 1992). As an integrated computerized support environment, an EPSS can effectively overcome the above limitations by providing performers with alternative tools on a computer. For example, overhead support can be replaced with online help; paper-based support can be replaced with online reference, and ad-hoc training can be replaced with computer-based training. In this case, support will be available anytime when it is needed; information can also be easily updated, and costs may be reduced. Gery (1991) lists several possible kinds of software found in EPSSs, including advisory or expert systems, interactive productivity software, help systems, interactive training systems, assessment and monitoring systems, etc. In addition, she mentions that an EPSS could incorporate anything else that an employee might need, such as a tutorial, a calculator, a notebook, etc. These kinds of software or components can be reclassified into the following four broad categories (cf. Hudzina, Rowley & Wager, 1997): information advice tools training.


39

Information refers to the static information and/or help usually organized in a hypertext/hypermedia format, which remain the same for various users. Advice refers to heuristic support information, which is given based on users' specific needs and the embedded intelligent expertise. For instance, advisory and monitoring (Gery, 1991) belong to this category. Tools usually refer to the support aiming at helping performers carry out tasks. Such tools may include external applications such as a word processor, a calculator, etc. The productivity and applications (Gery, 1991) belong to this category. Training usually refers to the support aiming at improving performers' skills. For instance, the training and assessment (Gery, 1991) belong to this category. The key difference between training and tools is that training focuses on skill improvement of users, whereas tools focus on task performance. In this study, the CASCADE-MUCH program includes all these four categories of support. Comparatively, information, advice and tools received more attention and efforts, but training got less. In the CASCADE-MUCH program, the support of training mainly includes some wizards and a tutorial program. The wizards were developed in the second prototype, but some were removed in successive prototypes. More discussion on wizards can be found in Section 6.2.2. The tutorial program has not been completed yet. The reason is that it was too early to develop a tutorial program for (novice) users to learn the prototype before the prototype has been completely finished. However, the tutorial program will be completed gradually in the future. 2.4.2 The potentials of EPSS Compared to conventional job aids, an EPSS has a number of assumed advantages. These advantages will be elaborated from three perspectives: users, end results and designers. From the users' (or employees') point of view, an EPSS can help them perform their tasks more efficiently than in a traditional training situation, because an EPSS can provide advice, information and/or instruction immediately when needed (Gery, 1991). With an EPSS, users do not need to remember all issues related to their work, and they can consult the EPSS on the issues they want at the time they really need it. From this point of view, an EPSS can also reduce information load during task performance. In addition, an EPSS can also improve users' professional development knowledge, because they can learn information from it by interactively using it.

40

Chapter 2

From the end results' point of view, an EPSS can improve the quality of the end results. During the process of using an EPSS, users can get expert advice on how to proceed with a task and/or how to improve the quality of the end result. Furthermore, some embedded or linked tools can help them perform a task easily and even improve the quality of the product as well. From the designers' point of view, designing an EPSS has the potential to increase its domain design and development knowledge, and make implicit knowledge become explicit. During the process of designing an EPSS, the designers need to collect information and intelligent expertise in order to provide users with information and advice. With the growth of the EPSS, the domain knowledge of the designers will also be enlarged (cf. Paquette, Aubin & Cervier, 1994). In addition, the content of the EPSS will be presented clearly and explicitly in the system. It will help the designers make their implicit knowledge become more explicit and transparent to the users. Furthermore, the use of EPSSS may help promote organizational learning (Nieveen, 1997). Practically every organization possesses a shared knowledge base, such as some collection of information regarding techniques, methods and procedures that are common to the work. Quite often, this knowledge base grows intuitively, and it is not formalized into a written form. The idea of organizational learning includes the notion that, on the one hand, the captured knowledge from individuals and teams will be stored in the knowledge base and will be available for the whole organization; on the other hand, the communication tools provided with the EPSSs will give the team members opportunities to exchange their individual ideas. When individuals change jobs, their knowledge doesn't leave with them; the stored information in a computer can be used for the future. Even better, less experienced individuals can use such a support system to learn from it, to communicate with other experienced members, and to become familiar with the development process. In addition to the assumed advantages above, some concerns about EPSS need to be addressed here. First of all, an EPSS may be not appropriate in some cases, for example, where successful performance depends on habitual, automatic, and seamless performance like in surgery (Nieveen 1997). Second, few exemplary systems of EPSS exist since the concept is new. It might be hard for performers, designers, as well as administrators to understand the concept without seeing and/or using an existing EPSS. Consequently, resistance may exist that may prevent shifting some resources from training to EPSS (Legent service, 1992). Third, people who lack competent computer skills may fear the use of an EPSS.


41

In this study, all of the above three advantages were expected to be exploited in the CASCADE-MUCH program. Also, by taking these advantages, the design aims (see Section 1.3.2) of the program were also expected to be effectively achieved. For instance, the first advantage of helping users perform tasks and improving their professional knowledge is closely related to the aim (a) and (c) on page 15. It means that the CASCADE-MUCH program, as an EPSS, has the potential to help teacher-designer to make instructional scenarios easily and to improve their professional knowledge. Similarly, the second advantage of improving the quality of the end results is also related to the aim (a) and (b). 2.4.3 The design of EPSS Typically two approaches can be used to design an EPSS (Gustafson, 2000). The first one is a classic instructional design approach (or waterfall approach in software engineering). Analysis, design, development, implementation, formative evaluation and summative evaluation are directly applicable for designing an EPSS. The classic design approach works best when the performance process is stable, and related information is available. The second approach is prototyping. It is a process of producing successive trial versions before developing a final system. The prototyping approach works best when the aims or context conditions of the support system are unclear, complex, complicated, or still changing. In this study, the design approach adopted is prototyping. The detailed design of the study will be introduced in the following chapters. During designing an EPSS, a number of considerations should be taken into account. For example, Gustafson (2000) summarizes several major considerations when creating (or selecting) an EPSS such as: 1. Black box/glass box design In a black box design, the inner structure (like rules, data structure, models) of a support system is invisible to the users. In contrast, a glass box design makes those underlying rules, models, algorithms, and other structures readily available, and perhaps even requires users to examine them. 2. Part-task/whole task support Some workplace tasks may be very complex or lengthy. It might be hard for an EPSS to provide support for the whole process, although support for parts of the task should be possible.

42

Chapter 2

3. Embedded/linked tools Some tools have been available externally such as flowcharting and concept mapping tools. In this case, it is not necessary to create them again. Decisions must be made as to whether support tools will be embedded or linked to the support system. When embedded, the support tools become an internal part of an EPSS, and all of the pieces of an EPSS are seamlessly integrated. When linked, the support tools are readily available, but access usually requires some action on the part of the users. 4. Static/dynamic system A static system remains the same always. Users will be unable to tailor or modify it, while a dynamic system may learn from the users and grow during the process. In addition, as an integrated computerized support system, an EPSS needs a lot of tools and technologies during its design. Gery (1991) summarized a list of tools for designing components of an EPSS, including information processing tools (such as hypertext, text management and retrieval systems), training development tools (such as CBT authoring systems, concurrent authoring systems), advisory system development tools (such as knowledge processors, expert systems), general purpose tools (such as word processors, graphical editors), special purpose tools (such as animation software), and programming environments (such as object-oriented programming languages). In this study, CASCADE-MUCH followed a glass-box objective design. By using the CASCADE-MUCH program, teacher-designers can, on the one hand, produce instructional scenarios; on the other hand, they can also learn some information from it by accessing the information and/or suggestions. It is a part-task support system, because it focused on the design phase rather than on the entire process. Several external tools, such as Microsoft Word and Mindman, were linked to the CASCADE-MUCH program. For the time being, the CASCADE-MUCH program is a rather static system. However, users can add or modify some information or tips in the text files or in the databases. In addition, in agreement with Gery (1991), developing the CASCADE-MUCH program required many tools and technologies, such as the Delphi programming platform, the communication technology between Delphi and Microsoft Word, and the technology of database management.


2.5 Existing computer development

support

43

systems

for

curriculum

Over the years, a number of computer support systems for curriculum or instructional design have been produced. In this section, some existing computer support systems for curriculum development are to be introduced. These support systems include two CASCADE studies (CASCADE and CASCADE-SEA) and several other support systems. 2.5.1 CASCADE The CASCADE study was initiated by the Department of Curriculum at the University of Twente together with the Dutch National Institute for Curriculum Development (SLO) in 1993. Although the concept of formative evaluation is familiar to many curriculum developers, research has shown that such activities are often neglected in the development process. The material developers (curriculum developers) at the SLO have been, in this sense, no exception (van den Akker, Boersma & Nies, 1990). This group has indicated that such evaluations are seen to be complex and time-consuming. As a result, opportunities for small-scale efficient evaluations are often underutilized. The initial CASCADE research (Nieveen, 1997) aimed to explore ways to support relatively quick and easy evaluation efforts. It was hoped that the CASCADE program could make formative evaluation activities less daunting and more efficient to be carried out. The CASCADE study contained three main phases: analysis, prototyping and evaluation. The analysis phase was driven by the question: 'In what ways can a computer-based support system for curriculum development contribute to the optimization of development practices within the SLO?' Analyses were conducted on various forms of computer supported curriculum development systems, and an initial prototype was developed for comments and discussion with experts in the domain of computer supported curriculum development. In the prototyping phase, five successive prototypes were created and revised based on formative evaluation results. The prototyping process was concluded with a final version of the program. This was then tested to determine if the system would be useful to a larger group of SLO developers, and to see if the system would yield real effects. 'What is the practicality and effectiveness of the final version of CASCADE in the SLO practice?' was the question leading this final phase of the study.

44

Chapter 2

The final evaluation concluded that the use of CASCADE could: improve consistency of evaluation plans and activities by helping to structure decision-making as well as aiding in weighing options; motivate developers and offer reassurance in one's ability to conduct formative evaluation activities, in part by offering an overview of such activities; save time by offering assistance in developing a framework for an evaluation plan and by offering sample documents (such as evaluation instruments) that can be adjusted for one's own situation; support the underpinnings of decisions regarding the design and execution of evaluation by offering explanations about the concepts used. In summary, CASCADE evolved as the name for both the line of research and a computer program. The original CASCADE program focused on the evaluation phase of the entire process. This system was designed to help professional curriculum developers conduct formative evaluations of exemplary teacher and learner materials. Among other findings, evaluation of this program indicated that such a tool may offer much to the world of curriculum development, particularly with regard to the creation of classroom materials (Nieveen, 1997). For additional information regarding CASCADE, please refer to Nieveen (1997), Nieveen and van den Akker (1999), and http://projects.edte.utwente.nl/cascade/original/. 2.5.2 CASCADE-SEA The CASCADE-SEA study (McKenney, 1999, 2001) has been set up to explore how computer supported curriculum development might be a potential solution to some of the problems encountered by materials developers in the domain of Science Education in southern Africa. Educational reform projects have been in operation in southern Africa for over 20 years, and the creation of locally relevant classroom materials has been a recurring theme among many of these projects. Various ongoing efforts stress the notion that teacher development is directly linked to curriculum development. CASCADE-SEA aids facilitator teachers through the process of translating their good ideas (about lesson materials) into a form that is valuable to others. This support system strives to promote improved task performance (better quality materials), organizational learning (among resource teachers) and improved curriculum design and development knowledge (teacher professional development). These factors could then


45

contribute to additional benefits. CASCADE-SEA evolved through four prototype versions. Throughout this evolution, two main kinds of changes took place. Each prototype aimed at learning more about a specific aspect or area of curriculum development (in this case, science education materials design) and at the same time, fine-tuning design-specifications for the program as a whole was also on the agenda. The CASCADE-SEA program currently consists of two main elements: CDROM and a web site. Although the number of CASCADE-SEA users with Internet access is rapidly increasing, many still work with the system in an offline setting. For this reason, the web site is a supplement to the main program. Whereas the CD-ROM aids the material designers in making personal decisions about how to create a series of lesson plans (a teacher guide), the web site aims to foster communication between material designers. The database contains a variety of completed lesson plans as well as building blocks for materials (clip art, activity ideas, etc.) which visitors are welcome to use. They are also encouraged to contribute any similar resources for biology, chemistry, physics or mathematics. The CASCADE-SEA CD-ROM aims to support those groups and individuals involved in the process of creating exemplary lesson materials or teacher guides, usually to be shared among colleagues in the same region. Toward that end, the program asks the user(s) to think about what they would like to achieve (why they are making materials, and what kinds of materials would be useful for their particular setting). If the developer already has a basic rationale in mind, then the computer helps to make this explicit and generates a 'rationale profile' that may then be used in discussion with co-developers. Should the user have difficulty-determining key issues related to the materials (s)he is about to develop, then CASCADE-SEA will recommend that the analysis section be visited. In the analysis portion of the program, support is offered in conducting a material needs and context analysis, which (when completed) will then aid in forming a rationale. Once the user has generated sufficient specifications regarding the kind(s) of materials to be developed, the design phase supports the creation of these materials. The design area helps the user to map out a lesson series, build individual lessons and to think about the layout, form and style to be applied. For users who have completed some of the development (ranging from rationale formation to a complete lesson series), support is also available for conducting a formative evaluation of the materials designed so far. The evaluation component is heavily based on the former CASCADE program, although it has been translated in terms of both language and context. For additional information regarding CASCADE-SEA,

46

Chapter 2

please refer to McKenney (1999, 2001) and http://projects.edte.utwente.nl/ cascade/seastudy/. 2.5.3 Other support systems In addition to these two existing CASCADE studies, another study CASCADE-IMEI has been started and is going on now. It is related to Innovative Mathematics Education in Indonesia. For additional information regarding the CASCADE-IMEI research, please refer to Zulkardi (1999), or visit the web site: http://projects.edte.utwente.nl/cascade/imei/ or http://www.cascadeimei.com. Furthermore, some other computer support systems for curriculum/instructional design can also be found in literature, such as AGD, ID Expert, ID Library, ILCE, CEDID, ECC COCOS, GAIDA, QUEUE, SimQuest and GTE. Brief introductions of these support systems can be found in Nieveen (1997), and also in Nieveen and Gustafson (1999, 2000). In this section, a computer support system called Designer's Edge is to be introduced, because many ideas behind the Designer's Edge are similar to those of the CASCADE-MUCH study. For example, the Designer's Edge aims to design a multimedia course running on a computer, not a conventional course on paper. This aim is the same as that of CASCADEMUCH. In Designer's Edge, the multimedia course can be a Windows-based multimedia course, or a web-based online course. It is also similar to CASCADE-MUCH, in which the final curriculum is a Windows-based multimedia curriculum. In addition, a storyboard is popularly used in Designer's Edge to help designers create screen layouts, and the designed course can be output to other authoring systems such as Quest or directly to HTML or JAVA templates for online delivery. In CASCADE-MUCH, the instructional scenario (or storyboard) is the outcome, which will be used by computer programmers to create a multimedia curriculum. The instructional scenario can be exported to Microsoft Word. More comparisons can be found in Table 2.3. According to the introduction on the web site: http://www.mentergy.com, Designer's Edge is an integrated computer support system, which focuses on the common activities of instructional design. Its creation was based on the standard instructional system design model, and it provides support for an entire 12-phase instructional development process with special emphasis on the analysis, design and evaluation of effective technology-based training.


47

There are two distinct target group users for Designer's Edge. The first group consists of experienced instructional designers (trainers/educators) who are familiar with the process of creating instructional materials. For this group, Designer's Edge can be used to accelerate the instructional design process by offering design consistency in organizing data, writing reports, storyboarding and media pre-production, etc. The second group of target users consists of subject matter experts (SMEs) and human resource personnel who find themselves in a training role because of their subject-matter expertise. Subject matter experts are most effective when involved in the early stages of development, making decisions about content, objectives and presentation sequencing. But many companies have engaged in failed attempts to teach subject matter experts to become authoring system specialists. For this group, Designer's Edge can help them become competent authoring system specialists. Designer's Edge can help designers effectively perform the following tasks: brainstorming during needs analysis; writing objectives; outlining content; writing test questions; mapping out a course; storyboarding; managing instructional media elements. In carrying out these tasks, Designer's Edge demonstrates a number of characteristics, such as: 12-phase instructional design process support; customizable interface; copying or sharing design data; full-screen and graphical storyboarding; instructional advice. The detailed and further information regarding Designer's Edge can be found in Chapman (1998) or on the web site: http://www. mentergy.com 2.5.4 Comparison of the support systems and implications on the design of the study Computer support systems can be compared in different ways based on different criteria. Here the comparison is made based on the scheme defined by Nieveen and Gustafson (1999), which mainly includes the following

48

Chapter 2

attributes: A. Type of output Type of output refers to both curriculum levels and characteristics of the results. With respect to curriculum levels, the output of a computer support system might be lessons, courses, or a (computer) product. With respect to characteristics of the result, the output might be oriented to learners or teachers in a form of paper-based, computer-based or webbased, and the output may be used in a site specific or generic context. B. Intended target groups Intended target group is another element worthy of comparison. It mainly includes two perspectives: expertise of user group (professional designers, subject matter experts, teachers, or learners), and computer experience (low or high). C. Type of development process Type of development process is mainly composed of two perspectives: paradigm for engaging in educational and training development, and underlying elements of the systematic approach to development of education and training. The paradigm can be divided into four perspectives (see also Section 2.2.2): instrumental, communicative, pragmatic, and artistic, defined by Visscher-Voerman (1999). Although various educational and training systems may involve various elements/activities, some certain key elements can be found in most literature, such as: analysis, design, development, implementation and evaluation. These are the elements used in the comparison scheme. D. Task support Task support is the last attribute to compare, which includes types of support and adaptability of support. In the scheme, type of support includes communication aids, job aids, and training aids. Support can be tailored inside the tool, outside the tool, inside networked tool, or not at all (closed). The comparison of the computer support systems introduced in this section is illustrated in Table 2.3, in which the design of CASCADE-MUCH is listed in the shaded column.


49

Table 2.3: The comparison of the computer support systems CASCADE A Curriculum - Lesson levels - Course Characteris- - Developertics of oriented results - Paper-based - Site specific and generic

- Low

CASCADEMUCH - Computer program - Learneroriented and teacheroriented - Computer based - Site specific and generic - Instructional - Teacherdesigners designers - SMEs or human resource personnel - Low - Low

- Pragmatic

- (not clear)

- Pragmatic

- Analysis - Design - Evaluation

- Analysis - Design

- Job aids - Communication aids - Training aids - Inside tool - Outside tool

- Job aids - Training aids - Communication aids - Outside tool

CASCADESEA - Lesson - Course - Teacheroriented - Paper-based - Site specific and generic

B Expertise of - Professional - Teachers user group designers

Computer - Low experience C Develop- Pragmatic ment paradigm Elements of - Formative systematic evaluation approach

- Analysis - Design - Development - Implementation - Evaluation D Types of - Job aids - Job aids support - Communication aids - Training aids Adaptability - Inside tool - Inside tool of support - Outside tool - Outside tool

Designer's Edge - Computer program - Developeroriented - Computerbased - Generic

50

Chapter 2

This table shows that the two CASCADE studies (CASCADE and CASCADE-SEA) result in instructional materials in a form of lessons or courses which are paper-based, whereas Designer' Edge results in a computerbased program. The CASCADE study is oriented to curriculum developers, who have expertise of professional curriculum development. The CASCADESEA study is oriented to subject teachers, who have expertise of their subject domains; Designer's Edge is oriented to educational and/or training systems developers, who have expertise of instructional design, subject matter or human resource development. The two CASCADE studies are site-specific but have the potential to be extended to other generic contexts, whereas Designer's Edge is rather generic. All of the support systems require only a low level of computer experience. The development approach adopted by the two CASCADE studies are both pragmatic, whereas the development approach adopted by Designer's Edge is not clear. The CASCADE study investigates computer support for the element of formative evaluation in the context of SLO. The CASCADE-SEA study investigates computer support for the whole process of teacher-based materials development in the context of Africa. Designer's Edge provides computer support for the 12-phase instructional design process, especially focusing on the elements of analysis, design, and evaluation. All of the above support systems provide job aids, and most of these systems, with the exception of CASCADE, provide communication aids. Also, CASCADE-SEA and Designer's Edge provide training tools. With respect to adaptability of support, the support of CASCADE, CASCADE-SEA and Designer's Edge can be tailored both inside and outside the systems. CASCADE-MUCH proceeds with a similar study to the above systems, but with different focus and context. The other CASCADE studies focus on the development of paper-based lesson and/or course materials in the sitespecific or generic context, whereas the CASCADE-MUCH study focuses on the development of computer-based multimedia curriculum in the context of China, particularly in Shanghai. The characteristics and design of these systems have the following implications for the design of the CASCADE-MUCH study. First, CASCADE-MUCH adopts the pragmatic development approach, which is the same as that of other CASCADE studies. This approach has been explained in Section 2.2.2.


51

Second, CASCADE-MUCH focuses on two phases --analysis and design, in particular the design phase-- of the systematic development process rather than on the whole process. Among the above support systems, only CASCADE-SEA focuses on the whole process, and the other systems mostly focus on some specific phases. For example, CASCADE focuses on the phase of formative evaluation, and Designer's Edge focuses on the phases of analysis, design, and evaluation. Third, CASCADE-MUCH attempts to involve a web site (http://projects.edte.utwente.nl/cascade/much/ or http://cascade-much. 20m.com) as a communication tool to provide users with updated information and technical service. Most of the support systems provide such communication tools. Furthermore, the rapid growth of the Internet being witnessed presently allows communication tools to be easily created as well. Fourth, the CASCADE-MUCH study borrows some design perspectives and experiences from the CASCADE study, such as components and quality criteria. In addition, object-oriented technology is used to describe screen elements in Designer's Edge. In the CASCADE-MUCH study, objectoriented technology is adopted to represent knowledge units with various presentation forms.

2.6 Conclusions Although the debate on relationships between media and learning and between multimedia and learning has not yet been resolved, a consensus has been reached that some proper media can make hard-to-implement instructional methods possible. Multimedia is considered to be at least as effective as conventional methods, and frequently it can improve the motivation of learners and the interaction between learners and a multimedia curriculum. A multimedia curriculum and a conventional curriculum have in common that they both cover the same content with the same presentation forms. However, in a conventional curriculum, text is usually the main presentation form. Even when several presentation forms are used, they are mostly from different media sources. In a multimedia curriculum, usually various presentation forms are used simultaneously, and they are encoded and presented on a single medium: the computer.

52

Chapter 2

Multimedia curriculum development should follow both a curriculum development process and a multimedia program development process. After an instructional scenario has been developed by teacher-designers (usually after several rounds of prototyping), computer programmers start to create a multimedia curriculum based on it. Similarly, computer programmers may also need several rounds of prototyping until a satisfying multimedia curriculum is finally created. EPSSs have some assumed advantages. CASCADE-MUCH, as an EPSS, is expected to be able to help teacher-designers develop instructional scenarios and improve the quality of the produced instructional scenarios by accessing the provided information, suggestion, tools and other supporting materials. Also, the process of designing scenarios is expected to improve teacherdesigners' professional knowledge about multimedia curriculum design.

53

Chapter 3 Prototype development

P

rototyping has been the main approach during the design of CASCADEMUCH. The study progressed through four rounds of prototyping, focusing on four components: content, support, interface, and scenario. In this chapter, each prototype will be introduced in detail. This chapter starts with the preliminary choices in Section 3.1. Section 3.2 presents an overview of the prototyping process and the prototypes, themselves. The first prototype is explained in Section 3.3, focusing on the formulation of the content, support and interface. The second prototype is elaborated in Section 3.4, focusing on the design of the content, support and scenario. The third prototype is covered in Section 3.5, focusing on the improvement of the content, support and scenario. The fourth prototype is described in Section 3.6, focusing on the revision of the content and support. In addition, each prototype underwent a formative evaluation; the design and results of those evaluations are presented at the end of each section.

3.1 Preliminary choices According to de Hoog, et al. (1994) and Nieveen (1997), a computer program can be broken down into some key components in order to make its development process more transparent. In this study, the prototype has been broken down into four components: content, support, interface and scenario. Based on the introduction of the context in Chapter 1, and the literature review and study of examples in Chapter 2, some preliminary choices for these four components were formulated at the beginning of the development process. In this section, these preliminary choices will be briefly introduced. With regard to the content, two preliminary choices were made. First, the program should be closely related to multimedia and curriculum development, since the prototype aimed at facilitating teacher-designers in designing a multimedia curriculum presented on a computer. Second, it should include both theoretical models and practical guidelines. The theoretical models should help users understand why the content was put on the screens, and it could also help them improve their professional knowledge for multimedia

54

Chapter 3

curriculum design. The practical guidelines should combine the theoretical models and the practical situation in the context of Shanghai, and provide teacher-designers with more practical information. With regard to the support, the prototype should integrate many types of support, since the prototype is designed to be an EPSS to help teacherdesigners develop instructional scenarios. As explained in Section 2.4, EPSS is an integrated computerized support environment in which users can get many types of support just in time. An EPSS usually integrates one or more of the four following categories of support: information, advice, tools, and training (cf. Hudzina, et al., 1997). The prototype should integrate some or all of these four broad categories of support. With regard to the interface, the following two choices were made. First, the program should be consistent. On the one hand, it should be externally consistent to some extent with other computer applications. In this regard, users should not feel confused when they first see the interface, and they should be able to start with it easily. On the other hand, it should be internally consistent. For instance, the fonts, colors, and buttons should be the same on different screens, and the same elements on different screens should be in the same positions. In this case, navigating within the prototype would be easy to learn. Second, it should be flexible. The prototype should include several parts so that different users can start with, or work on, different parts based on their specific experiences or needs. In addition, it should provide both linear and non-linear navigation tools, and most actions in the prototype should allow the user to choose between either mouse or keyboard in order to meet various users' preferences. With regard to the scenario, two preliminary choices were made. First, a scenario should include information, which might be useful for computer programmers and for stimulating discussions between teacher-designers and computer programmers, because the instructional scenario is intended to achieve these two purposes. Second, since a lot of information would be included in an instructional scenario, the information should be well structured, otherwise it might be hard for computer programmers to read and/or understand it. In order to be well structured, the instructional scenario should be in a format that can be readily exported to Microsoft Word. Microsoft Word is a powerful word processing tool and can help to organize the produced instructional scenario in an elegant style.

Prototype development

55

3.2 Overview of the prototyping process and the prototypes The program progressed through four rounds of prototyping, focusing on the four components: content, support, interface and scenario. At the end of each round of prototyping, formative evaluation activities were carried out and revision decisions were made based on the comments and suggestions. Each prototype was revised by integrating the revision decisions and new design ideas in a new round of prototyping. In this section, the prototyping process and the characteristics of the structure and the four components of each prototype will be briefly introduced. 3.2.1 Prototyping process The actual prototyping process of the program is depicted in Figure 3.1. The first round of prototyping focused on the components of content, interface and support. In the end, a formative evaluation was carried out in Shanghai. The group of participants was mainly composed of intended target users, curriculum or instructional designers, and computer-based learning (CBL) designers. More detailed information of the first prototype can be found in Section 3.3.

Content

Interface

IV III II

I

Scenario

Notes:

I, II, III, IV refer to the four prototypes respectively more focus less focus

Figure 3.1: The prototyping process

Support

56

Chapter 3

The second round of prototyping focused on the components of content, support and scenario. At the end of the second round of prototyping, another formative evaluation activity was carried out in Shanghai. The characteristics of the participants were similar to those in the first round of prototyping. More detailed information of the second prototype can be found in Section 3.4. The third round of prototyping focused on the content, support and scenario. At the end of this round of prototyping, two formative evaluation activities were carried out in Shanghai and at the University of Twente (UT) in the Netherlands respectively. In Shanghai, three micro-evaluation workshops were organized. The groups of participants of the first two workshops consisted of intended target users. The participants of the third workshop were computer programmers, who developed multimedia programs based on the produced scenarios. After this micro-evaluation activity in Shanghai, minor modifications were made, and an expert appraisal workshop was organized at the UT. The participants were experts in curriculum development, instructional design or multimedia technology. The detailed description of the third prototype can be found in Section 3.5. The fourth round of prototyping focused on the content and support. After the expert appraisal at the UT, some content and support of the prototype were revised. In order to check whether the revised prototype was valid in the context of Shanghai, another expert appraisal workshop was carried out in Shanghai following the revisions. The participants in this workshop were also experts of curriculum development, instructional design or multimedia technology. The detailed description of this prototype can be found in Section 3.6. 3.2.2 Evolution of the structure and the four components Table 3.1 gives an extensive overview of the structure and the four components (content, support, interface and scenario) of each prototype. In this section, the evolution of the structure and the four components of the program will be described. The first prototype was composed of two parts: Designer's Aid and Edit Panel. The Designer's Aid part aimed at guiding (novice) designers of a multimedia curriculum in designing an instructional scenario step-by-step; the Edit Panel part provided an editing environment in which (experienced) designers of a multimedia curriculum could directly design an instructional scenario without following the step-by-step guidance. In the second


57

prototype, one more part was added to the program: Main Frame. The Main Frame part aimed at providing a more comprehensive environment, in which the various parts were connected and users could easily visit other parts. In the third prototype, another part was added: Tutorial. The Tutorial part aimed at providing an interactive training environment in which novice users could learn how to use the program. More description of the relationship between these parts will be given in Section 4.1. The component of content got attention during each round of prototyping. Table 3.1 shows the evolution of some issues such as the definition of multimedia curriculum, the goals and usage of a multimedia curriculum. For example, in the first prototype, the goals of an electronic curriculum (this would be called multimedia curriculum since the second prototype) was determined to meet the needs of quality-driven education and test-driven education. Since the second prototype, the goals have been changed to be: i) basic knowledge and skills learning; ii) extended knowledge and higher-level skills improvement; and iii) attitudes development. This division of goals has remained the same to the final version. Four broad categories of support (information, advice, tools and training) were involved in each prototype, and each broad category was broken down into some specific types of support. As illustrated in Table 3.1, during the evolutionary process, some types of support were added, removed and/or changed. For example, the support of explanations and examples were separated in the first two prototypes, but they were merged in the third prototype; some wizards were added to the second prototype, but some were removed in the third prototype. The interface was basically formulated in the first prototype, but some improvements were made in the succeeding prototypes. For example, in the first prototype, each screen had a logo, but most logos were temporarily selected and they were not of the same size or with the same style. Since the third prototype, all logos have been changed to be more consistent. In addition, the interface of Edit Panel changed a lot in the third prototype. In the first two prototypes, all media formats were shown in a shared textbox, and only one media format could be shown at a time. In the third prototype, the content representation part provided a separate folder for each presentation form, and each presentation form was organized in an objectoriented style.

58

Chapter 3

The component of scenario was gradually formulated over several prototypes. In the first prototype, the concept of scenario was not explicitly introduced, but a tentative idea was that all information needed by computer programmers should be included in an instructional scenario. In the second prototype, the concept of scenario was temporarily formulated based on the perceptions of some related concepts such as storyboard, super storyboard and programmer-ready materials (cf. Champman, 1998; Harrison, 1995). A paper-based scenario was developed and formatively evaluated in the second round of prototyping. Since the third prototype, the program has included the ability to produce instructional scenarios and export them to Microsoft Word automatically. The unique characteristics of these four components of each prototype will be explained in the following sections, and the detailed description of each component in the final version will be given in Chapter 4.

1. It should cover related content. 2. Both theoretical models and practical guidelines should be included.

Choices

Prototype 2 (§ 3.4)


Classroom teaching Individual learning

Quality-driven education Test-driven education

Electronic curriculum

Basic knowledge and skills Extended knowledge and higher level skills Attitude + Collaborative learning

Multimedia curriculum Multimedia formats

-

-

*

Multimedia curriculum Presentation forms

+ Main Frame + Tutorial Designer's Aid Edit Panel From Designer's Aid + From the Main Invoked the same to Edit Panel Frame part to copy of other parts. Clicking the edit Designer's Aid, Edit button each time Panel, and Word. invoked a copy of Edit Panel.


*: see Section 2.3.1

Usage of a multimedia curriculum

Relationship between a multimedia curriculum and a conventional curriculum Goals of a multimedia curriculum

Concepts

Connection between the parts

Parts

Issues

Overview of the prototypes

To be continued on the next page

O N T E N T

C

T R U C T U R E

S

Table 3.1:

-

-

-

-

-

-


Prototype development 59


O N T E N T

C

Choices

Issues

-

Interface design Content selection Content description Content organization

Content elaboration Practice Test Experiment Simulation/virtual reality Case studies/ problem solving Content selection Content analysis Content description Content organization Interface design

Modules of a multimedia curriculum

Development elements

Biology Geography + Subject knowledge/skills

-


Computer skills

All subjects

Objectivism Constructivism


Learner analysis

Learning theories of multimedia curriculum design Subjects supported

Goals Content elaboration Resources Experiment/practice/case studies/ problem solving Test Collaborative learning Content selection Content representation Content organization Interface design

-

-

Move to the help file


-

Subject knowledge/skills Computer skills Learning styles Aims/goals/ objectives Content preparation Content elaboration Practice Assessment

-

-


60 Chapter 3

S


U P P O R T

Translation

Web site

Tutorial Concept mapping Flowchart

Preview Wizards

Learning materials Suggestions

Tips

Chinese

Theoretical models Practical guidelines Explanations of buttons Information about some operational skills Few Saved in local drivers Expert advice

Help

Hints

Used the status button to shift. Could be selected from the pop up menu.

1. Many types of support should be included


Issues

Explanations & Examples

Choices

Chinese

English

-

No proper tools were found.

Some wizards were removed. Very tentative 'Mindman' was linked.

-

What next Each element had one.

-

+ Links to the Internet

-

+ http://cascademuch.20m.com -

-

-

+ Intelligent expertise + Explanations + Explanations -

-

-

-

-

-


-

-

-

Added a hotspot of example to the explanation in the help file. Got an explanation first; got an example from the explanation window. -

-

-




I


N T E R F A C E

Keywords and new words used different colors (red & blue). Textual labels (||)

Colors

The tool buttons were temporarily located.

Temporary logos Not the same size

Content description used one textbox. Content organization used a table to illustrate the relationship.

Toolbar

Logos on screen Menu

Edit Panel

Buttons

Song (Chinese) Small size

1. It should be consistent. 2. It should be flexible.


Issues

Fonts

Choices

-

The 'Tutorial' button was moved to the help menu.

The content representation part added an indicator for each medium.

A main menu was added.

-

-

-

The content representation part was organized in an objectoriented style. Mouse right button could select or unselect knowledge units.

+ Home page and 'About …' items were added to the help menu. -

-

-

-

Song, Comic Sans MS Medium size

Song, Times New Roman Small size Keywords and new words used the same color. Each button had an icon and text. The 'Tutorial' button was on the first screen. Short-cut keys were added to some buttons. The tool buttons were grouped: one group was for all screens and another group was for specific screens. All logos had the same style and size. -




62 Chapter 3

Note:

C E N A R I O

S

Choices

- = No changes;

1. It should include much information. 2. It should be well structured.

Issues A paper-based scenario was made manually.


Analysis Modules to be included Flowchart Interface description Description of each knowledge unit (blank) = not considered


+ = Newly added;

Fragments

Formulation

Scenarios could be exported to Microsoft Word automatically. Revised description of each knowledge unit.


-

-



64

Chapter 3

3.3 The first prototype In order to offer flexibility to both novice designers and experienced designers of a multimedia curriculum, the first prototype was split up into two parts: Designer's Aid and Edit Panel. The first round of prototyping focused on the components of content, support and interface. The concept of a scenario was not formulated yet. In this section, the main characteristics of the content, support and interface will be explained. 3.3.1 Content

Concepts and relationships In the first prototype, the new form of curriculum was temporarily called electronic curriculum, and the regular curriculum was called conventional curriculum. According to Taba (1962), a (conventional) curriculum is usually defined as a plan for learning, mostly presented on paper. However, an electronic curriculum is often presented with many electronic formats. In this prototype, an electronic curriculum was temporarily defined to be a plan for learning with electronic formats, especially in multimedia formats on a computer. The relationship between an electronic curriculum and a conventional curriculum is depicted in Table 3.1. First, it was thought that an electronic curriculum was a new form of a curriculum, integrating more ICT (especially multimedia technology) and presented with more media formats. Second, an electronic curriculum and a conventional curriculum might include the same components, such as objectives and content, since an electronic curriculum was still a kind of curriculum. Third, compared to a conventional curriculum, the interface was thought to be an important component of an electronic curriculum, because it is also a computer program.

Goals and usage As explained in Section 1.1.1, the ongoing educational reform in China is intended to stimulate the shift from test-driven education to quality-driven education. However, any educational change is a complicated process (Fullan, 1991), which usually takes a long time. At the moment of designing the first prototype, test-driven education was still dominant in China. The goals of designing an electronic curriculum in the first prototype were to meet the needs of:


65

quality-driven education; and/or test-driven education. In China, classroom teaching has a long history and will continue to be used in the near future. In particular, nowadays computers are not very popular in families, and classroom teaching was assumed to remain for a long period (cf. Brock, 1994). However, with the rapid growth of personal computers, individual learning with computers outside of classrooms will become more and more popular, since it can provide a rich learning environment that enables students to freely experiment, test, and invent (Heinich, Molenda, Russell & Smaldino, 1996). Based on these considerations, in the first prototype the usage of an electronic curriculum was provisionally determined to facilitate: teacher's use for classroom teaching; and/or learner's use for individual learning.

Learning theories Designing a multimedia curriculum was a new challenge, in particular aiming at improving quality-driven education (see Section 1.1.1). It seemed appropriate to follow some existing learning theories in order to better achieve the design goals of a multimedia curriculum. The first prototype intended to follow two learning theories: objectivistic and constructivistic learning theories. According to Jonassen (1991a), objectivists believe in the existence of reliable knowledge about the world; and the objectivistic learning theory focuses on the transfer of knowledge from instructors to learners. Also, assessment is often carried out to check the degree of learners' ability to replicate or recall the transferred knowledge. It seemed to be rather consistent with the test-driven education. However, constructivists claim that reality is more in the mind of learners; and constructivistic learning theory highlights the knowledge construction process of learners. It seemed that the constructivistic learning theory might have positive impacts on the qualitydriven education. The first prototype provided a list of modules. It was expected that teacher-designers could pick up proper modules based on the learning theories to achieve the design goals. Subjects and learner analysis Each subject may need the support of multimedia, and it would also be possible to present each subject on a computer. Consequently, the first prototype did not focus on any specific subjects. It was expected to be able to provide generic support for all subjects.

66

Chapter 3

Learner analysis may include the analysis of general characteristics, prerequisite knowledge/skills and learning styles (Heinich, et al., 1996). In the first prototype, learner analysis focused on the computer skills. The teacherdesigners were assumed to have enough information about the learners concerning subject-related knowledge/skills and the general characteristics, since the teacher-designers were subject teachers and the learners were rather stable. In addition, as different learners might have different learning styles, a plausible way of meeting learners' needs with various learning styles would be to include as many as possible different types of learning materials and/or activities in a multimedia curriculum (cf. Fenrich, 1997). However, teacherdesigners might need further information about computer skills because they usually do not know what computer knowledge/skills the learners should have.

Modules Since an electronic curriculum is also a type of instructional software, its modules should be closely related to instructional strategies often included in instructional software. Instructional software is usually classified into tutorial, drill-and-practice, testing, micro-world, cognitive tools, case studies, and cognitive apprenticeship (Fenrich, 1997; Roblyer, Edwards & Havriluk, 1997; Zhu, 1996). Similarly, in the first prototype an electronic curriculum was classified into the following modules: 1. Content elaboration A tutorial program often includes content elaboration and practice. In the first prototype, the practice became a separate module. Furthermore, the content elaboration part would be enhanced in an electronic curriculum since content is an important part of a curriculum. 2. Practice Practice is usually an effective way of improving learner's knowledge and/or skills. Normally a practice module starts with questions, followed by diagnosis and immediate feedback. 3. Test Compared to practice, the aim of testing is to check whether learners have grasped knowledge/skills. Usually, a test should be done within a limited time. After it is finished, a summary of learners' performance including time and score is given. 4. Experiment In some subjects such as Biology, experimentation is a very important approach to knowledge acquisition and/or skills improvement. An electronic curriculum was expected to provide a rich experimental


67

environment in which learners can pick up virtual instruments, set up experiments and observe results. 5. Simulation/virtual reality Simulation normally presents or models the essential elements of a real or imaginary situation, in which learners can learn by manipulating on-screen models in ways that resemble real-world interactions (Fenrich, 1997). In particular, when a knowledge unit is a complex process, computer simulations may serve as an effective approach. Virtual reality, also called virtual environment, virtual world, or cyberspace (Biocca, 1992), is a more 'realistic' computerized simulated environment, in which learners can feel and see three-dimensional objects or images with electronic gloves and glasses. Several benefits of virtual reality can be found in literature, such as improvement of display, enhancement of control, and involvement of learners (Roblyer, et al., 1997). 6. Case studies/problem solving Constructivists believe that learners are best motivated by real problems in authentic situations (Coleman, Perry & Schwen, 1997; Jonassen, 1991a). Case studies and problem solving were expected to provide an authentic micro world in which learners can investigate the cases/problems by analyzing its complicated elements and relationships. It was also expected that knowledge gained from the cases or authentic problems could be easily transferred to other situations (Vockell & Schwartz, 1992).

Development elements Here the development elements refer to the activities to be performed when designing the six modules described in the previous section. Content and interface were considered to be two basic elements since an electronic curriculum is an integration of a conventional curriculum and instructional software (cf. Moonen, 1999). Specifically, it was intended that the modules be designed through the following activities: Content selection could be carried out based on the available curriculum standard and some criteria such as relationship with the curriculum goals, importance and difficulty, frequency included in tests, etc. Content analysis aimed at providing support for content description. The preliminary idea was to use the Allen's model (see Table 2.1) to guide content description based on the results of content analysis. Content description aimed at representing the selected knowledge units with various media such as text, pictures, sound, and video, etc. Content organization aimed at sequencing the selected content in a logical and/or flexible way. In the first prototype, three styles of content

68

Chapter 3

organization were outlined: linear, tree, and hypertext/hypermedia. The linear style meant that information was organized in a linear sequence. The tree style meant that information was organized into different levels in a hierarchical structure, in which users could go up or down. The hypertext/hypermedia style meant that information was organized into an interconnected network, in which users could go to any related pieces of information by following their embedded links. Interface design aimed at selecting proper interface styles for various modules. Due to the fact that the intended target users (subject teachers) were not entirely computer literate, they might expect to encounter difficulties while designing an interface. To solve this problem, a list of representative interface styles was provided from which they could directly select a proper interface. In the first prototype, three interface styles for an electronic curriculum were summarized: classic and elegant, modern and popular, vivid and vigorous. These three interface styles remained through the final version. 3.3.2 Support In the first prototype, the following specific types of support were included: explanations/examples help hints tips materials suggestions translation. Some key or new words needed elaboration for intended target users, especially for new users. The keywords referred to some important concepts related to multimedia curriculum design. The new words referred to some fresh words for subject teachers. In the first prototype, in addition to an explanation, each keyword had an example, but each new word did not. Clicking each keyword would get an explanation or an example depending on the status button on the toolbar. If the status button was depressed, clicking a keyword would call up an explanation, otherwise it would get an example. In addition, if users clicked keywords with the right mouse button, a pull-down menu would pop up, from which users could visit explanations or examples. Two kinds of examples were provided. One kind was shown in the help Window, and another kind was displayed in --automatically launched-external applications. In order to be more consistent, all examples were related


69

to the MCB project (see Section 1.2.1). Some theoretical models and practical guidelines were included in the prototype, aiming at explaining why such screen content was selected and improving users professional knowledge for multimedia curriculum design. Clicking on the help button on the toolbar could access such help. Some buttons on the screens --in particular speed buttons-- might not display enough information to indicate what the buttons were or intended to do. The hints were designed to show such pieces of information when the mouse moves over (speed) buttons or hotspots. The tips were designed to show pieces of information about how to perform tasks efficiently in the form of a 'tip of the day'. Few supporting materials such as available pictures, animation and video clips were collected and put together with the prototype. Some expert suggestions on how to select options or design an element were provided in the first prototype based on the users' profiles and the embedded expertise. The first prototype was developed in Chinese, but the translation function was designed to make the prototype usable in English. The detailed explanations of these kinds of support can be found in Section 4.3. 3.3.3 Interface When designing an interface, designers need to consider a lot of attributes such as learnability, satisfaction, memorability, error rate and efficiency (Nielsen, 1993). These attributes are mostly interconnected or affected by each other. For example, users may feel satisfied if an interface is efficient, no errors are occurring or has an attractive design; if an interface has attributes making it easy to memorize, then the error rate may be reduced. Brown (1988) summarizes some useful guidelines for interface design, including consistency; physical analogies; expectations and stereotypes; and ease of learning and use. During the process of interface design, it is necessary to establish a balance of these attributes/guidelines and emphasis may be shifted. For example, consistency should be considered from the beginning to the end, while error rate is not so serious at the beginning. Based on the above attributes/guidelines, the interface of the first prototype was formulated. Figure 3.2 shows a screen dump of Designer's Aid in the first prototype.

Fonts and colors In order to be consistent, the fonts and color for the same titles or at the same positions on different screens were kept the same. However, there might be some different fonts or colors on one screen for different purposes. For

70

Chapter 3

instance, all titles on different screens used the font 'Song' (Chinese) with size of 14 and style of bold, but the normal text used the same font with a smaller size and the normal style. The background color was gray throughout. The screen elements mostly used the same shade of black, but the keywords and new words used two different colors: red and blue.

Figure 3.2: A screen dump of Designer's Aid

Buttons, toolbar and logos There were two kinds of buttons: speed buttons and regular buttons. Most speed buttons were located on the toolbar for quick access to support, while the regular buttons were placed at the bottom of the screens and used for linearly browsing the prototype. The speed buttons were usually smaller with logos on them (without captions). However, the regular buttons were bigger and incorporated both logos and captions. In the first prototype, each screen and each button had a logo or an icon on it, but all logos were provisionally selected. Some of them were not very meaningful, and not all logos were of the same size. Also, the speed buttons on the toolbar were randomly located.


71

Edit Panel The Edit Panel part aimed at carrying out the detailed design of each element. The content selection could be performed through a checkbox. The content analysis could be performed by specifying the type of content (such as a fact or concept) through a list of radio buttons. The content description presented the selected knowledge units using various media formats. The process was that users first specified the media format they wanted to use (such as a picture) for describing the knowledge unit. Then, they could describe the attributes (such as colors, size, location and source, etc.) of the media format in a shared textbox. The content organization could be carried out by specifying the organizational relationships between the 'cells' of: i) knowledge units; ii) sections; iii) chapters; and iv) knowledge units, sections and chapters. The complex relationships among the above cells was illustrated in a table, in which the top row and the left column listed all selected knowledge units, sections and chapters. The interface design was carried out through selecting a proper interface style for a section and/or a chapter. In addition, the connection between the Designer's Aid and Edit Panel parts was that Designer's Aid provided a link (i.e. an edit button) to Edit Panel. Clicking the link would invoke Edit Panel to execute. However, due to technical limitations, each click would launch a separate copy of Edit Panel. This might cause two problems. First, too many copies of Edit Panel might cause Windows to freeze up. Second, users could not continue the work in the former copies. 3.3.4 Expert appraisal and micro evaluation After the first prototype had been formulated, a formative evaluation was carried out in Shanghai, focusing on the validity and potential practicality of the prototype. In this section, the formative evaluation activities will be elaborated in more detail.

Research question and participants The research question of the formative evaluation was: To what extent is the prototype valid and will it be practical for the intended target users?

72

Chapter 3

The following sub-questions were expected to be answered by the following types of participants: Intended target users (n=5) - Will the prototype be helpful for you in designing an electronic curriculum? - Can you learn some information from the prototype? - What are the main difficulties when you are using the prototype? Curriculum/instructional designers (n=2) - Are the explanations of keywords accurate? - Do you think the information included in the prototype would be helpful for the intended target users? - What information should be changed or added? CBL designers (n=3) - Do you think the content included in the prototype will be useful for intended target users? - Do you think the prototype is easy to use? - Do you agree with the choice of fonts, colors, buttons and other screen elements? - What should be changed or improved? Five intended target users took part in the initial formative evaluation. Among them, two were Biology teaching researchers, two were Geography teaching researchers and one was a Geography teacher. The two Biology teaching researchers gained experience in scenario development during their participation in the MCB project (see Section 1.2.1). The two Geography teaching researchers and single teacher had experience in multimedia instructional software design, but did not have experience in instructional scenario development. Two curriculum/instructional experts were involved in the formative evaluation, focusing on the component of content. Three students, working towards their master's degree in multimedia instructional design, participated in the formative evaluation, focusing on interface design, support and content.

Approaches, instruments and procedures The main approaches employed during the formative evaluation were expert appraisal and micro evaluation. The main instrument was a list of questions – the same as the sub-questions shown above. The formative evaluation was carried out at several times. The two Biology teaching researchers and one Geography teaching researcher did a micro evaluation as a small group, since they worked in the same district educational college. The other Geography


73

teacher and single Geography teaching researcher did micro evaluations separately at two different times. Because the prototype was very superficial and the five intended target users were off campus, they walked through the prototype using screen dumps. Because the two curriculum/instructional designers were close by, they gave expert appraisals on a computer individually at two different times. The three CBL designers did a micro evaluation as a small group on a computer operated by one of them. At the start of each evaluation meeting, the evaluator first gave a brief introduction of the main aim of the prototype as well as the aim of the formative evaluation. It usually took about five to ten minutes. After that, the participant(s) walked through the prototype in different ways as explained above. During this process, the evaluator briefly explained the elements on screens they were visiting, answered their questions, and wrote down their comments and suggestions. This process mostly took about one and a half hours. Each session was wrapped up with a short discussion. During the discussion, the participants were invited to answer the questions from the question list mentioned above. Their answers to the questions were written down by the evaluator. In total, the duration of each formative evaluation session was about two hours.

Comments and suggestions The main points of the various participants' comments, suggestions and answers to the questions are described as follows. Intended target users a1. They agreed that the prototype would be useful for them to make instructional scenarios, but they thought that users needed to have competent computer skills, otherwise they might have difficulties in using the prototype. Also, they suggested adding an easy-to-use tutorial, to help users learn how to use the prototype. a2. They believed that they learned some useful information from the prototype. Three users mentioned that the style of content organization using hypertext/ hypermedia was new for them. a3. All of them showed great interest in the embedded examples, and expected to have more added. a4. They said that they did not find the differences among the three interface styles from the screen dumps.

74

Chapter 3

a5. One participant mentioned that she expected to know the results caused by the current settings immediately. a6. Most of them mentioned that the content description part was very important and should be enhanced. But in the first prototype, this part seemed to be too simple. a7. They criticized some inappropriate labels such as 'often included in tests' and 'test-driven education'. They said that these labels were not consistent with the current educational reform trend. Curriculum/instructional designers b1. Both of them thought that the prototype was designed in detail, and they believed that the prototype would be useful for subject teachers to make instructional scenarios. b2. They agreed that the embedded examples would be very helpful for users. b3. One instructional designer suggested not using new terms, such as 'qualitydriven education' and 'nine-year compulsory education', because these new terms were either not well defined, or not commonly agreed upon. Alternatively, they suggested using the often-used and popularly accepted terms. b4. They thought the design aims, usage, and electronic curriculum modules were well chosen, but the rationales should be stated explicitly. b5. They thought that the learner analysis part looked simple; more information should be added. b6. They thought that the three interface styles were not enough; more styles should be added. b7. They suggested that the content description part should be elaborated in more details, because it was a key and useful part for users. CBL designers c1. They agreed that the user interface looked beautiful and consistent. The buttons and logos were at the same positions, and the screens looked attractive. The hierarchical and linear browsers provided users with flexible ways to use the prototype. c2. They thought the embedded examples would be useful for intended target users. They suggested adding more examples. Also, they mentioned that the tips at the bottom of each screen would provide users with useful information. c3. They mentioned that some screen elements could be designed better. For example, the signs '' on ' < Back' and '> Next' buttons may lead to misunderstanding. Users may doubt whether they should click the '

computer support for multimedia curriculum design - CiteSeerX

computer support for multimedia curriculum design - CiteSeerX

Suggest Documents

Computer-Based Support for Curriculum Designers - CiteSeerX

Computer-Based Support for Curriculum Designers - CiteSeerX

Multimedia Support for Databases - CiteSeerX

Computer Support for Curriculum Developers: CASCADE

Computer Support for Curriculum Developers ...

Computer Support for School Based Curriculum Development

beehive: global multimedia database support for ... - CiteSeerX

protocol support for distributed multimedia applications - CiteSeerX

System Support for Mobile Multimedia Applications - CiteSeerX

Storage Systems Support for Multimedia Applications - CiteSeerX

computer support for cooperative design - Semantic Scholar

Computer Support for Distributed Asynchronous Software Design ...

computer support for cooperative design - Semantic Scholar

Empirically Derived Multimedia Design Guidelines for ... - CiteSeerX

The Ideal Computer Curriculum - CiteSeerX

Multimedia-Systems: Design - Department Of Computer Science

Network Support for Multimedia - Montefiore

Multimedia support for MINIX 3

Curriculum-Focused Design - Cambridge Computer Laboratory

RMCase: Computer-Aided Support for Hypermedia ... - CiteSeerX

Runtime Support for Virtual BSP Computer - CiteSeerX

Computer Support for Distributed Collaborative Writing - CiteSeerX

Computer Support for Vicarious Learning - CiteSeerX

Designing interactive multimedia materials to support ... - CiteSeerX