A Cost Estimation Model for Multimedia-Based ... - CiteSeerX

A Cost Estimation Model for Multimedia-Based Courseware Development Nikolaos Papaspyrou 

Symeon Retalis 

Anastassios Koutoumanos  ([email protected]) ([email protected]) ([email protected]) Emmanuel Giakoumakis y Emmanuel Skordalakis  ([email protected]) ([email protected])  National Technical University of Athens, Department of Electrical and Computer Engineering, Division of Computer Science. y Economic University of Athens, Department of Informatics.

Abstract

New developments in information technology, like computer networks and hypermedia systems, enhance the role of computers in teaching and learning. Indeed, parts of the teaching process can be automated and the learning process can be helped substantially. The necessary software that accomplishes this goal is termed \courseware" and its development is not an easy task. One of the problems that are related to the development of courseware is that of accurate cost estimation. In this paper, we present a novel model for cost estimation in multimedia-based courseware development, using an algorithmic cost estimation mechanism based on the work breakdown structure of a typical courseware application. We suggest the \page", i.e. a screenful of educational material, as the basic unit in our size metric. Although our model has not been adequately tested yet, the rst results are promising.

1 Introduction The introduction of computers, and in general of the Information Technology, in education aims at partially automating the teaching task and facilitating the learning task in a new teaching and learning environment. It is anticipated that in such an environment some complex concepts will be better explained, various topics will be discussed in a more challenging way, less time will be spent to explain the same issues over and over again, better ways will be found to motivate the students, more information (feedback) about how well the students are doing in their courses will be received. The computer will act as a patient, tireless, impartial and motivating servant. The accomplishment of these goals requires the development of software which is speci c to this application area and is termed educational software [Self90] or courseware. As it is well known, visual stimulus is the most important means by which our central nervous system comes into contact with the external world. It is, also, generally accepted that sound and pictures bring with them tremendous mnemonic advantages [Fras90]. Re ecting this remark, contemporary courseware combines dierent media in the nal educational product. These media are typically:  text  graphics  sound  video  animation The multimedia approach, is an obvious match for training and education because of its ability to associate sound and pictures with text, and perhaps allow students to have fun while learning [Howe92]. For implementing the required courseware by following the multimedia approach, an appropriate environment must be used, which is termed \authoring environment". Authoring environments belong to three categories:  General Purpose Languages (e.g. C/C++, BASIC, Common LISP, Pascal). Considerable programming expertise is needed in this case.  Authoring Languages (e.g. PILOT, PODIUM, TUTOR, COURSEWRITER). These are special purpose languages, speci cally designed to facilitate the writing of courseware in the same way that simulation languages (e.g. GPSS) make it easier to program simulation models or arti cial intelligence languages (e.g. PLANNER) make it easier to conduct arti cial intelligence research. Despite the best of intentions, with few exceptions authoring languages are just as complex and require just as much time to learn as any other general purpose programming language [Kear82].

1

 Authoring Systems (e.g. TOOLBOOK, Course of Action, Authorware, ProPi, Quest). These systems intend

to assist non programmers in developing courseware in a relatively eortless and error-free way. They act like program generators [Velj90]. The user utilizes a high level interface to describe the software to be produced. This description is actually a speci cation of the intended software; prospective users specify, they do not program. Despite the reduction in the cost of hardware and improved functionality of authoring environments, the development eort required to produce multimedia courseware is still substantial. Courseware developers do not seem to have the ability to accurately estimate the development time and cost [Mars94]. Although in the software engineering bibliography a large number of metrics [Keme87, Pres94] exist to predict the cost of a software project, only few metrics exist for courseware development projects and their ecacy is uncertain. The aim of this paper is to present a model for cost estimation in courseware development, based on the breakdown structure [Tans88] of a typical multimedia application. Our model utilizes the \page", that is, a screenful of information, as the unit of our size measure. In section 2 we illustrate and discuss similar attempts in this area and present the innovative aspects of our research work. In section 3 we de ne the metrics and analyze the cost estimation model. The results of some case studies, used as test beds for evaluating our model, appear in section 4. Finally, section 5 presents our conclusions and comments on our future work.

2 Related work Courseware development is not an easy task. According to the extended literature in this area of software engineering, the process of such a development generally consists of six stages:  Analysis : setting the objectives and goals of the project; determining the approach; and ascertaining the hardware and software requirements.  Design : producing a storyboard and owchart for the program (a visual representation of what the program should do | including the design of the various media).  Programming : coding the actual software and making sure that it conforms to the desired objectives and

owchart.  Documentation : writing a student guide and a faculty manual, if the program is expected to be used by other faculty.  Evaluation : testing the program both during and after its development to see that it runs smoothly and according to its requirements.  Maintenance : providing for updates that will x bugs and incorporate improvements into the program. In fact, courseware development is an iterative process where these stages often overlap. Various instructional design and courseware engineering models have been proposed, all including the above stages [Tenn93, Mars95, Perr95]. Most of these models are used as a basis for cost estimation of courseware development. Five major methods for estimation of courseware development eort can be found in literature [Mars94]:  Educated guesses : the knowledge and experience of previous projects in addition to the analysis of the customer's requirements, allow the project developer to draw an estimation.  Industry averages : the developer is based on estimates of courseware development eort which cover a wide range of dierent types of courseware.  Air force interactive courseware method : a model is based on expert opinion of factors which aect the development time for various types of courseware, depending on the level of student training [Gola93].  Q factor analysis : uses a quality \Q" factor for estimating the development cost, which is similar to the cost per unit area used in house building [Berg90].  Algorithmic methods for courseware estimation : Two algorithmic methods exist: 1. Computer Based Training (CBT) analyst, which is a tabular method [Kear85]; and 2. Cost Estimation Algorithm for Courseware (CEAC), which uses an equation to calculate the development time by summing the contribution of tutorial, drill and practice, simulation and certi cation elements [Scho88]. Analysis of these methods showed that there is a large deviation between the expected estimates and the actual results, as clearly mentioned in [Mars94]. Among the weaknesses of some of these methods, one should note the following:  there is no clear de nition of the base units,  the estimate is based on incomplete or unreliable data,  the estimate is based on subjective remarks; no external validation exists, and  the estimate contains redundant cost factors.

2

Courseware

Course

Module

Page

Element

Module

...

Page

Element

...

Course

...

...

Course

Module

Page

Element

Figure 1: The work breakdown structure for a typical courseware application. An attempt to overcome these weaknesses appears in [Mars95]. However, this method is based on the waterfall life cycle model, although in the same paper it is mentioned that this is not the most appropriate for a multimedia courseware project. Moreover, the stages of analysis and maintenance are excluded, as it is dicult to calculate their contribution to the overall development cost. Taking into consideration the aforementioned approaches and their weaknesses, we present in this paper a dierent approach towards cost estimation in multimedia-based courseware development. One innovative aspect of our model is that we are not assuming the use of a speci c life cycle model. We concentrate on the multimedia application; using the breakdown structure technique [Tans88], the application is recursively divided into subparts. This division ends up with the basic units, which are easily recognized and clearly de ned. The use of the page as a unit of our size metric is also an innovative aspect. The suggested model is algorithmic and meets most of the criteria of Boehm and Wolverton [Boeh80].

3 Cost estimation In the following paragraphs, we present our model for cost estimation in multimedia-based courseware development. Paragraph 3.1 presents an overview of our model, based on the work breakdown structure. In paragraph 3.2 we suggest a size measure for courseware, we criticize our suggestion and we revise the work breakdown structure. Paragraphs 3.3, 3.4, 3.5 and 3.6 present in detail the various cost components of our model. In paragraph 3.7 we discuss the calibration process of our model for a given developer group and authoring environment. Finally, in paragraph 3.8, we attempt a theoretical evaluation of our model.

3.1 Overview of the model

The approach to cost estimation in courseware development that is suggested in this paper is based on the work breakdown structure technique [Tans88]. According to this technique, the software application is divided into parts, and each part into other subparts. In this way, the total cost is grossly equal to the sum of the costs of each subpart, which must therefore be estimated. The work breakdown structure for a typical courseware application tends to have a rather common form, which is supported directly by the majority of authoring systems. This structure is presented in Fig.1.

3

The courseware application is divided into courses. A course represents a quantity of related educational material that must be taught. For example, a courseware application intended for teaching mathematics to university students might consist of a course in linear algebra, a course in integral calculus, a course in dierential equations and so on. Each course can then be divided into modules, just as printed books are usually divided in chapters. There can be many levels of modules, if this clari es the structure of the educational material. However, for the sake of simplicity, both Fig.1 and the cost estimation model, that we shall present in the following sections, assume a single level of modules. Each module is a group of related pages. The page represents what the user sees on the computer screen and can be also be found as \screen" or \card" in the literature and the terminology of various authoring systems. A page can contain text, graphics, sound, video, animation and possibly other multimedia elements, depending on the application's requirements and the available hardware. The cost that will be related to each part in the work breakdown structure consists of secondary costs, corresponding to the following components of courseware development:  operational component  non operational component  documentation  contingencies The way in which these components aect the total cost of an courseware application is discussed in the following sections.

3.2 Size measure

In order to estimate the cost of developing a particular application, a measure of the application's size is needed. Software cost estimation is mostly useful in the early stages of software development, usually before the analysis and design phases are complete. Therefore, one should notice here that a size measure used in cost estimation inevitably refers to the application's expected size, since the actual size of the application is generally not known. A metric that has been reliable and widely used in software development is the number of lines of source code (LSC) [Albr83]. LSC can be applied to the development of courseware whenever this is developed by using a procedural language, or by using large amounts of code written in the built-in language of an authoring system. Nevertheless, in most cases this metric is not appropriate for this eld. Commercial authoring systems take pride in claiming that no programming is required, in order to develop an average courseware application. Besides, great eort and time is devoted to the creation of graphics, sound and other multimedia elements typical of courseware applications. This eort can hardly be measured in LSC and is considered to be more of an artist's job and less of a programmer's. It is clear that one needs to indicate a new metric for the size of courseware applications. The metric that we suggest originates from a typical courseware application's work breakdown structure, as depicted in Fig.1. It would not be wise to regard the elements of a page as the units in the work breakdown structure. These elements must be strongly related to each other and cooperate; the page's functionality relies on this cooperation. The necessary links usually take signi cant eort in designing, programming and testing. On the other hand, the page can be thought of as the basic unit of a courseware application, because the pages of a course module are |more or less| independent of each other. By adopting the page as the unit in a model for courseware software cost estimation, we implicitly choose the number of pages as a metric for the size of a courseware application. In this case, the cost of an application would be directly proportional to the number of pages. However, for the last statement to be true, one must assume:  that all the pages of a courseware application are of the same complexity; and  that the development of one page is entirely independent of that of other pages. Unfortunately, both assumptions are generally not true in practical courseware applications. All pages are not of the same complexity: there are simple pages, e.g. a page of contents, and complex pages, e.g. a page containing a ight simulator in a hypothetical course in aviation. Furthermore, contemporary authoring systems usually support a \background" feature, that is, similar pages can share the same background which implements the elements that are identical in all of them. The existence of a background feature invalidates the second assumption since, once a background for the rst page is developed, all subsequent similar pages can use it. Nevertheless, the fact that the two aforementioned assumptions are not generally true is not enough reason to abandon the number of pages as a metric for the size of a courseware application. We intend to overcome the problem by:  slightly changing the breakdown structure of a typical courseware application; and  allowing costs that are not proportional to the model's metric. We argue that there is a number of categories of pages in a courseware application. All pages in a given category have the same complexity, which depends on the level of nesse and sophistication that is required;

4

Courseware

Course

Module

Text

...

Module

Category

Category

Typical Page

number x of pages

Graphics

...

Course

Sound

...

Course

Module

Category

Background

Video

Animation

Other

Figure 2: The revised work breakdown structure for our cost estimation model. therefore, for a page category the rst assumption is true. We revise the breakdown structure of Fig.1 by adding a new level for page categories. The new breakdown structure is shown in Fig.2. In addition, we can satisfy the second assumption by not including the cost of the common \background" in the cost of each page of a given page category. In this way, the cost of each page is independent of other pages. The cost of the common background will have to be included separately for each page category.

3.3 Operational component

The operational component includes all the parts of software that compose the courseware application. The estimated cost of the operational component can be estimated according to the results of partial analysis and design. These results lead to a more or less accurate estimation of the number of courses, modules, page categories and pages. By using the revised breakdown structure depicted in Fig.2, the total cost of an application can be given by the following formulae: Cost[[Courseware]] =

X X

Num[[Courses]] =1

Cost[[Course]]

(1)

i

i

Cost[[Course]] = i

Num[[Modules]]i j

Cost[[Module]]

ij

=

=1

X

Cost[[Module]]

Num[[Categories]]ij k

5

=1

(2)

ij

Cost[[Category]]

ijk

(3)

where by Cost[[P ] we represent the cost of part P of the application and by Num[[P ] we represent the number of parts of type P . The subscripted indices denote a given course, module and page category, in this order. For example Num[[Categories]] represents the number of page categories in the j -th module of the application's i-th course. In this point, we have to deviate slightly from the breakdown structure, in order to comply with the discussion in the previous paragraph. The cost of a page category is not directly proportional to the number of pages in the category, but includes additionally the cost of the common background. ij

Cost[[Category]]

ijk

= Num[[Pages]]


Cost[[Page]]

ijk

ijk

+ Cost[[Background]]

ij

(4)

where Cost[[Page]] does not include any elements belonging to the common background. For a given page category, the cost of a typical page includes the costs of the multimedia elements that the page contains. In addition, we shall include the cost of designing, programming and testing the page. ijk

Cost[[Page]]

= Cost[[Text]] + Cost[[Graphics]] + Cost[[Sound]] + Cost[[Video]] + Cost[[Animation]] + + Cost[[Design]] + Cost[[Programming]]

ijk

ijk

ijk

ijk

+

ijk

(5)

ijk

ijk

ijk

A similar formula will give the cost of the common background. Cost[[Background]]

ij

= Cost[[Text]] + Cost[[Graphics]] + Cost[[Sound]] + + Cost[[Video]] + Cost[[Animation]] + + Cost[[Design]] + Cost[[Programming]] ij

ij

ij

ij

(6)

ij

ij

ij

The cost of each multimedia element can largely vary. Furthermore, it depends a lot on the element's origin. We shall distinguish the two extreme cases of a multimedia element's origin:  created from scratch, by using a suitable tool; and  copied from an existing repository of reusable elements. Many authoring systems are accompanied by large libraries, called clip-arts, containing commonly used graphics and sounds. The use of a scanner, a sound recorder and a electronic camera can signi cantly simplify the creation of graphics, sound and video respectively, and therefore help in the creation of such libraries. Similarly, electronic text can be easily created by using optical character recognition software on the scanned image of a typewritten text. Creating a multimedia element from scratch is generally expensive. The cost of using an existing one is usually much lower, although some times a copyright fee has to be paid to the owner of the existing repository. By considering only these two cases, the cost of all text elements of a page can be computed by the following formula. Cost[[Text]]

ijk

= Num[[Text]]

CostU [ Text]]

ijk

= R[ Text]]

ijk

ijk


CostU [ Text]]

(7)

ijk


CostR [ Text]] + (1 ? R[ Text]] )
CostN [ Text]] ijk

(8)

where Num[[Text]] is a measure of the amount of text in the page (e.g. number of characters or words), CostU [ Text]] is the cost of a unit of text, R[ Text]] is the fraction of text that can be found in an existing repository, CostR [ Text]] is the cost of a unit of text that is found in the repository and CostN [ Text]] the cost of a unit of text that is created from scratch. Similar formulae can be given for the costs of other types of multimedia elements. The two aforementioned cases of a multimedia element's origin are adequate for our cost estimation model. In practice, new multimedia elements can be created by changing existing ones, with the aid of appropriate tools. The cost of such elements generally depends on the number and the nature of changes to the originals. However, the use of such elements can be compensated in our model by setting appropriately the value of R[ M ] . The cost of a page's elements must cover the stages of analysis, design, programming and testing. In our model, this is partly re ected in the various costs Cost[[M ] . Nevertheless, we believe that the cost for the analysis and design of a page is not only equal to the sum of the costs for the analysis and design of the elements it contains. Additional eort will be needed, for example, in order to determine what these elements will be. This remark justi es the existence of Cost[[Design]] in equations 5 and 6, which corresponds to the cost of both analysis and design. The same is true for the stages of programming and testing, re ected by the existence of Cost[[Programming]] in the same equations, which corresponds to the cost of these two stages. Even when using an authoring system for the development of courseware, it is often required to use a conventional programming language, in order to perform something that the authoring system cannot do. This is usually the case for very complex pages, such as the ight simulator that was previously mentioned. In this case, the value of Cost[[Programming]] increases accordingly.

6

3.4 Non operational component

By non operational component we mean everything that is not part of the courseware application but was needed during its development. For example, when developing an application using a conventional programming language, it is often necessary to write code, in order to test the application's program units individually. This code does not become part of the nal application, but is essential for its debugging. The same applies to courseware applications, where the programmers are often forced to create elements and pages in order to test and debug individual parts of the application. Besides, during the development of courseware it is often appropriate to create prototypes of the required application, in order to validate or clarify the requirements. These prototypes are extremely useful when the developers are not certain whether a particular approach has better results than another. Such uncertainties are rather frequent since, as far as education is concerned, there are not objectively best solutions. The prototypes cover usually only a small part of the application and, after being evaluated, they are either re ned and used in the nal application or abandoned. Therefore, the cost of creating prototypes is not always included in the operational cost of the application. In order to compensate for non the operational component in our model, we will make the following corrections:  The term Cost[[Page]] in equation 4 will be substituted by ijk

(1 +  )
Cost[[Page]] ijk

ijk

where  represents the additional non operational fraction of elements in a page;  The term Cost[[Background]] in equation 4 will be substituted by ijk

ij

(1 + )
Cost[[Background]] ij

ij

where similarly represents the additional non operational fraction of elements in the common background of a page category; and  The term Num[[Pages]] in equation 4 will be substituted by ij

ijk

(1 + )
Num[[Pages]] ijk

ijk

where represents the additional fraction of pages that will be needed during the development but will not be used in the nal application. We should note here that the use of a given life cycle model sometimes requires the development of an excessive non operational software component. For example, the prototyping model, which is frequently used for the development of courseware, requires the development of several prototypes that are often abandoned after some results are obtained. In that case, it might be wise to treat the non operational component in the same way as the operational, that is, to make a thorough estimation of its size instead of using a multiplicative correction. ijk

3.5 Documentation

The documentation of courseware usually consists of three types of documents:  User's manuals (for the student).  User's manuals (for the teacher).  Internal reports. The need for two versions of user's manual arises from the fact that there are two groups of people involved in the use of courseware. The rst group consists of the students, that is the people who will be instructed, and the second consists of the teachers, that is the people who will instruct or survey. The dierences between the two versions depend on the characteristics of these two groups. The internal reports are written and used by the application's developers, during the various stages in the application's life cycle. Usually, one detailed report is written upon completion of each stage. The quality of these reports is determinative for the stage of maintenance. In the user's manuals |both for students and teachers| descriptions of guidelines for installing the application should be included, as well as the scope, the learning objectives and general information about the courseware. Furthermore, there should be detailed description of pages of each category, according to the revised work breakdown structure. A typical size metric for software documentation is the number of document pages [Pres94]. A model for the cost estimation of courseware documentation needs not dier much from the models commonly used for other

7

Table 1: Contingencies. Cost factors Contingency value range (%) Number of pages 10{20 Multimedia elements cost 25{50 Analysis and design cost 75{150 Programming and testing cost 25{50 types of software applications. However, in case that the application's breakdown structure is considered useful for the cost estimation of the documentation, our model consistently suggests the following formulae: Costdoc[ Courseware]] =

X X

Num[[Courses]] =1

Costdoc[ Course]]

(9)

i

i

Costdoc[ Course]] = i

Num[[Modules]]i j

Costdoc[ Module]]

ij

=

=1

Costdoc[ Module]] + Costdoc[ General]] ij

X

Num[[Categories]]ij k

=1

Costdoc[ Category]]

ijk

i

(10) (11)

3.6 Contingencies

Inaccuracies in software cost estimation are mainly due to the lack of the required detailed knowledge at the moment when the cost estimation task is performed and to a general optimistic tendency of every developing group. A way to compensate for these inaccuracies is to use contingencies. When LSC is used as the measure for software size estimates, the contingency can be expressed as a percentage of the estimated size in LSC. In our case, a contingency factor should be included in all size and cost measurements that are used in our model. More speci cally, the following elements are usually underestimated:  The estimated number of pages Num[[Pages]] for a given courseware application.  The estimated costs for the various types of multimedia elements CostU[ M ] .  The estimated cost for analysis and design Cost[[Design]].  The estimated cost for programming and testing Cost[[Programming]]. This underestimation can be compensated for by multiplying the estimated measurements by constant coecients. Similar models for estimating contingencies are frequent in literature, e.g. in [Hump89]. Exact values of the contingency coecients are hard to be found. The suggested value ranges presented in table 1, which result from the limited test cases that we have studied, are only indicative and can be used as a rule of thumb.

3.7 Calibration

The selection of values for the various parameters of our cost estimation model is a very hard task. Some of these values are highly dependent on the size and nature of the new project and the level of knowledge about it, for example the reusability factors R[ M ] for the various types of multimedia elements. Others are relatively independent of the new project and should be based on historical data, gathered through the development of previous projects. For example, the values used for the contingency coecients generally depend on the developing group's experience and maturity level. Concerning the parameters that are independent of the courseware to be developed, we believe that accurate estimates of their values can only be produced by a time consuming trial and error process, which is called calibration. In this process, speci c developing groups start with initial values for the model's parameters and continuously update them, by comparing estimates to actual costs.

3.8 Evaluation

The utility of a software cost estimation model can be evaluated according to certain criteria [Boeh80, Mars94]: 1. De nition : Has the model clearly de ned the costs it is estimating as well as the costs it is excluding? 2. Fidelity : Are the estimates close to the actual costs expended on the project?

8

3. Objectivity : Does the model avoid allocating most of the software cost variance to poorly calibrated subjective factors? 4. Constructiveness : Can you tell a user why the model gives the estimate that it does? 5. Detail : Does the model give an accurate phase and activity breakdown? 6. Stability : Do small dierences in inputs produce small dierences in output cost estimates? 7. Scope : Does the model cover the class of software projects whose costs you need to estimate? 8. Ease of use : Are the model inputs and options easy to understand and specify? 9. Prospectiveness : Does the model avoid the use of information which will not be known until the project is completed? 10. Parsimony : Does the model avoid the use of highly redundant factors or factors which make no appreciable contribution to the results? Our model has a clear de nition of terms and metrics used in as much detail as possible. Emphasis is given in illustrating a clear, easily understood algorithm which will not include redundant elements. Ease of use is achieved by the model's simplicity as well as the conciseness of its parameters. Concerning the \detail" criterion, our model primarily concentrates on the work breakdown structure and does not explicitly base the estimate on the various development phases. However, costs for analysis, design, programming and testing have been included in several equations. By using the work breakdown structure, the overall cost estimate can be based on speci c (non redundant) parts of courseware and be easily veri ed by the user. Thus, the criteria of \constructiveness" and \parsimony" are also met. In addition, the criterion of \stability" is met for two reasons. First of all, we use as a basic unit the page, which consists of various elements. Small changes in the elements that a page contains produce predictably small changes in the estimated cost. Besides, we do not use any metric for the experience or the subject matter expertise. Such parameters could cause a lot of problems in calculating their exact contribution and small changes could produce unpredictable changes in the overall cost. On the contrary, in our model the cost of each page will change predictably according to the expertise of the developing group. Furthermore, our model avoids making use of any information which will not be known until the late stages of the project. The values of all parameters that are needed in the algorithm can be speci ed after a preliminary analysis and design of the courseware to be developed. Thus, our model meets the criterion of \prospectiveness". The scope of our model extends to all kinds of courseware. We have avoided to base our cost estimation model on a speci c life cycle model, since there is great controversy in the literature regarding which is the most suitable for courseware development. However, our model can be easily adapted to dierent kinds of multimedia-based courseware projects, using any existing life cycle model. Further research should be done to provide evidence that our model meets the \ delity" and \objectivity" criteria, as further discussed in section 4.

4 Case study We have tested our model with a limited number of case studies, which can be divided in the following four major categories, with dierent characteristics. 1. Courseware for children. 2. Courseware for secondary education. 3. Courseware for higher education. 4. Courseware for professional training. In the tables to follow, we present the results of applying our model to four representative courseware projects, one from each major category. Namely, the projects were: 1. Pegasus : multimedia courseware based on a small (50 page) novel for children, entitled \With the wings of Pegasus" by Christos Boulotis. 2. Orion : an introduction to programming for high school students using the BASIC programming language, containing extensive theory, exercises and a small specialized BASIC interpreter [Tsan94]. 3. Pascal : an introduction to the programming language Pascal for rst year undergraduate students, containing theory and exercises. 4. Corel : multimedia courseware for training professionals in using the multimedia presentation package \Corel Show", containing extensive theory, examples and exercises. Table 2 presents the estimated values for the size of each project and the overall estimated cost without considering any adjustments due to contingencies. In a similar form, table 3 presents the actual values that were measured after the courseware's development. Tables 4 and 5 contain some of the model's parameters which were used in the cost estimation.

9

Table 2: Estimated cost. Courseware Number of Number of Number of Estimated cost project title courses categories pages (man hours) Pegasus 1 5 75 192 Orion 2 9 130 919 Pascal 1 7 150 286 Corel 1 5 100 316

Table 3: Actual cost. Courseware Number of Number of Number of Actual cost Deviation project title courses categories pages (man hours) (%) Pegasus 1 5 73 230 19.8 Orion 2 8 124 850 -7.5 Pascal 1 7 147 320 11.9 Corel 1 6 113 380 20.3 The results show that the actual cost does not deviate signi cantly from the estimated one. We should note here that no contingency adjustments were used, mainly because we did not have enough historical data at the time in order to calibrate the contingency parameters. For this reason, we believe that the deviations that were observed are reasonable. The value ranges of contingencies that were presented in table 1 de nitely cover the observed deviations. Our limited case studies do not allow us to claim that our model meets the criteria of \ delity" and \objectivity", as de ned in paragraph 3.8. As far as the \objectivity" criterion is concerned, an adequately large sample of statistic data is required before we can decide whether it is satis ed. Finally, concerning the \ delity" criterion, we believe that by means of a proper calibration, our model will result in estimates that will be much closer to reality.

5 Concluding remarks Cost estimation is one of the hardest tasks in the software engineering process. The approaches that are suggested in literature suer from known weaknesses, discussed in section 2. In order to overcome these weaknesses, we suggest a model based on the work breakdown structure analysis of a typical courseware application. The novelties of our approach lie in the fact that our algorithmic cost estimation model is independent of a speci c life cycle model, as well as in the suggested size metric. Furthermore, because of its simplicity and concise de nition, our model satis es most of the required evaluation criteria, discussed in [Boeh80]. We have not yet tested our model with an adequate number of case studies to be persuaded of its accuracy. However, the results from its limited application to selected representative cases are promising. Further research will primarily focus on the gathering of historical data and a proper calibration of our model. This will be attempted for our own developing group, as well as for similar groups in software houses of varying maturity levels. The aim of further research will also be to identify the contribution of other factors to the cost estimation in multimedia-based courseware development. Research is currently conducted to integrate the dierent roles played by members of the courseware developing group (e.g. educational material experts, managers, artists, programmers, etc.) into our cost estimation model. Based on our results so far, we believe that our work

Table 4: Model's parameters (reusability). Courseware Multimedia element reusability project title R[ Text]] R[ Graphics]] R[ Sound]] R[ Animation]] Pegasus 0.80 0.75 0.90 { Orion 0.00 0.50 { 0.00 Pascal 0.20 0.50 { 0.00 Corel 0.25 0.50 0.10 0.00

10

Table 5: Model's parameters (non operational component). Courseware project title Pegasus Orion Pascal



Non operational cost coecients

0.02 0.02 0.00

0.20 0.20 0.50

0.10 0.20 0.05

is in the right track and has the potential to evolve into an accurate cost estimation model for multimedia-based courseware development.

References [Albr83] A.J. Albrecht and J.E. Ganey, Jr., \Software function, source lines of code, and development eort prediction: A software science validation", IEEE Transactions in Software Engineering, vol. SE{9, no. 6, November 1983. [Berg90] R.B. Bergman and T.V. Moore, Managing interactive video/multimedia projects, Educational Technology Publications, Englewood Clis, NJ, 1990. [Boeh80] B.W. Boehm and R.W. Wolverton, \Software cost modelling: Some lessons learned", Journal of Systems and Software, vol. 1, no. 3, pp. 195{201, 1980. [Fras90] R. Fraser et al., \Learning activities and classroom roles with and without the microcomputer", in O. Boyd-Barrett and E. Scanlon, editors, Computers and Learning, pp. 205{228, Addison Wesley, 1990. [Gola93] K.C. Golas, \Estimating time to develop interactive courseware in the 1990's", in Proceedings of the Interservice/Industry Training and Education Conference, 1993. [Howe92] G.T. Howell, Building hypermedia applications, McGraw Hill, 1992. [Hump89] W.S. Humphrey, Managing the software process, Addison Wesley, 1989. [Kear82] G. Kearsley, \Authoring systems in computer based education", Communications of the ACM, vol. 25, no. 7, pp. 429{437, 1982. [Kear85] G. Kearsley, \The CBT advisor: An expert system program for making decisions about CBT", Performance and Instruction, vol. 24, no. 9, pp. 15{17, 1985. [Keme87] C.F. Kemerec, \An empirical validation of software cost estimation models", Communications of the ACM, vol. 70, no. 5, May 1987. [Mars94] I.M. Marshall, W.B. Samson, P.I. Dugard and G.R. Lund, \Courseware: How much will it cost to develop?", in 2nd All Ireland Conference on the Teaching of Computing, Dublin, Ireland, 1994. [Mars95] I.M. Marshall, W.B. Samson and P.I. Dugard, \The mythical courseware development to delivery time ratio", Computers in Education, vol. 25, no. 3, pp. 113{122, 1995. [Perr95] X. Perrot, Production des hypermedias et des interatifs multimedias pour les musees, Ph.D. thesis, Universite Paris 8, 1995. [Pres94] R.S. Pressman, Software engineering: A practitioner's approach, McGraw Hill International, european edition edition, 1994. [Scho88] R.B. Schooley, \Computer based training (CBT) cost estimating algorithm for courseware (CEAC)", in Proceedings of the Interservice/Industry Training Systems Conference, pp. 319{328, 1988. [Self90] J. Self, \A critical evaluation of educational software: Climate", in O. Boyd-Barret and E. Scanlon, editors, Computers and Learning, pp. 147{154, Addison Wesley, 1990. [Tans88] R.C. Tansworthe, \The work breakdown structure in software project management", Journal of Systems and Software, vol. 1, pp. 181{186, 1988. [Tenn93] R.D. Tennyson and R.L. Elmore, \Integrated courseware engineering system", in Automating Instructional Design, Development and Delivery, NATO ASI, Grimstad, Norway, 1993. [Tsan94] P. Tsanakas and G. Papakonstantinou, \Design and implementation of educational software for the teaching of informatics", in Proceedings of the 2nd Panhellenic Conference in Computer Science, Athens, 1994.

11

[Velj90]

M.D. Veljkov, \Managing multimedia: Authoring systems are the glue that holds multimedia applications together", BYTE, pp. 227{232, August 1990.

12