Conventional and Open Source Software Reuse at ... - Semantic Scholar

Conventional and Open Source Software Reuse at Orbotech – an Industrial Experience Shlomit Morad1 and Tsvi Kuflik2 1 Orbotech, 2University of Haifa 1 [email protected], 2kuflik@ is. haifa.ac.il

Abstract Orbotech, as part of the Israeli Software Reuse Industrial Consortium (ISWRIC), explored the possibilities of software reuse in a three-year project, supported by the Israel Ministry of Trade and Commerce. The positive economical results of the project made software reuse a common practice at Orbotech. Further experimentation of reusing Open Source Software (OSS) demonstrated the high potential of that approach, when carefully integrated with the standard organizational development process. The conclusions from Orbotech experience are that when carefully planned and implemented, software reuse provides the anticipated benefits of cost reduction, improved quality and shorter Time-to-Market. The reuse of OSS may provide even higher benefits than conventional software reuse. Nevertheless, as in many cases before, implementation of software reuse requires management support and commitment, as well as acceptance by the developers themselves. As such, software reuse implementation proves to be a complex task that needs to be tailored specifically to the implementing organization.

1. Introduction It is widely accepted that software reuse is a major component of many software productivity improvement efforts, because it can result in higher quality software at a lower cost and delivered within a shorter time period [5]. The recommended approach for software reuse is the adoption of the product line approach of Boehm [2]. This approach is detailed in [3], which describes a product line organized to operate in two circles: core asset development and products development. Core assets are software artifacts that are developed or acquired for the product line as a whole. These assets are later acquired as building blocks for specific products.

Software reuse is not merely a technical issue. On the contrary, the organizational challenges of software reuse outweigh the technical ones, as summarized by the “STARS” program report, for example [12], and by many other software reuse case studies like [1, 4, 11]. As a result, economical proofs supported by adequate metrics are needed in order to make “business decisions possible by quantifying and justifying the investment necessary to make reuse happen” [10]. In fact, there are many software reuse success stories where the economical benefits are evident in practice. For example, substantial costs were saved due to implementation of software reuse in the “STARS” demonstration project. The first system developed cost 43% of a reference baseline and a second cost only 10% [12]. Another example is the experience at Hewlett-Packard described in [8] where, by applying software reuse, a defect reduction of 15% was achieved and productivity increased by 57%. Other cases presented by Poulin [10] are just a few examples of the numerous success stories of software reuse. Jacobson et al. [5] suggested the “the incremental adoption of reuse”, described as a practical approach for transition toward becoming a domain-specific reuse-driven organization. However, the transition phase is lengthy, and every organization needs to find its own way. It is not obvious that the same approach is good for everyone. The need to change the nature of the organization while adopting software product line approach, the initial investments involved and the need to preserve current development activities, discourage organizations from implementing software reuse. In order to overcome the problem of initial implementation, the Israel Ministry of Trade and Commerce supported a consortium of eight system development companies, in a three-year project of studying and establishing software reuse. The mutual effort and the government support encouraged the management of these companies to support the experimentation.

Proceedings of the IEEE International Conference on Software - Science, Technology & Engineering (SwSTE’05) 0-7695-2335-8/05 $ 20.00 IEEE

The consortium members studied existing software reuse methodologies and various industrial software reuse cases and defined a mutual software reuse methodology. This methodology was later tailored to the individual companies and implemented in specific projects. The lessons were presented to the consortium members and later to the whole Israeli industry [7]. As part of the methodology, several practical software reuse scenarios were defined: systematic software reuse (following the classical software product line approach [3]), opportunistic software reuse, and controlled reuse. The first two are well known. The third is a practical compromise between the company’s budget, schedule and manpower constraints and the product line approach. In this scenario, software assets that are candidates for future reuse are kept in a repository, together with the definition of the required adaptations that will make them reusable. The assets are now available for reuse (while saving search and evaluation efforts needed in opportunistic reuse). A future project that uses such an asset will produce a reusable asset out of it, based on the adaptation requirements. An interesting finding of the consortium was the fact that adoption of software reuse is a slower process than expected. Most of the companies spend a lot of time in “transition” towards reuse-based organization as defined by [5], with gradual growth in software reuse, similar to what was demonstrated by [4]. It seems that the “transition” phase may even be a “steady state” – part of the organization produces reusable assets while other parts maintain their development process, while integrating some reusable assets. In this situation the reused software will not become a major part of the software products. The use of Open Source Software (OSS) has become popular in recent years, when more and more pieces of code, sub-systems and systems (such as Apache web-server, the Mozilla browser [9], Linux and the GNU project) become available. Taking a closer look at the OSS model, it becomes clearer that due to the nature of software development processes, integration issues and numerous business models, “there is no such thing as a free software” [6]. Nevertheless, OSS is an available source of potentially good software – this availability poses the challenge of how to adopt and reuse OSS as part of product development [14]. Orbotech, a developer of industrial solutions for the electronics industry, was one of the participating companies in the Israeli Software Reuse Consortium (ISWRIC). This paper presents the work done at Orbotech from the initial steps of software reuse efforts, from when management was convinced to authorize experimentation of software reuse, allocating

a single programmer to the task, until today, when software reuse has proven to be economically viable and has become a standard procedure at the company. In the process, in addition to the classical Software Products Line aspects, the possibilities of software reuse based on OSS were also exploited, again with positive economical results. The next section provides a brief description of Orbotech. Section 3 describes the process of software reuse adoption at Orbotech and its results and Section 4 provides conclusions and future plans.

2. Orbotech Orbotech is an Israeli systems development company that develops industrial machines and solutions for the electronics industry. It develops optical inspection machines for bare PCBs (Printed Circuit Boards), assembled PCBs, ICs (Integrated Circuits) packaging substrates and Flat Panel Displays (FPDs). It also develops verification stations, direct imaging and laser plotting solutions, Computer Aided Manufacturing (CAM) systems, and process control applications. Orbotech R&D has a staff of around 80 software engineers. The R&D organization in Orbotech is divided into three divisions, following three main markets. Each division is composed of development teams organized in product lines, marketing, and customer support. A software team is responsible for the software development and maintenance of a product or a product line. In addition to these divisions, the R&D organization includes a technologies and infrastructures division that supports the multi-disciplinary products. Since most of the products involve software, hardware, mechanics and optics, this division is organized by discipline – thus each department is focused on the needs of a single profession, its expertise, practices and tools. Each disciplinary group is also responsible for investigation of future technologies and making them available to the R&D projects. The software group of the technologies and infrastructures division is composed of three teams that are responsible for software development methodologies & standards, software quality assurance practices, development tools, computing environments, R&D servers, developing common reusable assets and developing products supporting tools for non-R&D organizations. Besides the development of common assets, the reuse team is responsible for maintaining the assets and assisting in their integration, search, evaluation and acquisition of OSS, as well as managing a central reuse repository.


3 Software Reuse at Orbotech Orbotech products are software intensive. However, even though many software assets can be shared among product lines due to their functional similarity, this has not been the case in practice. Every product line and often every product within the product line has its own software. Some experience with opportunistic reuse existed however. For example, when a software team moved from one product to another, the engineers reused the software of the existing product as the basis for the new one.

3.1 Initial Steps Although the products are divided into three distinct markets, the similarity among them is substantial: For all markets, Orbotech develops optical inspection machines that share similar operation approaches, image understanding techniques, motion mechanisms, and computing architectures. Above all, there are company-wide considerations like licensing, Look-&Feel, debugging tools, installation methods and customer support tools that may be common to all products in all markets. The manager of the software group in the technologies and infrastructures division was deeply involved with all software development activities in Orbotech. This involvement revealed the high commonality among the various software systems, and exposed the inefficiency of the development process. In 1998, she decided to launch a software reuse initiative with a single developer. This engineer developed (in 5 months) a single reusable asset (one executable and one C++ multi-platform library), within the security domain. Its first usage was immediate, but the second reuse did not occur in the same year or even the year after. In 1999, the Software Department Manager received another developer for studying a certain technology and preparing it for integration in new products. At the same time, the Customer Support organization asked for a special solution for its operations, and a third developer was added. The first asset developed for that solution was composed of 2 executables in Java and one C++ multi-platform library. The development of the first version took 12 man-months and it contains new technology not used before in Orbotech. This asset was used immediately in one product, but again, the next reuse of the asset took another year. At that point, the Software Department Manager prepared an initial list of 15 software components that

had the potential of supporting improved productivity and shorter Time-to-Market, by being reusable. Two components were quite large: one, a Graphic Interface tool, was estimated as a 10 man-years project, and the other, an Image handling application, was estimated as a 3 man-years project. The others, which were building blocks of a typical application on Orbotech machines, were estimated as 5-15 man-months each. At the end of 1999, the list was presented to project managers and team leaders, together with estimated effort and development schedule. The managers reacted enthusiastically and the list – detailing which projects are willing to use which assets – was the “winning card.” Top management approved three full-time developers for that purpose alone. Thanks to management commitment to software reuse, the commitment of the reuse team to the developers and their confidence in fair priorities, several new projects took the risk and committed to use the newly developed assets. The next asset was small (three man-months development effort), and it was integrated into two projects. The asset matched the requirements of both projects, and the integration took a few hours – 2-3% of the effort required to develop the same functionality that was actually used. Although there were two other assets (described previously) that were not reused (beyond the first use), the entire activity was a success. At the same time, the team has started developing a very large tool (three man-years for the first version) that encapsulated a difficult-to-learn technology with simple-to-use mechanisms. This tool was immediately adopted by 5 projects. The software development using this tool took 15% of developing the same application with the conventional tools used before. This was a real success. The next step was to build on the first successes for new assets: to achieve users’ commitment to use new components in advance, and to provide them with intermediate versions for early integration. At this point, getting this commitment was easier, and developers became cooperative and eagerly provided their requirements for future development of reusable assets. 3.1.1 Reuse Organization. As mentioned above, the software expertise, practices and development tools were already managed centrally. Hence, it was a natural decision to make this group responsible for software reuse as well. This was the basis for the central software assets development team that was formed in Orbotech. The advantages of having such a central team for reuse handling are clear:


• The team members are the only developers acquainted with all products. • The team members are the only experts in development of reusable assets (which is unique expertise beyond software development). • The team is committed to support all products. • The team is committed to maintain any software it ever developed. • The team priorities are derived from the organization priorities (not biased by individual project constraints). • The team is committed to promote software practices and technologies. 3.1.2 Software Reuse Strategy. The reuse team knew that projects will cooperate and use reusable assets, if – and only if – the project’s effort will be minimal. If they should need to study the asset in order to modify it or integrate it into their system, they will abandon it as unnecessary overhead. The selected strategy was thus based on the following principles: • All products (of all divisions) will be considered as a single, virtual, product line. Experienced engineers who were involved in multiple products confirmed that many software components are similar, or have the potential to be. • The reuse level will be code; meaning that all reusable assets will be code components. Reuse of requirements was considered impractical since requirements analysis is done informally. Reuse of design was considered to be constricted to real product lines and not accepted cross products; and reuse of testing was considered a premature technique. Code reuse however, was considered practical, beneficial and acceptable. • All assets will be planned for black-box reuse and will therefore be delivered as binaries rather than source code. The belief was that this is the only way that developers will see how simple the integration is, and how low is the integration overhead. • All assets will be developed from scratch. The belief was that developers do not trust other teams and may reject assets based on existing code taken from other products. They, however, can learn to trust the reuse team and willingly accept its products. 3.1.3 Software Reuse Implementation Tactics. The software reuse team knew they had a single chance to succeed, or the software reuse efforts would be discontinued. In order to increase the odds for success, the team followed the following development tactical principles: • Most of the first reusable assets should implement secondary, tedious and even marginal functions:

•

•

• • •

developers will be happy to eliminate such tasks and use existing components instead. At least one of the first reusable assets should introduce a new technology: developers will have a chance to use state-of-the-art technologies that they cannot bring into the project on their own. The developers will take an active part in the definition of the new assets: after defining asset specifications, the developers can not refuse to use them by claiming that the assets do not fit their needs. Since the products use various environments (operating systems and development tools), all assets will be available on all platforms. Assets integration will be closely supported by the software reuse team. Although the software assets will be delivered as black-box (binaries), the source code will be available as well (mainly for debugging purposes).

3.2 Current Status After three years, the reuse team acquired the developers’ recognition as an excellent engineering group that develops high quality software and provides outstanding support. This recognition was the basis for the third generation of assets – software assets that perform not only general services, but also core features and essential functions that were found common to multiple products. Moreover, new projects now start by inquiring what assets already exist (or planned in the near future) as an initial step of the project development plan. An important factor in the reuse success was – as expected – the black-box delivery: Although the source code is available, the assets are delivered and used as binaries (libraries or executables). Another important factor in this success is the reuse repository. The repository is composed of a vault with all the assets and their documentation, a database, which holds metadata about the assets, and an Intranetbased application. With this application, potential users can search for an asset, read its documentation, and previous users’ feedback. They can also download the asset and subscribe for news pertaining to it. The repository provides potential users with the ability to view, evaluate and download any asset without interrupting the reuse team. During the last four years costs and savings of the reuse operations were measured. Regardless of numerous metrics available, as presented by Polin [10] and Wiles [13], the only metric used was the development effort in man-days. The data is based on


comparison of actual measured development effort of one approach, to past experience (in Orbotech) of developing similar non-reusable components under the alternate scenarios. (Similar components means the same requirements/functionality as required by specific projects) These measurements were essential for the second buyin campaign carried out before the third cycle had started. The purpose of this campaign was to accept upper management approval for the planned move to get deeper into the applications and to develop reusable software assets with application core features. Today, four years after being established, the team includes 3-4 engineers (5% or less of the total software development manpower) who continue developing new assets while maintaining the existing assets and supporting their users. Eleven reusable assts were developed so far. All assets are multi-platform (Windows, Solaris & Linux) and are used in 16 projects. Six products use more than one asset and 7 assets use other assets. By reusing software, Orbotech saved 12-15 man-years over these four calendar years. In new products that use all the relevant assets, the reusable software can reach 10-15% of the software system. Table 1 presents the cost measurements/estimations comparison for two assets (called J and L). Table 1: Software Comparison Component name

Development

Options

J

L

Systematic Reuse (SR)

1660

133

New Development (ND)

3600

240

Opportunistic Reuse (OR)

3120

234

Controlled Reuse (CR)

2472

195

12

6

Savings of SR (new) over ND

54%

45%

Savings in development days

2000

97

100

5

Number of reuses

Saving in man-months

J is the large tool and L is a small component. All results refer to cumulative costs up-to-date. For every asset we have the measured development effort at the systematic reuse line and actual effort of developing the same functionality in past projects in Orbotech under all other alternative scenarios (all in man-days). The bottom three lines present the savings as relative cost compared to new development, and in man-days and man-months. In both cases, given the number of reuses, the systematic reuse is the best option. Looking at the

savings in effort, the total savings on these components alone is equal to 9 man-years! The accumulated data provides the following averages for software reuse at Orbotech in general: • Developing a black-box reusable component costs 160-250% of non-reusable (ad-hoc development) software. • Mining the existing software doubles the effort required for asset definition. • Integration of a black-box reusable component costs 1-3% of ad-hoc development. • Use of a development tool (rather than an application function) costs up to 15% of ad-hoc development. It seems that while the development of a new reusable asset is similar to numbers available in the literature, which is anywhere between 110% to 220%, relative to new development of non-reusable software as reported by Poulin [10], who accepts 150% as representative value. The integration effort is remarkably smaller, again, compared to the cases reported by Poulin, where most cases where 20% relative to new development of non-reusable software or above, few cases were less – 3% and 5%, but Poulin accepts the 20% as a representative value[10]. These minimal integration efforts, achieved due to careful modular design and thorough testing (as reflected by the high development costs) contribute to the software reuse success, not only in cost reduction but also in developers’ satisfaction.

3.3 Open Source Software Reuse at Orbotech Like many high-tech organizations, Orbotech’s R&D includes OSS fans (who will do whatever it takes to use open source components), followers (who use open source carefully but enthusiastically), and common developers (who will not search the OSS as a first option, but will not object to using OSS from time to time, when they have an opportunity). Indeed, OSS seems to offer an opportunity for development cost reduction due to its reliance on the cumulative efforts of OSS developers all over the world. However, integrating OSS into a product raises technical questions, like: Does the OSS support the specific product requirements? Was it developed and tested using the same development environment used in the organization? What kind of support may be available for that software? How are new version releases handled? All these issues need to be addressed when considering the use of OSS as part of a product. As stated by Karels [6] discussing the commercialization of OSS “there is no such thing as free software.”


3.3.1 OSS Scenarios. Due to the potential for development cost reduction offered by OSS, Orbotech decided to explore the possibility of combining it in its products. Again, as with classical software reuse (as presented briefly in the introduction), we can identify three similar scenarios of OSS based software reuse: Opportunistic Reuse A developer, who considers using OSS, searches for a component, evaluates it according to his specific requirements and downloads it. The code becomes an integral part of the project (compiled, integrated and tested as part of the system). This OSS use is done independently and unrelated to other developers/teams. Another project with similar requirements that could use that code is oblivious to it. Controlled Reuse When an open source asset is a tool, its reuse in Orbotech is closer to controlled reuse. Since all the tools in Orbotech reside on a central server, there are no multiple copies of the same tool. When a project finds such a tool, it uses the administration team to download the software into the server and compile it under the required platform(s). Another project may use it as is, or recompile it in another platform. Since the decision to use a certain tool was made by a single developer (or team) in a certain project, other projects may select other tools (OSS or commercial) for the same purpose. Moreover, there is no control over the quality of the software – its performance, robustness, scalability, reusability or documentation. Systematic Reuse Following this scenario, an OSS asset is selected, evaluated, compiled for the relevant platforms, (re)packed for delivery and uploaded into the reuse repository. At this point it becomes a regular reusable asset, like any in-house developed assets. 3.3.2 Systematic OSS Reuse at Orbotech. As presented earlier, systematic reuse was implemented by establishing a central reuse team that is responsible for developing common assets. As a strategic decision, this group became responsible for OSS as well. This responsibility was defined by the following principles: • OSS asset is selected based on a real need, after being evaluated against clearly defined requirements. • OSS asset should be evaluated for ensuring minimal quality, documentation, reusability and scalability. • OSS asset should be compiled for the relevant platforms, (re)packed for delivery and uploaded into the reuse repository.

• OSS asset should be studied in order to provide ongoing support to its users. • When not available from the original project, the reuse team should fix bugs and even adapt & expand the OSS asset with newer capabilities. The effort invested in each component in this approach is relatively small. The savings for the projects are huge – not only that integration is standard and simple, the developers are freed from searching the Internet, evaluating the software, testing the software and maintaining it. Moreover, the availability of in-house support enables the developers to use such components that otherwise would be ignored due to integration difficulties. Orbotech’s reuse team took the concept two steps ahead: 1. For each asset, the reuse experts examine the original requirements, compare them to the way the component is used and to the recommended way of using it, in order to identify weak points and difficulties. This information becomes the basis of requirements for a wrapping component – a piece of software that automates the most popular scenarios of using the component. In this case, the integration becomes as simple as the integration of any other component designed by this team. 2. With one component, the reuse team took an active role in the OSS development. After wrapping the asset (in order to guarantee a stable API), the team expanded the original OSS to new functionality and returned it to the (external) asset development team for integration. When the next release became available, the software team could remove those proprietary expansions that became part of the asset, while enjoying new features added by other developers. 3.3.3 OSS Reuse Results at Orbotech. The reuse team in Orbotech experimented systematic OSS reuse, using three OSS assets. Two assets are application components (become part of the products) and one asset is a development tool. All components are very large and complex, and their in-house development is cost-prohibitive and unfeasible. The only alternative is to purchase commercial components. In that case the OSS reuse is cost-effective from the get-go. For one of the three assets the expected reuse was in 3 products, and for the other two the expected reuse was 10 products. The graphs below display the comparison of costs to develop the assets in several development scenarios: • Development from scratch: each project will develop the subset of functions it needs (New Development).


• Opportunistic reuse: search for a similar asset, evaluate it and white-box reuse it. • Controlled reuse: get an existing candidate from the repository, adapt it for reuse and use it. • Systematic reuse (New): classical development of a new reusable asset by the reuse team. • Systematic reuse (Adapt): adapt an OSS asset for reuse. As before, only one scenario is measured (OSS adaptation) and all the rest are estimated, based on expert judgment. The following figures present the five software reuse alternatives (Systematic after adaptation, Systematic with a new asset, New (ad-hoc) development, Opportunistic reuse and Controlled reuse) for the three OSS assets. (The X-axis represents the number of reuses and the Y-axis represents the cost in man-days.) 600

500

400

300

200

12 0 0

10 0 0

80 0

60 0

40 0

20 0

0 1

2

3

4

5

6

Systematic (adapt)

Systematic (new )

Opportunistic

Controilled

7

8

9

10

New Development

Figure 3. Third asset When software reuse is based on the adaptation of an OSS asset, the savings begin immediately (which is no wonder since there is hardly any development, only search, evaluation and adaptation efforts). These cases emphasize the problematic aspect of classical systematic software reuse: the high initial investment required, and the time it takes until economical benefits appear. In contrast, if an adequate OSS asset exists, it can be adapted for reuse with relatively little effort and the potential benefits are obvious.

10 0

4. Conclusions and Future Plans

0 1

2

3

4

5

6

Systematic (adapt)

Systematic (new )

Opportunistic

Controilled

7

8

9

10

New Development

Figure 1. First asset Looking at figures 1-3, it is evident that new development (from scratch, every time) is always the worst case, while systematic software reuse (the asset is developed by the reuse team) seems economically viable in two out of the three cases. 3 00 0

2 50 0

2 00 0

15 0 0

10 0 0

50 0

0 1

2

3

4

5

6

Systematic (adapt)

Systematic (new )

Opportunistic

Controilled

7

8

9

10

New Development

Figure 2. Second asset In one case, its savings starts after the 4th reuse (Fig. 3), and in the second after the 11th reuse (Fig. 1).

From Orbotech experience so far, we can learn that even today, after about 30 years of experience in software reuse implementation, it is not a straightforward task. As is widely known, in addition to the need for management support, there is an initial investment and there is the basic resistance of developers to use something they did not develop on their own. In spite of the success, top management still hesitates to enlarge the reuse team or to empower it for critical tasks, and project managers still hesitate to ‘sacrifice’ an engineer for software reuse activities. Tailoring the software reuse implementation process to the specific organization, selecting the right strategy & tactics, involving the developers, getting their commitment and providing high quality reusable assets with on-site integration support, were the means with which Orbotech succeeded in the introduction of software reuse. Now Orbotech continues to enlarge the role of reused software in its products. As has been seen, this is an incremental, ongoing process that requires long-lasting commitment. Moreover, there were no major organizational changes; Orbotech does not practice the Software Product Lines approach fully, but is slowly moving in this direction and the process seems to be quite painless. In addition to classical software reuse, Orbotech successfully implemented a method for OSS


systematic reuse that seems to be highly productive. Future work will focus on developing assets for more crucial parts of the products, enlarging the reuse of OSS, and on trying to confirm these new initial findings in a larger scale.

5. References [1] P. A. Biggs, Case Study in Small Company Software Reuse. Available online at: http://students.cs.byu.edu/~pbiggs/smallcomp.html [2] B. Boehm, Managing Software Productivity and Reuse. IEEE Computer, 32(9) (1999) pp. 111–113. [3] P. Clements and L. M. Northrop, Software Product Lines: Practices and Patterns, Addison-Wesley, Reading, MA (2001). [4] S. Isoda, Experience Report on Software Reuse Project: Its Structure, Activities and Statistical Results. Proceedings th of the 14 International Conference on Software Engineering (1992) pp. 320-326. [5] Jacobson, I., Griss, M. and Johnsson, P. P. Software Reuse, Architecture, Process, and Organization for Business Success, Addison-Wesley, Reading, MA (1997). [6] M.J. Karels, Commercializing Open Source Software. QUEUE July/August 2003 (2003) pp. 47-55. [7] T. Kuflik, ISWRIC – Israeli Software Reuse Industrial Consortium Project presentation and initial lessons learned.

nd

Proceedings of the Israeli 22 IEEE Convention (2002) pp. 210-211. [8] W.Lim, Reuse Economics: A Comparison of Seventeen Models and Directions for Future Research. Proceedings of th the 4 Intentional Conference on. Software Reuse (1996) pp. 41-51. [9] A. Mockus, R.,T. Fielding, and J.D. Herbsleb, Two Case Studies of Open Source Software Development: Apache and Mozilla. ACM Transactions on Software Engineering and Methodology. 1(3) (2002) pp. 309-346. [10] J. S. Poulin, Measuring Software Reuse, AddisonWesley, Reading, MA, (1997). [11] D. C. Rine, Success Factors for Software Reuse That are Applicable Across Domains and Businesses. Proceedings of the 1997 ACM Symposium on Applied Computing (1997) pp. 182-186. [12] Software Technology for Adaptable, Reliable systems, Air Force/STARS Demonstration Project Experience Report, Version 3.1, Volume I, USAF Material Command, Electronics Systems Center, Hanscom AFB (1996). [13] E. Wiles, Economic Models of Software Reuse: A Survey, Comparison and Partial Validation. Technical Report UWA-DCS-99-032, Department of Computer Science, University of Wales, Aberystwyth, U.K (1999). [14] S. Hissam, C.B. Weinstock, D. lakosh and J. Asundi, Perspectives on Open Source Software, technical report CMU/SEI-2001-TR-019 ESC-TR-2001-019 (2001)
