Open-Source Software Reuse Service for SMEs

FP7-SME-2008-2 – 243768 OPEN-SME “Open-Source Software Reuse Service for SMEs” Deliverable D4.2 Synthesis and Evaluation Report Deliverable Type: Nature of the Deliverable: Date: Distribution: Code: Editor: Contributors:

PU* R** July 31st, 2012 WP4 OPEN-SME/EMYPEE/WP4/D42 EMYPEE ALL

* Deliverable Type:

PU= Public, RE= Restricted to a group specified by the Consortium, PP= Restricted to other program participants (including the Commission services), CO= Confidential, only for members of the Consortium (including the Commission services) ** Nature of the Deliverable: P= Prototype, R= Report, S= Specification, T= Tool, O= Other

Abstract: This document presents the results of the user trials performed with the OPEN-SME toolset. The analysis and evaluation included, is based on the collected user feedback and is made against the requirements specified in D2.1.  Copyright by the OPEN-SME Consortium. The OPEN-SME Consortium consists of: Enosi Mihanikon Pliroforikis & Epikinonion Ellados

Project Coordinator

Greece

Drustvo za informacione sisteme I racunarske mreze-Informaciono drustvo Srbije

Partner

Serbia

Epistimoniko Techniko Epimelitirio Kyprou (Technical Chamber of Cyprus)

Partner

Cyprus

Teknikbyn Science Park Vasteras AB

Partner

Sweden

Bitgear Wireless Design Services d.o.o.

Partner

Serbia

GNOMON Informatics SA

Partner

Greece

Maelardalens Hoegskola

Partner

Sweden

Teletel S.A. - Telecommunications and Information Technology

Partner

Greece

Aristotelio Panepistimio Thessalonikis

Partner

Greece

Universiteit Maastricht

Partner

Netherlands

Deliverable 4.2: Synthesis and Evaluation Report

Page 2 of 41

This page has been intentionally left blank.

OPEN-SME/EMYPEE/WP4/D42

© OPEN-SME Consortium – July 2012


Page 3 of 41

Table of Contents ABBREVIATIONS ............................................................................................................................ 7 1. INTRODUCTION ...................................................................................................................... 8 2. BRIEF OVERVIEW OF THE TRIALS .................................................................................. 9 2.1

THE OPEN-SME TOOLSET .................................................................................................... 9

2.1.1 2.1.2

OCEAN ...................................................................................................................................... 9 COPE ....................................................................................................................................... 10 2.1.2.1 Modifying the Domain Ontology ........................................................................................................ 10 2.1.2.2 Uploading a component ..................................................................................................................... 11 2.1.2.3 Synchronizing the Domain Ontology with COMPARE ...................................................................... 12 2.1.3 COMPARE .............................................................................................................................. 12 2.1.3.1 Using COMPARE as a re-user........................................................................................................... 12 2.1.3.2 Using COMPARE as a reuse-engineer .............................................................................................. 13 2.1.4 System Architecture as a whole ............................................................................................... 14

2.2

TRIAL SET-UP ...................................................................................................................... 15

2.2.1

CRM Application ..................................................................................................................... 15 2.2.1.1 Use Cases ........................................................................................................................................... 15 2.2.1.2 Applying The OPEN-SME Application Engineering Process for the CRM Application .................... 17 (P1) Application Requirements phase ................................................................................................................................17 (P2) Physical architecture definition phase .......................................................................................................................19 (P3) Application Design Phase ..........................................................................................................................................19 (P4) Implementation- Component Realization ...................................................................................................................21 (P5) System integration phase ............................................................................................................................................23 (P6) System testing phase ...................................................................................................................................................23 2.2.1.3 Known Limitations ............................................................................................................................. 25 2.2.2 Embedded Systems Verification Application ........................................................................... 26 2.2.2.1 Use Cases ........................................................................................................................................... 26 2.2.2.2 Applying The Open-SME Application Engineering Process for the Embedded Verification Application .......................................................................................................................................................... 27 2.2.2.3 Known Limitations ............................................................................................................................. 27

3. EVALUATION ........................................................................................................................... 28 3.1

USER REQUIREMENTS DEFINED............................................................................................ 28

3.2

EVALUATION METHODOLOGY .............................................................................................. 28

3.2.1 Assessing The Cost Reduction ................................................................................................. 28 3.2.2 Assessing the Qualitative Criteria: User Feedback Form (Questionnaire) ............................ 29 3.3 Collected Results ......................................................................................................................... 31




Page 4 of 41

3.3.1 3.3.2

Overview .................................................................................................................................. 31 CRM Application ..................................................................................................................... 31 3.3.2.1 Quantitative evaluation ...................................................................................................................... 31 3.3.2.2 Qualitative Evaluation ....................................................................................................................... 31 3.3.3 Embedded Systems Verification Application ........................................................................... 32 3.3.3.1 Quantitative evaluation ...................................................................................................................... 32 3.3.3.2 Qualitative Evaluation ....................................................................................................................... 32 3.3.4 Analysis of Results ................................................................................................................... 33 3.3.5 Status of the OPEN-SME toolset and Critical Review ............................................................ 34 3.4 Remarks On Competitive Aspects and Market Environment ....................................................... 35

3.5

REMARKS ON SME BUSINESS ORIENTATION AND STRATEGY.............................................. 36

4. LESSONS LEARNED AND FUTURE STEPS ..................................................................... 39 4.1

LESSONS LEARNED ............................................................................................................... 39

4.1.1 4.1.2

4.2

PROS ....................................................................................................................................... 39 CONS ....................................................................................................................................... 39

FUTURE PLANS FOR DEVELOPMENT AND MAINTENANCE ..................................................... 40

5. REFERENCES ......................................................................................................................... 41




Page 5 of 41

List of Figures Figure 1: The physical Architecture of OCEAN ....................................................................................9 Figure 2: COPE interacting with the Knowledge Manager Server ......................................................11 Figure 3: Uploading a component ........................................................................................................11 Figure 4: COPE interacting with the Compare Server .........................................................................12 Figure 5: Browsing the Compare Server as a re-user ...........................................................................13 Figure 6: COMPARE interacting with the Versioning Server .............................................................13 Figure 7: An overview of the whole physical architecture*..................................................................14 Figure 8: Use Case Diagram of the CRM prototype application ..........................................................15 Figure 9: Overview of the Application Requirements phase ................................................................18 Figure 10: Overview of the Application Design phase ........................................................................20 Figure 11: Overview of the Component Realisation phase ..................................................................21 Figure 12: LCSAJ Coverage for the component as a whole.................................................................22 Figure 13: LCSAJ Coverage for the component accessible through it’s interface ...............................22 Figure 14: Infeasible paths ...................................................................................................................23 Figure 15: Overview of the Integration Phase ......................................................................................23 Figure 16: Overview of the System Testing phase ...............................................................................24 Figure 17: CRM Trial – Qualitative results ..........................................................................................31 Figure 18: Embedded Systems Trial – Qualitative results ...................................................................32




Page 6 of 41

List of Tables Table 1: Use Cases of the CRM prototype application ..............................................................................17 Table 2: Coverage results for reusable components ...................................................................................24 Table 3: Use Cases of the Embedded Systems Verification Application ...................................................27 Table 4: Re-users’ requirements for the evaluation ....................................................................................28 Table 5: Questionnaire for Requirements R2-R5 .......................................................................................30 Table 6: Questionnaires’ replies for the two trials......................................................................................33




Page 7 of 41

ABBREVIATIONS CBD

Component Based Development

CBSE

Component Based Software Engineering

CMMI

Capability Maturity Model Integration

COMPARE

Component Repository and Search Engine

COTS

Commercial Off The Shelf

CPU

Central Processing Unit

EFP

Extra Functional Property

ISO

International Standard Organisation

JSP

Java Server Pages

PI

Provided Interface

ProCom

Progress Component Model

QoS

Quality of Service

RCP

Rich Control Platform

RI

Required Interface

RTOS

Real-time Operating System

RUP

Rational Unified Process

SCM

Software Configuration Management

SME

Small and Medium scale Enterprise

SWEET

Swedish Worst Case Execution Time Tool

V&V

Verification and Validation

WCET

Worst Case Execution Time




Page 8 of 41

1. INTRODUCTION The OPEN-SME main idea is to introduce a reuse service that will be operated by SME Association Groups (AGs) (e.g. Greek Association of Computer Engineers, Vasteras Science Park etc.) on behalf of their SME software development members. This service will be operated by software experts of the SME AGs who will produce components from OSS projects, test them, generate documentation, resolve licensing etc. asynchronously to application development by SMEs and independently from the SMEs. The components will be related to domains that are relevant to the SMEs. Therefore when the SMEs will want to reuse them, the components will already be there. The OPEN-SME project collectively provides two processes and three tools, namely: 1. The Reuse-Oriented Domain Engineering (RODE) process. 2. The Application Engineering process. 3. The OCEAN tool. OCEAN is a tool for searching OSS code search engines. Essentially a metasearch engine. 4. The COPE tool. COPE is a tool for extracting, testing, documenting and packaging software components originating from OSS projects. 5. The COMPARE tool. COMPARE is a repository for storing the extracted component packages and delivering them to SMEs. As originally planned in order to demonstrate and assess the actual capabilities of the toolset and the methodologies a set user trials was planned by the participating SME Associations. The trials were set up according to the details described in Deliverables D41a and D41b. In addition, during this phase the project has arranged the collection of user feedback and has assessed the results against the specified user requirements. The input was based on a selected team of 3-5 software development SME members, per SME-AG, who participated in the trials and evaluated the effectiveness of the offered Open-Source Software Reuse services. In the following sections we synthesise the results of the project, the services offered through the trials and we provide a documented evaluation of the final project deliveries. The evaluation includes both a qualitative and a quantitative part, concluding in a rather positive view by the participating re-users. In fact considering the quantitative and qualitative evaluation described earlier we can conclude that overall the OPEN-SME toolset is effective in addressing the reuse needs for domain-specific open source software components. However there are still areas that can be improved, which are also brought forward.




Page 9 of 41

2. BRIEF OVERVIEW OF THE TRIALS 2.1 THE OPEN-SME TOOLSET In OPEN-SME, several tools are instantiated for an SME-AG to provide the services to its belonging SMEs. In this section the physical architecture of each one of these tools is presented as appeared during the OPEN-SME trials. We present the physical architecture of the Open-Source Search Engine (OCEAN), the Component Adaptation Environment (COPE) and the Component Repository (COMPARE) respectively and we analyse each tool and each scenario of use separately. Finally, in the last part of this section a consolidated architecture of these tools is outlined capturing the flow of information and the interaction of the implicating roles (reuse engineer, re-user) with those tools, thus capturing the “big picture”.

2.1.1

OCEAN

The Open-Source Search Engine (OCEAN) is a meta-search engine that provides unified access to existing Open Source Software (OSS) search engines. This allows the reuse-engineer to find open source software assets (i.e projects, packages, files etc.) satisfying certain criteria, such as software that is written in a specific programming language, containing certain keywords, having a specific license etc. Moreover, it allows the re-user to detect a software asset that is of some value and place an order to adapt that specific asset to the reuse-engineer. OCEAN is a web portal (http://ocean.gnomon.com.gr/web/guest/home , root/test) that allows mainly locating and browsing open-source files and projects that are available on popular open-source search engines. OCEAN is extensible to incorporate any open-source search engine available regardless of the integration strategy. What this means is that the integration of an arbitrary search engine can be performed in any way possible (i.e. use of provided api, web-scrapping, etc).

Figure 1: The physical Architecture of OCEAN OPEN-SME/EMYPEE/WP4/D42



Page 10 of 41

In Figure 1, the flow of information for a typical use of OCEAN is depicted, as it is instantiated for the purposes of the OPEN-SME trials. In this context the reuse-engineer will access the Ocean server, using a browser1, which then will act as a mediator for different servers that host OSS projects or provide a search engine for a fraction of the OSS projects which are publicly available through the internet. More specifically, the reuse-engineer when performing a search query for a software asset, this query is passed to the Ocean Server (which is located at ocean.gnomon.com.gr). Then the Ocean server is responsible to perform a filtered query (usually by an http request or an API call) per selected engine, which is contacting the appropriate server, perform the call and then gather the filtered results. After completion, the Ocean server aggregates the results based on their relevance and presents those to reuseengineer. Finally, the reuse-engineer can review the software assets filtered according to his criteria which may include specific search engines, programming language, software license, etc. In this manner, the reuseengineer can focus on exploring specific OSS projects which meet those criteria, rather than being lost in the ocean of publicly available OSS projects

2.1.2

COPE

The Component Adaptation Environment (COPE) is a tool-chain that provides an environment for the enactment of the domain engineering process of OPEN-SME, thus allowing the reuse-engineer produce reusable components for the domain(s) of interest (http://opensme.eu/deliverables/86-deliverable-d32b , trialsuser/opensmeuser). COPE is a desktop application to perform the following tasks in order to achieve the aforementioned result: 

Identify and model primary concepts of the domain (using: Knowledge Manager)



Analyse the different aspects (using: Static Analysis, Design-pattern Analysis, etc.) of an OpenSource project



Comprehend the project (using: the outcome of the Analysis, Documentation Generation, inproject Search)



Detect candidate components (using: the outcomes of the project Analysis (ii) and project Comprehension (iii) )



Generate components (using: the various Component Makers)



Validate them (using: Dynamic Analysis)



Classify the produced component (using: Knowledge Manager)



Upload the Component to COMPARE component repository and search engine.

As far the physical architecture is concerned, only a couple of the aforementioned tasks (specifically i., vii. & viii.) initiate an interaction with one of the OPEN-SME servers. In the following subsection we describe each scenario of use and the associated servers as they are instantiated for the OPEN-SME trials.

2.1.2.1

MODIFYING THE DOMAIN ONTOLOGY

The interaction between COPE and the Knowledge Manager Server is captured Figure 2, presenting the tasks performed by the reuse-engineer when modifying either by adding, deleting or altering the representation of the domain.

1http://ocean.gnomon.com.gr/




Page 11 of 41

Figure 2: COPE interacting with the Knowledge Manager Server In this instantiation of the OPEN-SME tools (i.e. the specific instantiation for the trials), each program instance of the COPE tool accesses the Knowledge Manager Server using its API as described in “D2.2 :Functional/Technical specification of domain engineering processes and tools”. This API is published as a web service and can be accessed using specific libraries through the following connection information: 

Repository URL: http://swserv2.csd.auth.gr:8084/openrdf-sesame



Repository Name: opensme

When the reuse-engineer performs an activity that alters the domain ontology, (i.e. adds a new domain, maps a concept to a domain, renames a domain, deletes a concept, imports a new ontology etc.) the COPE tool passes these changes using the Knowledge Manager API to the server. In this way, multiple instances of COPE can exist, altering the Domain Ontology in a distributed (but not concurrent) way, providing the flexibility of using multiple program instances of COPE at different locations and thus multiple users.

2.1.2.2

UPLOADING A COMPONENT

Another common server connection is when the reuse-engineer has completed the component extraction and adaptation process, has produced all the accompanying information (documentation, reports, etc.) and wants to upload them to the versioning server. This task is shown in Figure 3, showing the triggering event and capturing the communication between different tools and the versioning server.

Figure 3: Uploading a component The versioning server, provides the capability of providing different versions of the component as it changes and evolves. This means that the reuse-engineer can constantly modify or evolve the different elements (source code, documentation, execution scenarios) of the component without losing access to previous versions of the component's elements. Moreover, in case of a specific asset order from the reuser, the versioning server provides branching capabilities that allows the reuse-engineer to create more concrete and tailor-made components according to the re-user's specification. The versioning server can be accessed using either an Integrated Development Environment (IDE) tool, an Subversion Client (SVN Client) tool or even through the terminal provided that the required tools and OPEN-SME/EMYPEE/WP4/D42



Page 12 of 41

libraries are already installed. The server can be accessed through the following connection information: 

Http URL2: http://blade03.teilar.gr/opensme



SVN URL: svn://blade03.teilar.gr/opensme

(Note: In order to get a username & password to access the repository you should contact the OPENSME webmaster) The versioning server can also be browsed using a Web browser at the aforementioned URL and using the provided credentials.

2.1.2.3

SYNCHRONIZING THE DOMAIN ONTOLOGY WITH COMPARE

In Figure 4, the interaction between COPE and the Compare Server is depicted showing the creation and publication of a component to the Compare Server.

Figure 4: COPE interacting with the Compare Server By performing synchronization with the Compare server, the COPE tool forwards the changes performed since the last synchronization. This means that based on reuse-engineer's actions concerning the Domain Ontology, and specifically the classification of the components, the Compare server is notified for the changes and updates the internal representation of the component repository. Information regarding access to the Compare server can be found in the following Section.

2.1.3

COMPARE

The Component Repository and Search Engine (COMPARE) is a web portal (http://www.teletelprojects.net/compare , demo/1234), that allows SME software re-users to search and discover software artefacts, technical documents, test suites related to open-source software. In addition, it allows the stakeholders of the Domain Engineering Process (reuse engineers, domain experts, component testers and certifiers) to manage and maintain the assets stored in the repository. The end-users can be endowed by using the advanced filtering capabilities as well as by accessing information about the verification and certification attributes of a component. Finally, it provides a communication mechanism between the re-users and the reuse-engineers

2.1.3.1

USING COMPARE AS A RE-USER

In Figure 5 and Figure 6, typical uses of COMPARE are presented depicting the flow of information as it is instantiated for the purposes of the OPEN-SME trials. In this context the re-user will access the Compare server using a browser3 in order to search for a component and the download it.

2

provides only repository browsing 3http://teletel-projects.net/compare/




Page 13 of 41

Figure 5: Browsing the Compare Server as a re-user Typically a re-user (as shown in Figure 5), when s/he has decided which parts of his application can be provided perhaps by reusing ready-made components, accesses the Compare server and searches for components that meet his/hers requirements. The re-user can then examine the components' information (e.g. rating, number of downloads, programming language, license, etc.) and then decide which one fits best his/hers needs.

Figure 6: COMPARE interacting with the Versioning Server Having detected the desired component, the re-user places a download request (as shown in Figure 6). This request is processed by the server according to the re-user permissions; if the re-user is allowed to retrieve the component then the Compare server delivers it, otherwise the administrator of Compare evaluates the request and modifies the re-user's permissions according to his/her decision. Finally, the re-user can view the components wiki, place a comment and rate the component as well as communicate with the administrator of the system in order to request/recommend specific changes, provide feedback etc.

2.1.3.2

USING COMPARE AS A REUSE-ENGINEER

Apart from the actions described in the previous section, that the reuse-engineer can also perform, the reuse-engineer is able to manage the components, provide general instructions that are not included in the component “per se”, provide feedback to the re-users, view pending requests, etc. OPEN-SME/EMYPEE/WP4/D42



Page 14 of 41

The reuse-engineer can access compare through the same interface as the re-user, i.e. by browsing to the Compare URL4 using a web-browser. Using this web-interface, the reuse-engineer can manage the uploaded components, i.e. provide specific privileges to the re-users and other reuse engineers and add additional documentation or any other file associated using the file repository. Furthermore, s/he can discuss with re-users using the component's wiki about future extensions, bugs or provide feedback regarding how to use that specific component. Also, using the SVN Browse s/he can review the current as well as older versions of the component which allows him to provide more concrete information to the re-users.

2.1.4

SYSTEM ARCHITECTURE AS A WHOLE

In Figure 7, a consolidated view of the system architecture is provided, depicting all the OPESME servers and the roles for both the SME-AGs and the SMEs.

Figure 7: An overview of the whole physical architecture*. The reason this decentralized topology was selected is that the end-users perceive it as a robust, faulttolerant system. So, if one of the servers malfunctions, the rest of the functionalities provided by the system do not cease to exist, but on the contrary the associated users can still perform their tasks without being affected by a malfunction that is irrelevant with what they have to perform. Moreover, this architecture makes the evolution of the services independent from each other which is both desirable and necessary. It is desirable, not only for purposes of robustness and fault-tolerance but also for tracking and maintaining reasons. It is also necessary because at any moment during the trials the end-users may require additions or enhancements in order to successfully use the services, so the services should be easily maintainable thus independent from each other. Nevertheless these services can be hosted on a single physical server and thus do not impose additional costs to the SME-AGs. 4http://teletel-projects.net/compare/ *Some return messages are omitted.




Page 15 of 41

2.2 TRIAL SET-UP 2.2.1

CRM APPLICATION

The trial concerns the creation of a CRM system prototype reusing the components produced by the reuse engineers who applied the Reuse Oriented Domain Engineering (RODE) process using the COPE tool. As we mentioned the components belong to the Customer Relationships Management domain and originated from OSS projects that were analysed with COPE. These components were then described using the Knowledge Management feature of COPE and were uploaded to the COMPARE repository. In the following we will provide a description of the use cases of the CRM system which is intended as a prototype for the demonstration of the effectiveness of the OPEN-SME approach.

2.2.1.1

USE CASES

The Use Case diagram is given in Figure 8 and the use cases are described in some more detail in the following Table 1.

Figure 8: Use Case Diagram of the CRM prototype application OPEN-SME/EMYPEE/WP4/D42



Page 16 of 41

Use Case

Actors

Description

1

User Log In/Out

Administrator, Employee, Customer

User types user name and password to log in to the system/Users logs out from the system.

2

Register Employee

Administrator

Administrator registers a new Employee.

3

Display Employees List

Administrator

Administrator accesses the employees list.

4

Display/Edit Employee Data

Administrator

Administrator reads and updates the employee data.

5

Delete Employee

Administrator

Remove employee from the system.

6

Register Customer

Customer

Employee registers a new Customer.

7

Display Customers List

Employee

Employee accesses to the customers list.

8

Display/Edit Customer Data

Employee

Employee reads and updates the customer data.

9

Delete Customer

Employee

Remove a customer from the system.

10 Register Product

Employee

Register a new product.

11 Display Products List

Employee, Customer

Display a list with all the products.

12 Delete Product

Employee

Remove a product from the system.

13 Display Orders List

Employee

Display a list with all the orders.

14 Display Order Details

Employee, Customer

Display details about a specific order.

15 Issue an Order

Employee

Issue an Invoice for an order.

16 Add/Remove Product to/from the Customer shopping cart

Add or Remove a product from the shopping cart.

17 Display shopping cart details

Customer

Display a list with the products stored in the shopping cart.

18 Check Out

Customer

Review shopping cart products and proceed to the payment.

19 Payment

Customer

Select a stored credit card to pay for the order.




Page 17 of 41

20 Display Stored Credit Cards

Customer

Display a list with the stored credit cards.

21 Register new Credit Card

Customer

Register a new credit card to the system.

22 Display Notifications List

Employee, Customer

Display a list with notifications

23 Display Notification Details

Employee, Customer

Display notification Details.

Table 1: Use Cases of the CRM prototype application 2.2.1.2 APPLYING THE OPEN-SME APPLICATION ENGINEERING PROCESS FOR THE CRM APPLICATION We applied the Application Engineering Process (AEP) of the OPEN-SME project in building a Customer Relationship Management (CRM) system prototype. The purpose of the prototype was to testdrive the AEP and the COMPARE repository. The OPEN-SME AEP has the following phases: P1. Application requirements phase P2. Physical architecture definition phase P3. Application Design Phase P4. Implementation- Component Realization P5. System integration phase P6. System testing phase P7. Release Phase P8. Maintenance Phase In the following we will review the process of building the CRM prototype in a series of steps in which we have applied the OPEN-SME phases (P1-P8).

(P1) Application Requirements phase As can be seen in Figure 9, one of the main tasks of the AEP is that requirements are collected and then possible reusable components are identified that could be reused for the fulfilment of the requirements.




Page 18 of 41

Requirement Collection Requirements Reuse Analysis and Adaptation Requirement Analysis

Modify Requirement

Identification of Candidate Components

[Component not found]

[Component found]

[Requirement could be changed]

Requirement cannot be changed Mark requirement as "non-reusable"

[Remaining requirements] [No more requirements]

Figure 9: Overview of the Application Requirements phase Consequently, requirements specification is not only input to further development but also input for the consideration of the availability of existing components. In addition in cases where there is limited experience in the application domain, existing OSS components, from the COMPARE reuse repository, can be used to further elaborate the requirements. In our case, requirements were appropriately formed in order to reuse most existing components as they are or with small changes which are analysed in detail in the following sections. Initially, requirements were collected and specified without taking into account the available software components in COMPARE. This is the ‘Requirements Analysis’ activity in Figure 9. The requirements were grouped under the following main categories: 

Functional Requirements: Customer - employee – administrator registration, deletion, editing, logging in, logging out, or product – order – note registration, deletion, editing, etc.



Platform requirements: The Java programming language was chosen providing support for Windows, Linux, MacOS platforms. MySQL database was chosen to be used to store the CRM application information using JDBC.



External Quality Requirements: Reliability, Availability, and Safety Requirements, like user password and credit card details encryption.



Internal Quality Requirements: Code Readability, Complexity, Maintainability Requirements. A simple three layer architecture was selected “GUI – Controllers – Entities” allowing functional code only in the Controllers layer, optimizing readability, maintainability and simplifying the project.

Subsequently, followed the identification of candidate components based on the initial requirements specification (‘Identification of Candidate Components’ activity in Figure 9). COMPARE reuse repository was used to look for potential candidates of components satisfying the requirements produced during the previous step. Ten components were selected from three different projects. The selected components and their parent projects are listed below:  CentraView Project o User OPEN-SME/EMYPEE/WP4/D42







Page 19 of 41

o Contact o Note o Order o Payment o Invoice Ebineutrino Project o Campaign_Entity o Product_Entity o Base64_Encrypter Plazma Project o TimeSheet

For the selected components first we confirmed that they satisfy the functional requirements and then requirements of the other categories were also validated. Having selected the components the next step was to modify requirements that could not be easily satisfied (‘Modify Requirements’ activity in Figure 9). Considering both requirements and selected components from the previous steps, we decided to modify a platform requirement stating that the database would be accessed from Java using JDBC. Ebineutrino's components came with ready to use Hibernate XML mapping files, simplifying the database access considerably. As a result the requirement considering the use of JDBC was changed to the use of the Hibernate persistence framework. This requirement modification positively affected another requirement considering the complexity and also the maintainability of the system. As was expected not all requirements could be satisfied by the available components. More specifically credit card information storing and user handling (logging in, logging out) requirements were not fully met by any component. The luck of suitable components led us to the development the missing functionality, satisfying requirements specification.

(P2) Physical architecture definition phase The objective of this phase is to provide a model-level description of the relevant hardware of the system and it is considered more relevant for embedded systems. In the case of the CRM the hardware of the system is mostly standardized and so this phase is not considered further.

(P3) Application Design Phase Having satisfied the requirements specifications we could proceed to the design of the application. The main activities of this phase are depicted in Figure 10.




Page 20 of 41

Conceptual Design

Architectural Deployment Architecure level Analysis

[No]

[OK]

Detailed Design

Identification of Candidate Components

[Component Create New not found] Component [Component found] [Yes]

Deatiled Analysis [Not OK]

[OK]

[No]

[Not Feasible] [Feasible Design]

Figure 10: Overview of the Application Design phase Starting with the system analysis and a conceptual design based on the selected components, a system overall architecture was provided which assisted us to the design of a more detailed system architecture. In the ‘Conceptual Design’ activity (see Figure 10) the overall software architecture is identified. Subsystems like “Shopping” or “Order-handling” etc., comprising different functional units, were identified and their functional requirements led to the identification of other required subsystems. For example “Shopping” requires the existence of a “Customer” user, the listing of the available products, the existence of shopping cart and a “check out” procedure which leads to identification of other subsystems such as the “Payment” subsystem. Architectural deployment which followed the system analysis formed the high level architectural decisions such as which functional units run on which nodes/platform etc. Aspects, which have wider system level implications, were addressed and analysed in the architectural level analysis. A representative aspect was the security aspect, with issues such as the type of the encryption that was used for the passwords and the credit cards details. Then followed the ‘Detailed Design’ activity. This activity first involves the design of subsystems and detailed breakdown to functional units built from available software components but also adapted software components and newly developed software components. This activity also includes the specification of behaviours, sequence diagrams and state diagrams describing the collaborations among the components and the behaviour of individual components respectively. Based on the system requirements most of the model level analysis with respect to extra functional properties was performed at the stage of detailed analysis, to analyse the design soundness. For example, according to the architectural level analysis about the entity access availability and encryption type, the normal form of the database was decided to be the third and the type of the encryption the “base64 encryption” (based on the available component).




Page 21 of 41

(P4) Implementation‐ Component Realization

Component Selection

[Not Found] [Found]

Component

Search somewhere else

Repository

[Adaptation needed] [Adaptation not needed]

[Found] [Not Found] In-hous e development

Adaptation

Verification

Figure 11: Overview of the Component Realisation phase In the first activity of ‘Component Selection’ we have tested the already selected components against the detailed design outcome and found that they were appropriate for the system under construction. It was however necessary to perform some component adaptations as well as to develop new software components Component adaptation: As mentioned in previous Sections, Hibernate persistence framework was used to handle database access. Ebineutrino's components came with ready to use Hibernate XML mapping files however components originating from other projects like Centraview were not Hibernate-ready. We decided to use Hibernate annotations due to the simplicity of this approach, so we had to adapt every component to this requirement. Ebinuetrino's components XML mapping files simplified this process as they already included complex one to many, many to many, etc. relationships. Obviously, our database schema differs from Ebinuetrino's schema so we had to make some modifications to the existing information from the XML mapping files. Due to application specific data, some fields of both Ebinuetrino's and Centraview's components were marked as “Transient”, meaning that we wanted to avoid their persistence in the database. As we had to combine components from different projects we had to add some extra fields and relationship annotations in some cases. “Payment” and “Invoice” components adaptation is a noteworthy example. Both “Payment” and “Invoice” components came from Centraview however since our database schema is different from Centraview's we had to add an Invoice field in the payment class with one-to-one relationship. In addition, as Centraview uses its own way to handle and store products we had to adapt Invoice class to store products according to our system design, as products are handled by an Ebinuetrino's component “Product_Entity”. In-house development: Based on system requirements the system had to handle and store credit card information. As COMPARE repository has no component satisfying this requirement we had to develop the missing component. Credit Card component is an Entity component with information about credit cards like card number, card holder, expiration date, etc. As with other entities the required hibernate annotations had also to be developed.




Page 22 of 41

Component verification: The adaptation process made no major changes to any component, so we can consider that selected from COMPARE components where already verified as units using the Testing and Validation features of COPE. As an example we present the “User” component's verification results coming from the HTML report included in the component package. This HTML report is generated automatically by COPE using testing and validation results.

Figure 12: LCSAJ Coverage for the component as a whole

Figure 13: LCSAJ Coverage for the component accessible through its interface Figure 12 Illustrates the coverage metrics of the “User” component as a whole whereas Figure 13 the coverage metrics from the Interface of the component that we have used. As we can observe the coverage metrics of Figure 12 are very low, however this should not concern us as we only use the functionality provided by the interface which has 90% LCSAJ coverage and 100% statement coverage, as depicted in Figure 13. A comment in the HTML test report states that the LCSAJ coverage of the “User” interface is 90% due to two infeasible paths, so the real LCSAJ coverage is 100%. Those two infeasible paths are presented below.




Page 23 of 41

Figure 14: Infeasible paths As we observe in Figure 14 these two paths are really infeasible since, in both cases, the flow cannot start from line 4 and jump to line 7 without first visiting line 2.

(P5) System integration phase

Component Instance definition

Allocation of component instances

Component deployment

Component binding

Figure 15: Overview of the Integration Phase The developed CRM system uses Apache Server and MySQL database. For the database handling Hibernate persistence framework is used and more specifically the annotation interface of the framework. Every component was adapted to the Hibernate's annotation interface requirements.

(P6) System testing phase




Page 24 of 41

Test case Generation

Test case Execution

Test result Analysis [More tests remaining] [Bugs found]

Debug and Identify faulty component Fixing the Component

[No bugs found]

Regression Testing [No more test cases]

Figure 16: Overview of the System Testing phase Every component from COMPARE repository came with its own HTML test documentation automatically generated by the COPE tool based on testing and validation results. This information help us reduce the time needed to examine our system components us we focus our testing only in new functionality added during the adaptation and in-house development processes. Also testing the integrated system is done with traditional means and is not easier or harder due to the existence of testing results for individual components. Table 2 contains the coverage metrics of some components we have reused. These results come from the HTML test report of every component contained in the component package in the COMPARE repository.

Component

Statement Coverage (%)

LCSAJ Coverage (%)

# of LCSAJ infeasible paths

User

100

90

2

Contact

100

92

3

Note

100

95

1

Payment

100

100

-

Order

100

100

-

Base64

44

8

20

Invoice

100

100

-

Product

100

100

-

Table 2: Coverage results for reusable components OPEN-SME/EMYPEE/WP4/D42



Page 25 of 41

The number of infeasible paths is important because this is the reason the LCSAJ coverage is not 100% in some of the above components. “Contact” component had additionally two uncovered feasible paths which we have explicitly examined. As we observe, “Base64” component has many infeasible paths. This is caused by a weakness of the LCSAJ algorithm which is unable to handle properly exception handling blocks. Additionally “Base64” has 4 uncovered feasible paths that we have examined. “User” and “Note” components have actual LCSAJ value 100% after excluding infeasible paths. Note: The remaining phases (namely “(P7) Release Phase” and “(P8) Maintenance Phase”) are not covered in this report because they were not completed in the context of this work. We will leave the study of these two phases as future work.

2.2.1.3

KNOWN LIMITATIONS

Some of the limitations of the case study follow. It is worth noticing that the CRM application described was intended as a prototype for the demonstration of the OPEN-SME approach to the reusability of domain-specific components. In fact there was an attempt to reuse as many as possible and start an integration effort towards creating a component framework. Therefore the application itself has many limitations as a CRM application, such as: 

There are no constraints for the user's password and user name.



There is no check to the user's input.



Email address used as a user name is not checked for its validity.



Currently there is no way to create an administrator account. The only one administrator account is created on the system start-up.



Credit card number validation is not performed.




2.2.2

Page 26 of 41

EMBEDDED SYSTEMS VERIFICATION APPLICATION

The trial concerns the creation of an Embedded Systems Verification prototype application using the software components that were produced by the reuse engineers using the COPE tool. As mentioned, the components relate to the Embedded Systems Verification domain and originate from various open source projects that were analysed with COPE and were uploaded to the COMPARE repository.

2.2.2.1

USE CASES

The following table presents the use cases of the Embedded Systems Verification Application. Use Case

Actor

Description

1

Connection to embedded system via USB

Verification Engineer

The application identifies and connects to an embedded system via USB.

2

Connection to embedded system via RS232


The application identifies and connects to an embedded system via RS232

3

Communication via USB


The application transmits and receives data from an embedded system via USB.

4

Communication via USB in isochronous Verification Engineer mode

The application communicates with the embedded system via USB in isochronous mode.

5

Communication via USB using the HID Verification Engineer profile.

The application communicates via USB and treats the embedded system as a HID.

6

Communication via RS323


The application transmits and receives data from an embedded system via RS232.

7

Display transmitted/received data


The application displays the transmitted and received data.

8

Chart generation using incoming data.


The application generates a chart diagram using the incoming data.

9

Metadata generation.


The application timestamps the transmitted/received data and adds




Page 27 of 41

information regarding the used device (name, address, port). 10

Log file saving


The application saves the received and transmitted data to a log file along with the generated metadata.

11

Data searching


The engineer searches for stored data using metadata information.

12

Log file loading


The engineer selects a log file to load.

13

Chart generation using data from the log Verification Engineer files.

The application generates a chart using data from a log file.

Table 3: Use Cases of the Embedded Systems Verification Application 2.2.2.2 APPLYING THE OPEN-SME APPLICATION ENGINEERING PROCESS EMBEDDED VERIFICATION APPLICATION

FOR THE

The Application Engineering Process followed a similar approach as in the CRM application discussed in detail in Section 2.2.1.2. For reasons of brevity we will not present the details of the AEP for the Embedded Verification Application in detail.

2.2.2.3

KNOWN LIMITATIONS

The Embedded Systems Verification Application is a software prototype and attempts to cover a wide range of common functionalities present the tools of a very hardware specific domain such as the Embedded Systems Verification domain. Therefore, due to the application’s prototype and generic status, it features the following limitations: 

Although the interfaces are present, the underlying communication protocol is specific to the embedded system under verification.



Data transmission and retrieval visualization is performed in byte form, with engines depending on the underlying embedded system.



USB communication with the HID profile is limited to the application. A USB HID driver for the underlying OS needs to be written in order for the embedded system to function as a real HID device.



The chart generation engine only generates line charts.




3.

Page 28 of 41

EVALUATION

3.1 USER REQUIREMENTS DEFINED In D2.1 “State-of-the-art and requirements analysis” we have conducted a survey to identify the needs of the SMEs re-users. The most important findings from this survey concerning the requirements from the re-users’ points of view are the following: 1. The users expect from a tool-assisted reuse-oriented application engineering process to lower development costs and decrease development and testing time: 2. The users consider that the most important artefacts to reuse are source code and then design, documentation and test suites 3. The users consider that the most important factors preventing reuse are the lack of documentation for reusable components, the quality uncertainty for these components and also the difficulty in searching and retrieving the reusable components in the first place. Consequently we formed the following user requirements as our goals for the evaluation activities. Requirement #

Requirement Description

R1

Lower Development Costs.

R2

The OPEN-SME process provides access to the most reusable artefacts required by the re-users

R3

The OPEN-SME components were adequately described from the documentation provided with each component

R4

The OPEN-SME components were tested adequately and the ‘quality uncertainty’ factor preventing reuse was less of a problem

R5

The COMPARE repository was effective in enabling re-users to find appropriate components for their application.

Table 4: Re-users’ requirements for the evaluation

3.2 EVALUATION METHODOLOGY In order to evaluate the requirements in Table 4 we present in this section the evaluation methodology used. In Table 4 requirement R1 is quantitative and requirements R2-R5 are qualitative. In order therefore to carry out the evaluation we have devised an approach that users both measurements and questionnaires. In Section 3.2.1 we will describe the approach for measuring the quantitative requirement and in the following Section 3.2.2 we will

3.2.1

ASSESSING THE COST REDUCTION

The cost reduction is based on the approach described by Poulin in [1] and also in the basic reuse calculator web page by the same author [2]. In order to assess the cost reduction the so called Reuse Cost Avoidance (RCA) in [1] the following formula is used:

RCA  DCA  SCA

Equation 1: Reuse Cost Avoidance In Equation 1, DCA stands for Development Cost Avoidance and SCA stands for Service Cost Avoidance. Therefore the Reuse Cost Avoidance is calculated as the cost avoided from new development due to the reuse of components (DCA) and the cost avoided from maintaining these reusable components OPEN-SME/EMYPEE/WP4/D42



Page 29 of 41

(SCA). More specifically in Equation 2 we can see the formula for DCA and in Equation 3 the formula for calculating SCA.

DCA  (1  0.2)  RSI  NEW _ CODE _ COST Equation 2: Development Cost Avoidance RSI stands for Reused Source Instructions and is basically the Lines of Code that we reused and did not develop anew. The NEW_CODE_COST is the cost of a line of code as is calculated from the company historical data. The rational in Equation 2 is that the cost that we avoid from reusing code is roughly equal 80% of the cost that we would have to invest to develop it. A 20% cost remains for adaptations, code comprehension etc. RSI is a measurable quantity and therefore does not differ for different organizations. NEW_CODE_COST however depends on the country and other subjective factors such as the performance of the programmers of an organization. In our study we have used an average cost of 25 euros per line of code. This average comes from a number of studies referenced in [3] that report costs per line of code in the range of $18-45. However we note that an organization applying the OPEN-SME process can also use a more accurate cost per line estimation based on the organization’s specific historical data if available. SCA (Service Cost Avoidance) assumes that the organization does not have to maintain the reusable components that it reuses and consequently also avoids the cost of fixing them. SCA therefore is the cost an organization avoided during the maintenance phase because of the reuse of components. SCA is calculated by multiplying your organization's historical error rate by the historical cost to fix those errors and by the amount of reused code (RSI) [1, 2], as shown in Equation 3:

SCA 

RSI  ErrorRate  CostPerError 1,000

Equation 3: Service Cost Avoidance In our calculations we have used an Error Rate of 0.5 per KLOC (thousands lines of code) and 8,000 euros Cost per Error. This is the cost of fixing an error after the software product has been released (during the maintenance phase of the project’s lifecycle) and again it should be based on data from contracts and other financial documents of an organization. The figures that we used for the error rate and cost per error are proposed averages in [2].

3.2.2 ASSESSING (QUESTIONNAIRE)

THE

QUALITATIVE CRITERIA: USER FEEDBACK FORM

In order to access the qualitative criteria (R2-R5) we have created a small questionnaire of Likert-type questions for each requirement. The questionnaire was filled by the re-users that participated in each trial. The questions for each requirement along with the possible answers are presented in Table 5.

Req. #

Question

Answer

R2

Would you agree that the components that you reused from COMPARE Strongly contained all the artefacts that are Agree necessary to reuse them?

Agree

Neutral

Disagree

Strongly Disagree

R3

Would you agree that the OPEN-SME components were adequately described Strongly from their documentation provided with Agree each component?

Agree

Neutral

Disagree

Strongly Disagree

R4

Would you agree that the OPEN-SME Strongly

Agree

Neutral

Disagree

Strongly




components were tested adequately and the ‘quality uncertainty’ factor preventing reuse was less of a problem?

R5

Page 30 of 41

Agree

Would you agree that the COMPARE repository was effective in enabling re- Strongly users to find appropriate components for Agree their application?

Disagree

Agree

Neutral

Disagree

Strongly Disagree

Table 5: Questionnaire for Requirements R2-R5




3.3

COLLECTED RESULTS

3.3.1

OVERVIEW

Page 31 of 41

In the following sections we will provide the results for the quantitative and qualitative assessment for the two trials.

3.3.2

CRM APPLICATION

3.3.2.1

QUANTITATIVE EVALUATION

For the quantitative assessment of the CRM application we will use the Equations presented in Section 3.2.1. In the case of the CRM application the lines of code that were developed anew (without measuring the lines of code generated by tools) are 4,778 LOC. The lines that were reused (i.e. the LOC of the reusable components) were 3,041. As we mentioned already we have used an average cost per new line of code equal to 25 euros, an average Error Rate of 0.5 errors per thousand lines of code (KLOC) and an average 8,000 euros Cost per Error. By putting these figures in the equations presented in Section 3.2.1, we have the following: The Development Cost Avoidance (DCA) is 60,820 euros:

DCA  (1  0.2)  RSI  NEW _ CODE _ COST  (1  0.2)  3,041 25  60,820 The Service Cost Avoidance (SCA) is 12,164 euros.

SCA  RSI  ErrorRate  CostPerError  3.041 0.5  8,000  12,164 And consequently the Reuse Cost Avoidance (RCA) is equal to approximately 73K euros:

RCA  DCA  SCA  60,820  12,164  72,984 3.3.2.2

QUALITATIVE EVALUATION

There were 7 reuse engineers involved in the CRM trials, including SME and SME-AGs participation. The collected results are shown in the following diagram.

Figure 17: CRM Trial – Qualitative results




Page 32 of 41

3.3.3

EMBEDDED SYSTEMS VERIFICATION APPLICATION

3.3.3.1

QUANTITATIVE EVALUATION

For the quantitative assessment of the Embedded Systems application we will use the Equations presented in Section 3.2.1. In the case of the Embedded Systems application the lines of code that were developed anew (without measuring the lines of code generated by tools) are 2,881 LOC. The lines that were reused (i.e. the LOC of the reusable components) were 1,309. As we mentioned already we have used an average cost per new line of code equal to 25 euros, an average Error Rate of 0.5 errors per thousand lines of code (KLOC) and an average 8,000 euros Cost per Error. By putting these figures in the equations presented in Section 3.2.1, we have the following: The Development Cost Avoidance (DCA) is 26,180 euros:

DCA  (1  0.2)  RSI  NEW _ CODE _ COST  (1  0.2)  1,309  25  26,180 The Service Cost Avoidance (SCA) is 5,236 euros.

SCA  RSI  ErrorRate  CostPerError  1,309  0.5  8,000  5,236 And consequently the Reuse Cost Avoidance (RCA) is equal to approximately 31K euros:

RCA  DCA  SCA  26,180  5,236  31,416 3.3.3.2

QUALITATIVE EVALUATION

There were 11 reuse engineers involved in the Embedded Systems trials, including SME and SME-AGs participation. The collected results are shown in the following diagram.

Figure 18: Embedded Systems Trial – Qualitative results




3.3.4

Page 33 of 41

ANALYSIS OF RESULTS

In both trials we can see noticeable cost avoidance due to reuse. Reuse Cost Avoidance for the CRM application is €72,984 and for the embedded systems application is €31,416. The reason for this difference is that the reused lines of code in the first case are more in absolute numbers compared to the embedded application (3,041 vs. 1,309). The ratio of the reused code to the new code for the CRM application was almost 64% whereas for the embedded application was 45%. This reuse ratio is impressive since we are not counting in these figures generated lines of code by the IDEs (e.g. user interfaces classes generated by the IDE) that is another form of reuse, but we are only counting domain specific components. Arguably these figures would be more accurate if we had carried out larger-scale trials or more trials that have not been possible to the time limitations of the current project. However we are planning follow-up studies in the future. Given that the OPEN-SME main objective was the reuse of domain-specific components these first results are very encouraging. Considering the qualitative results that aimed at indicating the extent to which the OPEN-SME project was successful at addressing the requirements that were set forward in D2.1 we have the following answers collectively and per trial depicted in Table 6:

R2 CRM EM. TOT. CRM

R3 EM. TOT.

CRM

R4 R5 EM. TOT. CRM EM. TOT.

Strongly Agree

1

2

3

2

4

6

0

0

0

2

3

5

Agree

3

5

8

4

4

8

7

10

17

5

6

11

Neutral

2

2

4

1

2

3

0

0

0

0

2

2

Disagree Strongly Disagree

1

2

3

0

1

1

0

1

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

Table 6: Questionnaires’ replies for the two trials The conclusions that we can draw from the provided answers per requirement are the following: 

(R2) The OPEN-SME process provides access to the most reusable artefacts required by the reusers: As can be seen the answers to this question were mostly positive. 61% found that OPENSME was providing the most reusable assets (e.g. source code, documentation etc.). 22% was neutral and 17% disagreed with the statement.



(R3) The OPEN-SME components were adequately described from the documentation provided with each component: 78% responses were positive and very positive in agreeing that OPENSME provided adequate documentation for the reusable artefacts, 17% were neutral and only 5% disagreed with this statement.



(R4) The OPEN-SME components were tested adequately and the ‘quality uncertainty’ factor preventing reuse was less of a problem: Regarding this requirement an impressive 94% said that they agree with the statement and only 6% disagreed. We think that the provision of HTML test documentation with each reusable component, including different types of test coverage, had a very positive effect.



(R5) The COMPARE repository was effective in enabling re-users to find appropriate components for their application: 89% agreed or strongly agreed that COMPARE repository was effective in enabling re-users to find appropriate components for their applications with only 11% being neutral and none negative. We think that this is because the OPEN-SME metadata (including domain classification) represent a considerable progress from what is usually expected




Page 34 of 41

from an Open-Source Software search engine, which as was discussed in D2.1, were not used very much by the developers.

3.3.5

STATUS OF THE OPEN-SME TOOLSET AND CRITICAL REVIEW

After SME-AGs reuse engineers and SME re-users have used and evaluated the tools during the trials and considering also the quantitative and qualitative evaluation described earlier we can conclude that overall the OPEN-SME toolset is effective in addressing the reuse needs for domain-specific open source software components. However there are some areas that can be improved. 1. The OCEAN search engine is a federated search engine with search capabilities that unify access to a number of search engines (e.g. Krugle, Merobase, Koders etc.). However using OCEAN it became evident that it is only as good as the existing engines that reuses. Therefore more research is necessary for more efficient open source software search engines. Towards this goal we describe later in Section Σφάλμα! Το αρχείο προέλευσης της αναφοράς δεν βρέθηκε. the integration of the Merobase search engine and the COPE tool for component extraction as a strategic goal of the OPEN-SME project in the future. 2. The COPE tool was very effective in supporting the reuse engineers of SME AGs in extracting domain specific components for their software development SMEs. However opportunities for improvement also exist such as the coverage of more programming languages. Currently COPE supports the Java programming language. However the modular design of COPE allows the extension of COPE to more programming languages. 3. The COMPARE repository was very effective in supporting re-users with 89% in our survey agreeing or strongly agreeing in the questionnaire’s related question (R5). The main need regarding COMPARE is the expansion of COMPARE with more components (currently COMPARE hosts around 100 components). However this is something that can and will be achieved with time as the OPEN-SME tool-chain is used in the future.




3.4

Page 35 of 41

REMARKS ON COMPETITIVE ASPECTS AND MARKET ENVIRONMENT

Reuse of open source software is just as common in the commercial world as in the academic world. In fact many commercial software organizations would not be able to function without some open source software. Using open source code in commercial projects imposes responsibilities on the user and provides interesting business opportunities since a demand arises for detection and reuse. There are already companies providing tools and consultancy in this market. Two examples are Palamida and Black Duck. In particular Black Duck provides consulting on best practices for managing code reuse. Evidently, a competitive business environment fosters the reuse of software. In a competitive sector the time-to-market becomes more important, and it becomes more critical to reduce the development effort and reduce the cycle time, which is a strong incentive for software reuse. In market segments that are less competitive, businesses are less likely to reuse software. Examples of competitive business environments include financial services and wireless networking. The defense sector is an example of a relatively uncompetitive market, and even though the Ada programming language was designed explicitly to support reuse, little reuse initially took place in the defense industry. Furthermore, the more complex the application domain, the more likely is the reuse of components. If it is trivial to program a stack, for example, this might be more cost and time efficient than searching for an existing piece of code or component. However, as the application domain becomes more complex, the cost benefit analysis of reuse is different as it is more likely less time-consuming and hence less costly to identify a component than programming from scratch, alongside with benefits of the component being better tested. Another important factor for re-usage of software code is the availability of tools and services, as these help significantly to decrease the search costs and time. Thus, the more and the better software reuse tools and services are available, the easier and more interesting it becomes for companies to consider code reuse. This is further supported by an increasing availability of components, and good documentation also helps the process. Further, well documented and maintained framework and repositories increases the organizational and developer confidence in them. There are however also potential problems and challenges, such as increased maintenance costs, lack of tool support, the “not invented here” syndrome, the creation and maintenance of a component library, as well as finding, understanding and adapting reusable components. However the major challenge when reusing code is to determine quality, documentation and licensing of the code considered for reuse. The lack of a way to establish the code quality through certification or download statistics is a market restraint as potential users may remain skeptical. Regarding the positioning of the OPEN-SME tools and repository in the market for OSS reuse it must be noted that despite the available wide range of tools companies mostly rely on tools that are widely known in the OSS world (such as forges), on existing test results and on basic tools that are commonly used in OSS projects. A tentative conclusion to be drawn from these observations is that usually companies, although used to OSS reuse, have only a limited overview of OSS reuse tools and services that are available at current. They use what everybody uses or what partner companies and project partners in the community recommend. Given the vast array of tools and the fragmentation aligned with this, it is obviously very difficult for a company to identify those tools and services that are most suitable for their purposes and to combine them systematically and efficiently. We have to assume that the vast majority of companies in the software industry identify and evaluate OSS code and components for reuse in a rather random manner based on isolated sub-tools. Given the fragmented market for tools, the competition in this market is characterized by a multitude of competitors (of which many are non-commercial in the sense that the tools are offered for free) which focus on specialized niche tools. These tools vary across application areas, technical characteristics, environment in which they operate and the intended target groups. Only Black Duck appears as a competitor in terms of toolsets. OPEN-SME/EMYPEE/WP4/D42



Page 36 of 41

That there exist a wide range of competitors for the OPEN-SME tool may be an advantage, it means that the competition is high in the sector in general, which generally leads to improved tool quality, which ultimately benefits the consumer and increases consumer welfare. The fragmentation of the market has some disadvantages, like difficulties for users to identify the most suitable tool, but it also means that there is an opportunity for diversification, which benefits both the supply and demand side. The growing demand for tools and toolsets also indicates that there is a real need for these products and services, and it is obviously a great opportunity for OPEN-SME to be in a growth market. The main value of the OPEN-SME tools therefore derives from the fact that they are offered as a set, and not from the isolated sub-tools. A possible further advantage of the OPEN-SME toolset could possibly be achieved if the OPEN-SME tools facilitate or provide helpful information for test implementations of the code or component to be analysed. Possible key success factors of the OPEN-SME toolset include branding/name recognition, access to unique resources, customer loyalty, financial resources, and access to distribution channels. The achievement of all these objectives can be facilitated through collaboration with SME-AGs and other actors in the OSS ecosystems. From a regulatory point of view it appears necessary to consider Black Duck in particular, as this company appears to be the main competitor for the OPEN-SME toolset since they are working on suites. Black Duck has already have established a very good reputation in the market. It could be argued that the OPEN-SME toolset creates unfair competition to Black Duck offerings because OPEN-SME is a publicly funded research project. However, it should also be noted that OPEN-SME combines and improves existing tools that are released under an open source software license, similar to what Black Duck does. In any case Black Duck needs to compete with other existing tools, and Black Duck is as everybody else of course free to use the OPEN-SME toolset itself, which will also be made available under an OSS license. Finally, it should be noted that Black Duck is currently the only vendor that develops and offers OSS reuse suites, which might lead to a monopoly if there exists no alternative product in the market that can be freely used and modified. Having two or more toolsets might have a positive effect on healthy competition – and secondly- help developing trust in this kind of tools and services on the side of consumers.

3.5 REMARKS ON SME BUSINESS ORIENTATION AND STRATEGY Software reuse is widespread in the European economy. Our research has shown that many SMEs consider software reuse as a means to save development costs and time. Most SMEs performing OSS reuse perceive these activities as a strategic and systematic undertaking. However, the practices that we could observe suggest that OSS reuse is performed rather as an ad-hoc approach than as a planned strategy. An ad-hoc approach to software reuse is characterised by a lack of or only limited effort in active search for reusable software and / or reuse of software in areas where the reuse of some software components is so wide-spread that almost all businesses operating in these markets do it without much pondering. In contrast to the ad-hoc approach, open source software as a planned strategy often aims at taking an advantage of the open source community, both as a target group for the product or services which can provide access to commercial software vendors’ services and encourage interactions, or as a resource for development help. It is found that this type of strategy often is focused on popular open source products such as Linux and Apache. Similarly, this approach can also be implemented through industry consortia, where a group of commercial software development organizations including, inter alia, IBM, Borland, Rational, Red Hat, SuSE, TogetherSoft, Webgain, Merant and QNX, have committed to using the Eclipse platform. In offering, delivering and pricing the reuse products and services there are various aspects to factor in, namely the time-frame, the software lifetime, the background, skills and experience of the development team, complexity, how critical the software is, non-functional requirements, and the application domain. The requirements and conditions for and the benefits from software reuse obviously vary across sectors OPEN-SME/EMYPEE/WP4/D42



Page 37 of 41

such as aerospace, data and telecom, medical, process automation, simulation, and financial services, along with the execution platform for the software. Given all the forms of reuse, the benefits and challenges, there is a differentiated reuse market, with the opportunity for a wide range of products and services. There are five main drivers of reuse by developers, namely the developers’ belief in: 

the effectiveness,



efficiency,



quality benefits,



task selection and



the potential loss of control that reuse brings.

This is a potential opportunity for OPEN-SME tools that automate the process and makes reuse easier, since it can decrease the reliance of belief or personal attitude in the search and decision process. Regarding the market size and trends in OSS reuse a number of studies come to different conclusions, resulting mainly from different methodologies. However, all these studies indicate a huge market potential for OSS reuse tools and services. Financial estimates for the market of code reuse are scarce. McConnel (1996 - quoted in Pedersen 2006) claims that “buying a third-party component typically costs the team 1 to 20 percent of what it would cost to develop internally.” A study by Black Duck Software performed in 2009 has analyzed 1311 popular open source projects, mostly written in Java. They found more than 365,000 instances of reuse, which they calculated corresponds to 316,000 staff years in terms of effort, which again represents billions of dollars in savings on development cost (Naik 2009). The possibility for reusing code is one of the main advantages of open source as compared to proprietary software, and reuse represents a great potential for value creation. In this context, it is necessary to consider the value networks where reuse takes place. In Deliverable D2.6b we argue that usually the end users (e.g. different industry segments, public or private organisations) do not directly interact with the Open Source developer communities. This opens leeway for “intermediators” between the end users and the developer community. These intermediators are companies that may be full OSS enterprises – as software developers, integrators, service providers or consultants - in the open source software business, either as software developers, integrators, service providers or pure consultants or proprietary-based software enterprises that produce software products and systems that are used in OSS environments. This range of intermediators also includes OSS reuse service and tools providers. These features apply fully to the value network of the OPEN-SME toolset and repository. As laid out in Deliverables D2.6a/b, the various actors in the OSS value network play different roles. In our case, there are two key actors in the value network of the OPEN-SME toolset: the technical academic partners of the OPEN-SME project provide the developers of a toolset for OSS re-usage and re-usage services, and the OPEN-SME -AGs in the consortium provide the distributors of the toolset and these services. The technical academic partners, primarily AUTH, compile and analyse a set of existing tools for the identification and evaluation of reusable OSS code and OSS components. These existing tools are transferred into a suite that allows fast and comprehensive reusability checks of OSS code and components, which is not offered by any single tool underlying the suite. This act provides the key value creation process within the OPEN-SME project. However, in a second step the suite must be adapted to the capacities and needs of the SME-AGs within the OPEN-SME consortium, which play the role of the key distributors of the Open OPEN-SME SME suite, as the academic partners within the consortium do not dispose of the required distribution channels and distribution expertise. The end users – primarily the target groups of the OPEN-SME -AGs, usually other SMEs and start-ups – either receive the results of an OSS reusability analysis carried out with the OPEN-SME suite by another actor (an SME-AG, a technical academic partner like AUTH, another company) based on requirements specified by the end user, use the OPEN-SME suite themselves in order to evaluate OSS code or components they want to reuse, or offer reusability services based on the OPEN-SME suite provided to them by an SME-AG. There is a vast variety of possibilities to design business models so that the interests and capacities of all actors within an OSS value network can be secured. Osterwalder & Pigneur (2010) suggest an OPEN-SME/EMYPEE/WP4/D42



Page 38 of 41

ontological approach to this subject that allows configuring business models based on 9 building blocks: 

Customer segments o





Value proposition o

Describes the value that is actually is sold to the customer segments

o

Can be a the OPEN-SME suite or services based on it

o

Depends on customer needs and capacities, e.g.

o





can he resell it?



can he modify it?



…

Delivery to customers through 

Communication channels



Distribution channels



Sales channels

Depends on SME-AGs (or AUTH’s) target groups and technical and organizational resources and constraints

Established and maintained with each customer segment

Result from value propositions that are successfully offered to customers

All assets required to offer and deliver the previously described elements

Key activities o



(how much) can he pay?

Key resources o





Revenue streams o



for what purpose does the customer need it?

Customer relationships o





Channels o



SME-AGs (or / and AUTH) have to define what their target group(s) shall be

A number of activities performed in order to offer and deliver the previously described elements

Key partnerships o

Some activities may be outsourced

o

Some resources may be acquired outside the enterprise

Cost structure o

All business model elements together result in the cost structure

The concrete application of this model to the OPEN-SME partners is described in detail in Deliverable D2.6b.




Page 39 of 41

4. LESSONS LEARNED AND FUTURE STEPS 4.1 LESSONS LEARNED 4.1.1

PROS



The OPEN-SME project was successful at addressing the requirements that were set forward in D2.1.



In both trials we can see noticeable cost avoidance due to reuse.



The OPEN-SME process provides access to the most reusable artefacts required by the re-users.



The OPEN-SME components are adequately described from the documentation provided with each component.



The OPEN-SME components are tested adequately and the ‘quality uncertainty’ factor preventing reuse is minimised.



The COMPARE repository is effective in enabling re-users to find appropriate components for their application.



There is a growing demand for similar tools and toolsets, which indicates that there is a real need for these products and services, and it is obviously a great opportunity for OPEN-SME to be in a growth market.



The main value of the OPEN-SME tools derives from the fact that they are offered as a set, and not from the isolated sub-tools. A possible further advantage of the OPEN-SME toolset could possibly be achieved if the OPEN-SME tools facilitate or provide helpful information for test implementations of the code or component to be analysed.

4.1.2

CONS



The OCEAN search engine is a federated search engine with search capabilities that unify access to a number of search engines (e.g. Krugle, Merobase, Koders etc.). However using OCEAN it became evident that it is only as good as the existing engines that reuses.



Currently COPE supports only the Java programming language.



Currently COMPARE hosts around 100 components only.



Diversity of the OPEN-SME-AGs in the OPEN-SME project consortium as they dispose of very different capacities to distribute the toolset / services and they pursue diverse strategies with this toolset.



ETEK and ISS have to raise awareness of OSS among their members as well as in their members’ domestic and regional key markets.




Page 40 of 41

4.2 FUTURE PLANS FOR DEVELOPMENT AND MAINTENANCE A follow-up strategy involves scaling-up the results of the OPEN-SME project and performing marketoriented adaptations of the tools in order to make them (a) more useful (i.e. covering more programming languages), (b) more accessible directly by the developers of SMEs as much as possible and (c) more widely used with an expanded component repository. Towards achieving these goals we propose the following activities: 1. Scale-up activities: a. Cover more programming languages. The OPEN-SME project prototype for the COPE (Component Adaptation Environment) tool was mainly built for the Java programming language. However an expanding percentage of Open-Source software is written in the C programming language, as can be seen in http://osrc.blackducksoftware.com/data/projects/, which is also important for the partner SMEs involved in embedded systems. Therefore one of the scale-up activities will be to enhance the tooling in order to support the C programming language. b. Expand the component repository and reverse engineer larger components. The OPEN-SME component repository currently hosts 100 components approximately. In order to make it more useful it would require components in the thousands. Also the components should be bigger (e.g. instead of tens of classes comprising hundreds of classes). The technology to achieve these goals was already developed as part of the OPEN-SME project. However in the context of a follow-up strategy more reuse engineers will be used to put the tooling to work in order to expand the component repository with larger and more useful components. 2. Market-oriented adaptation activities: a. Integrate the Merobase open source code search engine with the COPE tool. The COPE (Component Adaptation Environment) tool is designed to be used offline by a reuse engineer who analyses with it an Open Source project and extracts reusable components. On the other hand an existing and already powerful Open Source code search engine such as Merobase already provides accurate searching for possibly suitable reusable code. However with even a powerful search engine such as Merobase the reuser will still have to find the dependencies of a piece of software that he/she wants to reuse, assess its reusability, create manually a component for it etc. The integration of Merobase and COPE would provide an ideal market-oriented adaptation of the COPE tool making it immediately available from a web browser to a number of re-users. RTD performers have already held discussions with the developers of Merobase OSS search engine that took place in the last workshop of the OPEN-SME project ROSS in Bertinoro, Italy.




Page 41 of 41

5. REFERENCES 1. Jeffrey S. Poulin: “Measuring Software Reuse: Principles, Practices, and Economic Models”, Addison-Wesley, 1997 2. Jeffrey S. Poulin Reuse Calculator: http://jeffreypoulin.info/html/reucalc_basic.html 3. William M. Ulrich and Philip Newcomb: “Information Systems Transformation: Architecture-Driven Modernization Case Studies”, Morgan Kaufmann, 2010. 4. Black Duck, 2009, Black Duck Analysis of Open Source Shows Code Reuse, available at: http://www.newsfactor.com/story.xhtml?story_id=132002B3JZ7O&page=1 5. Helander, N. & Rissanen, T., 2005. Value-Creating Networks Approach to Open Source Software Business Models. Frontiers of E-Business Research, 2005, 840-854. Available online at: http://www.ebrc.fi/kuvat/840-854_05.pdf 6. Naik, V. 2009, Black Duck Study: Open Source Code Reuse Saves Big Time, available at: http://asterisk.tmcnet.com/topics/open-source/articles/53459-black-duck-study-open-source-codereuse-saves.htm 7. Pedersen, P. 2006, Best Practices for Managing Code Reuse, Black Duck Software, available at: http://www.blackducksoftware.com/files/webmedia/_Slides/Code-Reuse-Slides.pdf

