Identifying processes that support COTS-Based Systems in Developing Countries Douglas Kunda and Laurence Brooks Department of Computer Science, University of York, England
[email protected] Abstract Although there have been many advances in the information technology (IT) field, developing countries (DCs) have not yet fully benefited because of some of the specific problems experienced by these countries. Examples include a lack of systems infrastructure and resources to invest in IT. COTS-Based Systems (CBS) offers a number of benefits that the developing countries can tap into such as reducing development and maintenance costs and improving reuse across projects. CBS success depends on successful selection of COTS software to fit requirements. Although a number of initiatives have been proposed to deal with the COTS evaluation problems, they do not adequately address the non-technical issues or "soft" factor. The paper presents the research study to identify processes that support CBS in developing countries, for example it proposes a method of applying social-technical approach for COTS software selection.
Keywords COTS-Based Systems, developing countries, COTS software evaluation, socialtechnical evaluation criteria.
1
INTRODUCTION
The diffusion of computer technology in developing countries is at an embryonic stage and does not impinge upon overwhelming majority of the people (Corr, 1995). A range of factors, including various infrastructural, financial, political and cultural aspects have acted against the effective development and exploitation of information technology. Redressing this situation will require significant resources and a willingness to tackle the long-term underlying causes of the problem rather than offering short-term solutions to symptoms. Organisations in developing countries are turning to COTS-Based Systems because modern information systems are becoming increasingly expensive to build and maintain. These components are typically bought from third-party vendors and integrated into a system (Vigder, 1996). Alternatively an application that satisfies most of the system requirements could be purchased, and then extended, tailored for local requirements. In both cases neither is the source code for these components available to the system developer nor do the system developer control specification, release schedule and evolution of the components. Building systems from Commercial Off-The-Shelf (COTS) also known as COTSBased Systems development is focused on improving the technologies and practices used for assembling previously existing components (COTS and other non-developmental items) into large software systems, and migrating existing systems toward CBS approaches (Carnegie, 1998)(Clements, 1995). CBS can potentially be used to reduce software development and maintenance costs, and
reducing software development time by bringing the system to markets as early as possible (Clements, 1995)(Haines, 1997). CBS also improves reuse across programs and promotes a competitive component marketplace. CBS therefore has a higher potential to benefit developing countries compared to other systems development methods. Software systems do not exist in isolation they are used in social and organisational contexts. Experience and many studies show that the major cause of most software failures is the people rather than technical issues (Curtis, Krasner and Iscoel, 1988)(Le Quesne, 1988). For example poor training may result in people not co-operating with the information system leading to failure and project abandonment (Avison, 1995). CBS success depends on successful selection of COTS software to fit requirements. The current approaches and proposed frameworks for COTS software selection do not adequately deal with these human, social and organisational issues. For example the SEI Technology Delta framework (Brown 1996), Off-The-Shelf Option (Kontio, 1995), and Procurement Oriented Requirements Engineering (Maiden, 1998) do not adequately deal with political, social and economic factors (Powell, 1997)(Carnegie, 1998). This research aims at investigating a method of applying social-technical approach for COTS software selection.
2
MOTIVATION FOR RESEARCH
There are a number of reasons for interest in identifying processes that support CBS in developing countries and these include problems of developing software systems in DCs, potential benefits of CBS and experience of developing countries with CBS.
2.1 Problems of developing software systems in DCs Developed nations have used IT to help them change the way they do business so as to give them a strategic advantage in their operations (e.g. the use of ATMs in banks to improve customer service). However the investment returns in DCs have fallen short of the potential due problems unique to them. (Bogod, 1979) (Bhatnagar, 1992) attribute these to skilled human resources deficiency, economic constraints, systems infrastructure deficiency and applications problems. Solving these problems will require significant resources and government policies to tackle the underlying causes of the problem. Therefore the motivation for this research is not to solve these problems, rather it is to observe why and how COTS-based systems development can provide support for organisations in DCs to reduce costs related with software systems development and procurements.
2.2 Potential benefits of CBS CBS can potentially be used to reduce software development costs, assemble systems rapidly, and reduce the spiraling maintenance burden associated with support and upgrade of large systems (Clements, 1995)(Haines, 1997). Other potential benefits of building systems from COTS include cost benefit obsolescence management, improving reuse across programs and promoting competitive marketplace enabling system integrators a wide range of choices.
In addition to the increasing availability of components applicable to certain domains, understanding of the issues and technologies required to expand CBS practice is also growing, although significant work remains. Various new technical developments and products, including CORBA COM, DCOM, and related capabilities and changes in acquisition and business practices has further stimulated the move to CBS.
2.3 Experience of developing countries with CBS Developing countries are getting more and more familiar with COTS-based systems as the quality and variety of COTS products increase in the commercial marketplace. This means that the amount of existing software in these organisations that can be reused in new systems increases and this encourages CBS. Experience of developing countries in CBS also means that COTS-Based Systems can easily be adapted to their local needs and minimal training for customers (users) thus making substantial cost savings. Therefore COTS based systems leads to increased customer acceptance and systems ownership.
3
RESEARCH PROPOSAL
CBS success depends on successful evaluation and selection of COTS software components to fit customer requirements. COTS selection is a process of determining "fitness for use" of previously developed components that are being applied in a new system context (Haines, 1997). Component selection is also a process for selecting components when a marketplace of competing products exists. Successful selection of COTS software to fit requirements is still a problem because of a number of reasons (Oberndorf, 1997)(Maiden, 1998). These include the following: 1. Lack of well defined process - Most organisations are under pressure to perform and therefore do not use a well-defined repeatable process (Kontio, 1996). Therefore COTS selection process is implemented in an ad hoc manner. 2. Evaluation criteria - There are problems associated with the definition of the evaluation criteria. Sometimes the evaluators tend to focus on technical capabilities at the expense of the non-technical or "soft" factors such as the human and business issues (Powell, 1997). 3. "Black box" nature of COTS components - Lack of access to the COTS internals makes it difficult to understand COTS components and therefore makes evaluation hard (Oberndorf, 1997). Sometimes even the supporting documentation for these components is incomplete or wrong. 4. Rapid changes in market place - COTS evaluation is hard because of rapid changes of COTS components in the market place (Oberndorf, 1997). For example a new release of the COTS component may have a feature which is not available in the component that is currently being evaluated. Therefore the central research question can be framed as: What processes for the evaluation and selection of COTS software components provide support for CBS, in developing countries?
In order to address the problem of COTS evaluation in DCs, the best practices from more developed countries will be elicited, compared with relevant literature and then adapted for developing countries. Therefore the immediate objectives that arise from the central research question are: • To elicit and analyse current CBS practices of organisations from developed countries (UK) and developing countries (Zambia). • To identify processes and factors that support COTS software component selection in CBS from a developed country (the UK). • To explore how social-technical approaches can be used in COTS software components evaluation and adapted to developing countries. • To develop a theoretical framework showing how social-technical approach to COTS software evaluation can support CBS in developing countries.
4
LITERATURE REVIEW
4.1 COTS-Based systems CBS focuses on building large software systems by integrating previously existing software components (Haines, 1997). By enhancing the flexibility and maintainability of systems, this approach can potentially be used to reduce software development costs, assemble systems rapidly, and reduce the spiralling maintenance burden associated with the support and upgrade of large systems. At the foundation of this approach is the assumption that certain parts of large software systems reappear with sufficient regularity that common parts should be written once, rather than many times, and that common systems should be assembled through reuse rather than rewritten over and over. In CBS (Haines, 1997), the notion of building a system by writing code has been replaced with building a system by assembling and integrating existing software components. In contrast to traditional development, where system integration is often the tail end of an implementation effort, component integration is the centrepiece of the approach; thus, implementation has given way to integration as the focus of system construction. Because of this, a key consideration in the decision whether to acquire, reuse, or build the components is whether they can be integrated or not. In CBS, an organisation's software development process and philosophy may need to change (Haines, 1997). System integration can no longer be at the end of the implementation phase, but must be planned early and be continually managed throughout the development process. It is also recommended that as trade-offs are being made among components during the development process, the rationale used in making the trade off decisions should be recorded and then evaluated in the final product (Brown, 1996). Many of the problems encountered when integrating COTS components cannot be determined before integration begins. Thus, estimating development schedules and resource requirements is extremely difficult (Vigder, 1996). Developing component-based systems (Haines, 1997) is becoming feasible due to the following: • the increase in the quality and variety of COTS products; • economic pressures to reduce system development and maintenance costs;
• •
the emergence of component integration technology such as Object Request Broker; and the increasing amount of existing software in organisations that can be reused in new systems.
4.2 COTS software selection COTS software selection also known as component qualification is a process of determining "fitness for use" of previously-developed components that are being applied in a new system context (Haines, 1997). Component qualification is also a process for selecting components when a marketplace of competing products exists. Qualification of a component can also extend to include qualification of the development process used to create and maintain it (for example, ensuring algorithms have been validated, and that rigorous code inspection has taken place). This is most obvious in safety-critical applications, but can also reduce some of the attraction of using pre-existing components. There are three phases of COTS software selection: criteria definition, alternatives identification and evaluation (Kontio, 1996). The evaluation criteria definition process essentially decomposes the requirements for the COTS into a hierarchical criteria set and each branch in this hierarchy ends in an evaluation attribute. The criteria include component functionality (what services are provided), other aspects of a component's interface (such as the use of standards) and quality aspects that are more difficult to isolate, such as component reliability, predictability, and usability. The alternative identification involves the search and screening for COTS candidate components that should be included for detailed evaluation. In the evaluation phase the properties of the candidate components are identified and assessed according to the evaluation criteria. While there are several efforts focusing on component qualification, there is little agreement on which quality attributes or measures of a component are critical to its use in a COTS-Based system. A useful work includes Software Engineering Institute (SEI) delta technology framework that helps evaluate new software technology (Brown, 1996) and PORE, a template based method to support requirements acquisition for COTS product selection (Maiden, 1998). Another technique Off-The-Shelf Option (OTSO) addresses the complexity of component selection and provides a decision framework that supports multi-variable component selection analysis (Kontio, 1996). Other approaches, such as the one by Boloix and Robilland (Boloix, 1995) focus on assessing the software product, process and their impact on the organisation. These evaluation approaches typically involve a combination of paper-based studies of the components, discussion with other users of those components, and hands-on benchmarking and prototyping. However, the non-technical factors or "soft" issues are not adequately addressed by these approaches. An appropriate COTS software evaluation framework should be simple to use and address the political and economic factors that often separate a winning technology from other contenders (Carnegie, 1998)(Powell 1997).
5
PROPOSED APPROACH AND PRELIMINARY RESULTS
The research process consists of five main stages literature review, elicit current CBS practices, identify factors that support COTS selection, develop COTS selection framework and validate framework as illustrated in the Figure 1 below. In literature review phase, literature and papers on building systems from COTS software components, requirements engineering, IT capabilities in developing countries and other relevant topics will be reviewed. The purpose of this stage is to identify problems associated with building systems from COTS software components, to establish a sound background in software engineering and produce a theoretical framework for future study. Literature review will be an ongoing task throughout the period of this research. Elicit current CBS practices
Develop COTS selection framework
Literature Review
Validate framework
Identify factors that support COTS selection framework
Figure 1: Research Process
5.1 Eliciting current CBS practices 5.1.1 Goal and objectives The overall goal of the case study was to elicit and synthesise current practices and potential benefits of CBS in a developing country (Zambia). The outcome of this study would be to document and validate understanding of the current situation, problems (and solutions) people have experienced in relation to CBS. The following are immediate objective of the study: • obtain a thorough understanding of the CBS development process; • identify current practices for building systems using COTS software; • review current benefits, costs and risks associated with COTS software reuse; and • learn what kinds of problems (and solutions) people have experienced from the past in relation to building systems from COTS software.
5.1.2 Data collection method There are several data collection methods such as documentary evidence, questionnaires, interviews, diaries and observation studies. Data was collected through the administration of self-completion questionnaires to a sample of organisations. This method has been adopted because questionnaire provides wide
access to geographically dispersed samples at low cost i.e. large population can be surveyed relatively cheaply (Frankfort-Nachmias, 1996). In addition questionnaire provides a high degree of anonymity. This is especially important when sensitive issues are involved. The main arguments against using mail questionnaires is that it does not offer the researchers the opportunity to probe for additional information or to clarify answers and researchers cannot control the respondent's environment. Questionnaires also require simple, easily understood questions and instructions. The final disadvantage of a mail questionnaire - and perhaps the most serious problem - is that it is often difficult to obtain an adequate response rate (FrankfortNachmias, 1996). The questionnaire was based on literature review conducted in the first three months and discussion with faculty members and other research students within the Department of Computer Science. After piloting the questionnaire was modified to incorporate comments and then administered to the sample companies in Zambia and the UK. The questionnaire was organised under three main themes, the overview of development process; requirements engineering phase and approaches to CBS. The sources of data are personnel within institutions responsible for specifying, procuring and developing software systems. Zambia institutions were obtained by purposively sampling from the National Directory for those that were familiar with CBS. The list of UK organisations was acquired from Kompass database and systematic sampled to reduce the sample to 1000.
5.1.3 Data analysis procedures The data collected were coded and entered in SPSS (Statistical Package for Social Science) for analysis. Frequency distributions were used to categorise the demographic data variables. The mean (8 ) were calculated to measure the central tendency of the variables unlike the mode or median it takes into account all the values in the distribution, making it sensitive to extreme values. The standard deviation (S.D.) were calculated to measure variability because it is more stable from sample to sample and can be used for two or more combined groups. However the coefficient of variation (CV) were calculated and used as standard to compare the relative importance of the variables.
CV =
( S .D .)100 X
5.1.4 Preliminary results The respondents were asked to rate their strength of agreement to some factors related to building systems from COTS software. The questionnaire consisted of scaled-response from 1 (strongly disagree) to 5 (strongly agree). Table 1 presents some of the results from this study with significant factors shown inbold font. This selection is based on comparing the investigated factor's coefficient of variation (C.V.). The full report detailing the results of this empirical study is documented separately (Kunda, 1998) and this paper presents the important findings and conclusions.
Table 1: Risks associated with COTS-Based system development Factors Investigated Lack of guidelines Technical capability Periodic releases of COTS Loss of schedule control Legal implications Product mismatches Side effects Additional tasks Failure to meet requirements Lack of provider support Difficult to select
UK Organisations (C.V.) 2.95 4.58 3.63 3.13 2.80 3.69 3.46 3.51 3.96 4.24 3.53
Zambian Organisations (C.V.) 1.75 2.67 3.34 2.28 1.90 3.54 2.45 1.78 3.27 2.50 2.85
The results indicate that: 1. The most significant benefit of building systems from COTS components is reducing the software development costs and improving reuse across projects. This supports the findings already identified in literature (Haines, 1997)(Oberndorf, 1997). 2. The notable problems identified in this study are lack of access to the technical information and COTS product mismatches (refer to table 1). The difficult to discover the technical capabilities of COTS components was expected because of the "black box" nature of COTS components. 3. The lack of adequate trained human resources was identified as the main obstacle to developing software systems. This is not surprising because it has been identified in literature on developing countries (Woherem, 1992). What is surprising is that although literature (Curtis, 1988) indicates the importance and impact of political and external issues on the success of the information system it scored lowly in this survey. 4. The most notable technique for requirement acquisition and specification is observation, prototyping and demonstrations. It was interesting to note that the techniques advocated by social-technical approaches such as rich pictures and SSM conceptual models scored very low in this survey. 5. COTS component selection is still a problem especially in developing countries where donors sometimes impose these on them. For example the study results indicated that most of the institutions surveyed conduct software selection in ad hoc manner focussing on technical issues.
5.2 Identifying processes and factors that support COTS component selection 5.2.1 Goal and objectives The overall goal of this study is to identify best practices from the industry in the UK regarding evaluation and selection of COTS software components for CBS. The following are the immediate objective of the case study: • Identify problems (and solutions) experienced by companies in evaluating COTS components for CBS and define criteria for judging COTS evaluation success;
• •
Elicit techniques and tools for evaluating COTS components and consider in what ways they promote evaluation success; Review the COTS components evaluation criteria used by companies (whether it includes the social-economic factors) and investigate in what ways customer participation can contribute to COTS components evaluation success.
This study will provide an opportunity to learn the best practices from UK enterprises, validate the literature review and document the best practices from the UK industries. The outcome of this study would lead to the development of a validated framework for selecting COTS components for CBS. The framework would also incorporate the often-neglected non-technical issues.
5.2.2 Research strategy A multiple case study approach has been used to identify the factors that support COTS component selection in CBS. A Case study is an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not evident (Yin, 1994). The case study is preferred when the investigator cannot manipulate or control the relevant behavioural events. The argument against the case study approach is that It provides little for basis for scientific generalisation, how can you generalise from a single case study it is argued. In addition case studies take too long and they result in massive, unreadable documents.
5.2.3 Data Collection and analysis Data collection for this case study relied on face-to-face interviews and documentation. The advantage with the interview technique is that it focuses directly on case study topic and provides perceived causal inferences. The major criticism against interviews is that it expensive and bias from response or poorly constructed questions. The grounded theory method is being used to analyse data from this case study. A grounded theory is one that is inductively derived from the study of the phenomenon it represents (Strauss and Corbin, 1990). That is, it is discovered, developed, and provisionally verified through systematic data collection and analysis of data pertaining to that phenomenon. This involved data preparation, developing concepts through coding and building causal networks. Data preparation involves converting field notes into write up notes and transcribing the tape recordings. Developing concepts was done through coding in Atlas-ti software and these concepts were further categorised into themes. Atlas-ti is a software tool for qualitative data analysis and is very efficient at handling both textual and conceptual analysis. The next stage was to tabulate the results and build causal models or networks that link variables or factors together.
5.2.4 Preliminary results A total of nine interviews were conducted in five organisations within the UK. These organisations were selected on the basis of their experience with CBS and willingness to participate in the case study. However deliberate effort was made to ensure wide variety organisations from different sectors are included in the case study. A number of factors that support COTS software component selections
were elicited from this study. The identified factors relating to the criteria definition are presented in table 2. Table 2: Identified social-technical criteria factors affecting COTS component selection Compliance issues (Functionality) • Customer/Organi sations standards • Functionality (domain specific)
Product quality characteristics • Interoperability • Portability • Scalability • Performance • Dependability • Efficiency • Reliability • Usability
Socio-economic (non-technical) • Costs in general • Licensing arrangements • Market trends • Vendor capability • Vendor reputation
Technology • Architectural styles and frameworks • Interface issues • Multi-user support • Security
The Socio-economic factors are non-technical factors such as costs, market trends, vendor capability and reputation. These are the most frequently overlooked factors that bring in the social dimension in the evaluation criteria. Costs include direct cost such as the price of the COTS software products and indirect costs such as cost of adapting and integrating the products, cost of maintenance or replacement with upgrades, cost of training and support. Vendor capability factors are performance and capability of the vendor, for example vendor profile, reputation, period of vendor business, vendor reputation, references, customer base and track record, certification, stability, available training and support.
6
CONCLUDING REMARKS
Although there are some risks associated with CBS, it has great potential in the developing countries not only because of potential cost saving but also because developing countries are very familiar with COTS components, at least in terms of use and adapting them to local needs. Developing countries should use CBS to its full potential by integrating COTS components in an engineering manner rather than simply adapting them. CBS success depends on successful evaluation and selection of COTS software components to fit customer requirements. The literature shows that successful selection of off-the-shelf systems to fit customer requirements remains problematic (Maiden, 1998). Existing frameworks for COTS evaluation such as the Delta and OTSO are inadequate because they do not incorporate the socialeconomic factors satisfactorily. In addition these frameworks are laborious and too complex to be adopted for general use. This research will contribute towards a social-technical approach to CBSD in developing countries. There are research efforts both in academic and empirical world relating to CBSD, social-technical approaches and developing countries.
This research will concentrate its efforts where the three research fields intersect (see Figure 2).
Developing Countries
CBS
Socialtechnical
Research contribution
Figure 2: Research Contribution This research work will therefore provide an original contribution to knowledge in its contribution of a social-technical approach for CBS applicable in developing countries. The contribution will be three-fold: 1. The project will elicit a better understanding of CBS constraints faced by developing countries. 2. The project will provide a list of techniques and tools that practitioners can use for COTS software evaluation. 3. The project will provide a framework for social-technical approach to COTS software evaluation supporting CBS in developing countries. This framework will: • Provide guidance for organisation in developing countries regarding the process of evaluating and selecting COTS software components in CBS. • Provide techniques, tools and support for COTS software component evaluation phase of CBS for developing countries. This will contribute to reducing risks associated with CBS. • Provide support for a social-technical approach to COTS software component evaluation process for developing countries.
REFERENCES Avison D. E and Fitzgerald G., Information Systems Development: Methods, techniques and tools, McGraw-Hill Book Company, London, 1995. Bhatnagar S C (editor), Social Implications of Computers in Developing Countries, Tata McGraw-Hill Publishing Company Limited, New Delhi, 1992. Bogod Julian, The role of computing in developing countries, British Computer Society, London, 1979. Boloix Germinal and Robillard Pierre,A Software System Evaluation Framework, IEEE Computer Vol. 28 No. 12, pages 17-26, December 1995. Brown Alan W. and Wallnau Kurt C., A Framework for Systematic Evaluation of Software Technologies, IEEE Software, September 1996.
Carnegie Mellon University, CBS Overview, Carnegie Mellon SEI, Available WWW (online) , 1998. Clements Paul C., From Subroutines to Subsystems: COTS-Based Systems, American Programmer, V8#11, Cutter Information Corp., November 1995. Corr Patrick H., Computer Education in Least Developed Countries: A Case Study of Zambia chapter in Technology and developing countries: Practical applications, theoretical issues, Heeks Richard, Bhatt Prabhakara, Huq Mozammel, Lewis Chris, Shibli Ahmed (editors), Frank Cass & Co. Ltd, London, pages 87-100, 1995. Curtis Bill, Krasner Herb and Iscoe Neil, A field study of the software design process for large systems, Communication of the ACM, 31(11):1268-1286, November 1988. Frankfort-Nachmias Chava and Nachmias David,Research Methods in the Social Sciences, 5th Edition, Arnold a member of the Hodder Headline Group, London, 1996. Haines Capt Gary, Carney David and Foreman John, COTS-Based Systems/ COTS Integration, Software Technology Review, Available WWW (online) , 1997. Kontio Jyrki, A Case Study in Applying a Systematic Method for COTS Selection, Proceedings of the 18th ICSE, IEEE Computer Society, 1996. Kontio Jyrki, OTSO: A Systematic Process for Reusable Software Component Selection, University of Maryland Technical Report No. CS-TR-3478, 1995 Kunda Douglas, Identifying current practices for COTS software – leading to best practices for developing countries, Qualifying Dissertation, Department of Computer Science, University of York, York, 1998. Le Quesne P. N., Individual and Organisational factors and the Design of IPSEs, Computer Journal, 31(5):391-397, 1988. Maiden Neil A. and Ncube Cornelius, Acquiring COTS Software Selection Requirements, IEEE Software, pages 46-56, March/April 1998. Oberndorf Patricia A. and Brownsword Lisa and Morris Ed, Workshop on COTSBased Systems, Software Engineering Institute, Carnegie Mellon University, Special Report, CMU/SEI-97-SR-019, November 1997. Powell Antony, Vickers Andrew, Lam Wing and Williams Eddie,Evaluating Tools to Support Component Based Software Engineering, Proceedings of the 5th International Symposium on assessment of software tools, IEEE Computer Society, Los Alamitos, 1997. Strauss Anselm and Corbin Juliet, Basics of Qualitative Research: Grounded Theory Procedures and Techniques, Sage Publications, London, 1990. Vigder Mark R., Gentleman W. Morven and Dean John, COTS Software Integration: State of the art, National Research Council, Canada, NRC Report Number 39198, 1996. Woherem Evans E., IT Manpower Development Strategy at the Organisational level chapter in Information Technology Manpower: Key Issues for Developing Countries, Bhatnagar S.C. (editor), Tata McGraw-Hill Publishing Company Limited, New Delhi, pages 147-157, 1992. Yin Robert K., Case study Research: Design and Methods, Sage Publications, London, 1994.
