The Development and Experimentation of an ...

Computing Professionals 2010, April 21-23 2010, Montréal, Canada

The Development and Experimentation of an International Standard for Very Small Entities Involved in Software Development Claude Y. Laporte, Department of Software and IT Engineering, École de technologie supérieure, Montréal, Canada, Rory V. O'Connor, School of Computing, Dublin City University, Dublin, Ireland, and Lero, The Irish Software Engineering Research Centre

Abstract Very Small Entities (VSEs) are enterprises, departments, or projects having up to 25 people. IT VSEs are very important to the world economy, as their software components are often integrated into the products of larger entities. A large percentage of software entities worldwide have up to 25 employees. In Europe, for instance, 85% of Information Technology (IT) sector companies have 1 to 10 employees. A survey of the Montreal, Canada, area has revealed that 78% of software development enterprises have up to 25 employees and 50% have fewer than 10 (Gauthier 2004). Their software components are often integrated into the products of larger entities. Failure to deliver a quality product on time and within budget threatens the competitiveness of VSEs and impacts customers. Also, with the current trend towards outsourcing, it is critical for customers to be able to depend on VSEs to deliver their products, or the business will go elsewhere. Failure to deliver a quality product on time and within budget threatens the competitiveness of organizations of all sizes. One way to mitigate these risks is for the suppliers of a product chain to put in place proven software engineering practices, such as those documented in standards. Many international standards and models, like ISO/IEC12207 or the Capability Maturity Model Integration developed by the Software Engineering Institute, have been developed to capture proven engineering practices. However, these standards and models were not designed having in mind VSEs and are consequently difficult to apply in such settings. Also, research shows that VSEs often find it difficult to relate ISO/IEC standards to their business needs and to justify the application of the standards to their business practices. Most of them cannot afford the resources  in number of employees, cost, or time  to establish software life cycle processes, or see a net benefit to doing so. An ISO/IEC JTC1/SC71 Working Group, Working Group 24, was established in 2005 to address these difficulties by producing an international software engineering standard, the ISO/IEC 29110, which is specifically tailored to the problems and needs of VSEs. An international survey was developed and translated into 9 languages. It was conducted among VSEs and concerned their utilization of standards. The data collected were used to validate the list of needs and problems that had been documented by working group 24. Over 435 responses were received from 32 countries. The working group used the results to define the requirements that would form the basis for the standards to be developed for VSEs. This paper presents a perspective on the development of ISO/IEC 29110 standards and technical reports and outlines the development of a series of deployment packages facilitating the implementation of a set of ISO/IEC 29110 practices. A deployment package is a set of artifacts developed to facilitate the implementation of

1 ISO/IEC JTC 1/SC7 stands for the International Organization for Standardization / International Electrotechnical Commission Joint Technical Committee 1 / Sub Committee 7, which is in charge of the development and maintenance of software and systems engineering standards

Computing Professionals 2010, April 21-23 2010, Montréal, Canada a set of practices, of the selected framework, in a VSE. The elements of a typical deployment package are: a detailed description of processes, activities, tasks, roles, and products; a template; a checklist; an example; a reference and a mapping to standards and models; and a list of tools. This paper also details the formation of a Network of Support Centers for VSEs to accelerate the diffusion and facilitate pilot implementation of this emerging ISO standard. The recent establishment of an education interest group is also presented. The aim of this group is to accelerate the teaching of the new ISO standards in educational institutions. The course material developed for academia will consist of education deployment packages similar to the development packages for VSEs. Finally, a series of pilot projects in VSEs around the world conducted to validate the work of the working group as well as the future work are discussed. The set of documents have been sent for a series of 4 reviews by ISO participating countries. Over 1249 comments have been discussed and disposed by the working group. It is anticipated that the standards and technical reports will be ready for publication by ISO in late 2010 or early 2011. Keywords: Very Small Entities, life cycle, ISO Standards, deployment package, pilot projects

1 Introduction In the current economic environment, software quality is increasingly seen as a subject of concern for the growth and evolution of software companies in general, no matter what their size or the type of products or services they offer. In particular, Very Small Entities (VSEs) have a pressing need to develop high-quality products efficiently and effectively. With the current trend towards outsourcing, it is critical for customers to be able to depend on these enterprises to deliver their products on time, or the business will go elsewhere. It is equally important that the businesses perform well while making a profit. Of course, all companies have these needs, but, because of the limited resources of VSEs, even a small problem can have huge repercussions for them. Consequently, it is particularly important that VSE management identify resource issues before they turn into major difficulties. Unfortunately, most software development and maintenance time is spent on the development of new products and features, not on fixing old bugs which were not noticed until they became big problems. Quality-orientated process approaches and standards are maturing and gaining acceptance in many companies. More than anything else, standards emphasize communication and shared understanding. For example, all documentation must be consistent and serve the needs of the organization, and all users must agree on the meaning of the terms used (that is, if an individual says, “Testing is completed!,” then all the affected bodies understand what is meant). This kind of understanding is not only important in a global development environment, as even a small group working in the same office might have difficulty communicating and understanding issues shared by all. Standards can help in these and other areas to make the business more profitable, because less time is spent on non productive work. However, at a time when software quality is becoming key to competitive advantage, the use of ISO/IEC systems and software engineering standards remains limited to a few of the most popular ones. Research shows that VSEs can find it difficult to relate ISO/IEC standards to their business needs and to justify the application of the standards to their business practices. Most of them cannot afford the resources  in number of employees, cost, or time  to establish software life cycle processes, or see a net benefit to doing so. There is sometimes a disconnect between the short-term vision of the company, looking at what will keep it in business for another six months or so, and the long-term benefits of gradually improving the ways the company can manage its

Computing Professionals 2010, April 21-23 2010, Montréal, Canada software development and maintenance. A primary reason cited by many small software companies for this lack of adoption of ISO standards is the perception that they have been developed for large multi-national software companies and not with the small organization in mind (Coleman & O'Connor, 2008). Consequently, VSEs have a very limited number of ways (or none) to be recognized as enterprises that produce quality software systems in their domain, and may therefore be cut off from some economic activities. Accordingly, there is a need to help such organizations understand and use the concepts, processes, and practices proposed in ISO/IEC JTC1/SC7’s international software engineering standards. This paper presents a new project intended to facilitate access to, and utilization of, ISO/IEC JTC1/SC7 software engineering standards in VSEs. This paper is divided into nine sections. Section 2 presents the concept of a VSE and describes the characteristics that distinguish it from other organizations. Section 3 presents a historical perspective on the events that led to an ISO/IEC JTC1 SC7 project proposal for VSEs, and section 4 presents the results of a survey that was developed to question VSEs about their use of ISO/SC7 standards. Section 5 explains the approach being taken by the VSE working group, section 6 describes recent developments, section 7 presents the experimentations conducted, section 8 presents the Education Interest Group, whose aim is to accelerate the teaching of the new ISO standards in educational institutions. Finally, section 9 presents concluding remarks and discusses future actions.

2 Very Small Entities The definition of small and very small entities is challenging, as there is no general agreement on what the terms “small” and “very small” mean in this context, and even the participants in the 1995 Capability Maturity Model (CMM) tailoring workshop (Ginsberg & Quinn, 1995) could not agree on their meaning. Subsequently, the 1998 SEPG conference panel on the CMM and small projects (Hadden, 1998) defined a small project as “3-4 months in duration with 5 or fewer staff.” Johnson and Brodman (1998) define a small organization as “fewer than 50 software developers” and a small project as “fewer than 20 software developers.” Another definition of the VSE introduced by Laporte et al. (2006a) is “any IT services, organizations and projects with between 1 and 25 employees.” The more legalistic perspective of the European Commission (2005) defines three levels of small to medium-sized enterprises (SMEs) as follows: small to medium-sized  “employ fewer than 250 persons and which have an annual turnover not exceeding 50 million Euro, and/or an annual balance sheet total not exceeding 43 million Euro”; small  “employ fewer than 50 persons, and whose annual turnover or annual balance sheet total does not exceed 10 million Euro”; and micro  “employ fewer than 10 persons and whose annual turnover or annual balance sheet total does not exceed 2 million euro ”. To better understand the apparent contradictions in the above definitions, we need to examine the size of software companies operating in the market today. In Europe, 85% of Information Technology (IT) sector companies have 1 to 10 employees. In the case of indigenous Irish software firms, only 1.9% (10 companies) out of a total of 630 employed more than 100 people, while 61% of the total employed 10 or fewer, with their average size being about 16 employees (Coleman & O'Connor, 2008). In Canada, a survey of software development enterprises in the Montreal area has revealed that 78% have fewer than 25 employees and 50% have fewer than 10

Computing Professionals 2010, April 21-23 2010, Montréal, Canada employees (Laporte et al., 2006b). In Brazil, small IT companies (companies with fewer than 50 employees) represent about 70% of the total number of companies (Anacleto et al., 2004). Based on the above statistics and the debate within the ISO community, we are adopting the definition of a VSE introduced in Laporte et al. (2006b) for the purposes of this paper, as follows: “any enterprise, organization, department, or project having up to 25 people.” This is the definition agreed to by the various national representatives of Working Group 24 of ISO/IEC JTC1 SC7, whose objectives and history are described in section 3. 2.1 Characteristics of a VSE The unique characteristics, and needs, of small enterprises make their styles of business different (Mtigwe, 2005). Some of the unique features of small and large enterprise behavior are given in Table 1. Moreover, software VSEs are subject to a number of distinctive and intrinsic characteristics that differentiate them from their larger counterparts, which affects the content, nature, and extent of their activities. In the table below, we base our comparison of VSE characteristics to those of large enterprises on four main categories: financial constraints, typical customer profile, the focus of internal business processes, and the constraints on learning and growth:

Characteristic Planning orientation Flexibility Risk orientation Managerial process Learning and knowledge absorption capacity Impact of negative market effects Competitive advantage

Small enterprise Unstructured/operational High High Informal Limited

Large enterprise Structured/strategic Structured/strategic Medium Low High

More profound Human capital-centered

More manageable Organizational capital-centered

Table 1. Characteristic differences between small and large enterprises VSEs are economically vulnerable, as they are driven by cash flow and depend on project profits, so they need to carry out their projects within budget. They tend to have low budgets, which has many consequences, such as: lack of funds to perform corrective post delivery maintenance; few resources allocated for training; little or no budget to perform quality assurance activities; no budget for software reuse processes; low budget to respond to risks; and limited budget to perform Process Improvement or to obtain a certification/assessment of their processes. Typically, a VSE product has a single customer, and it is that customer who is in charge of the management of the system, as well as software integration, installation, and operation. It is not current practice for the customer to define quantitative quality requirements or for customer satisfaction to depend on the fulfilment of specific requirements that may change during the project. A close relationship among all the involved project members, including the customer, shows that software development in small and very small companies is strongly human-oriented and communication between them is important. For example, in contrast to small companies, very small companies often do not have regularly scheduled project meetings (Laporte & April, 2006).

Computing Professionals 2010, April 21-23 2010, Montréal, Canada The internal business process of VSEs is usually focused on developing custom software systems, where the software product is elaborated progressively and incrementally, and typically software projects are independent of one another. Usually, most management processes (such as human resource and infrastructure management) are performed through informal mechanisms, with the majority of communication, decision making, and problem resolution being performed face to face. The learning and growth characteristics of VSE are characterized by a lack of knowledge (or acceptance) of software process assessment and improvement, and a lack of human resources to engage in standardization. It is usual for there to be a negative perception of standards as being made by the staff of large enterprises, for large enterprises (Coleman & O'Connor, 2008).

3 History of the ISO/IEC Working Group for VSEs The mandate of ISO/IEC JTC1/SC7 is the standardization of processes, supporting tools, and supporting technologies for the engineering of software products and systems. A description of SC7 and of the development of ISO/IEC JTC1/SC7 standards is presented in (Coallier, 2003). In this section, a brief history of the events leading to the creation of a new ISO/IEC JTC1/SC7 Working Group (WG) is presented. A detailed description of its history is available in (Laporte & April, 2006). At the May 2004 SC7 Plenary meeting in Brisbane, Australia, Canada raised the issue of small enterprises requiring standards adapted to their size and maturity level. The current software engineering standards target (or are perceived to target) large organizations. A meeting of interested parties was organized and a consensus was reached on general objectives for a future working group:  To make the current software engineering standards more accessible to VSEs;  To provide documentation requiring minimal tailoring and adaptation effort;  To provide harmonized documentation integrating available standards: process standards, work products and deliverables, assessment and quality, modeling, and tools;  To align profiles, if desirable, with the notions of the maturity levels presented in ISO/IEC 15504. In March 2005, the Thailand Industrial Standards Institute (TISI) invited a Special Working Group (SWG) to advance the work items defined at the Brisbane meeting. A key topic of discussion was to clearly define the size of VSE that the SWG would target, consensus being reached on IT services, organizations, and projects with 1 to 25 employees. The major output of this one-week meeting was a draft of the New Work Item (NWI) to be tabled at the next SC7 meeting. In May 2005, a resolution was approved to distribute for ballot the NWI Proposal for the Development of Software Life Cycle Profiles and Guidelines for use in Very Small Enterprises. Twelve countries voted in favor of the NWI Proposal (NWI, 2005). As a result of this vote, the project was approved and the new working group, WG24, was established. The TISI sent out a second invitation to participate in the SWG, to be held in September 2005 in Bangkok. The main objective of the meeting was to prepare material that would be presented to WG24 in order to facilitate the start-up of the working group that was scheduled for October 2005 in Italy.

Computing Professionals 2010, April 21-23 2010, Montréal, Canada In October 2005, Italy hosted the ISO/IEC JTC1 SC7 Interim Meeting. The NWI was updated to take into account relevant comments received during balloting, and the requirements were validated by WG members. In addition, some VSE Business Models were identified, along with a strategy for creating profiles. Finally, WG24 decided to conduct a survey to collect relevant information from VSEs around the world.

4 Gathering VSE Requirements In 1997, the Technical Council on Software Engineering responsible for IEEE Software Engineering Standards conducted a survey to capture information from software engineering standards users in order to improve those standards (Land, 1997). They gathered 148 answers, mainly from the USA (79%) and large companies (87% of them having more than 100 employees). The main application domains of the survey respondents were IT (22%), military (15%), and aerospace (11%). (It should be noted that the purpose of this section is not to systematically compare the two sets of survey results.) Even though the IEEE survey objectives differ from those of the ISO/IEC survey, there are some interesting common findings. In response to the question concerning the reasons why their organization does not use standards, 37% said that the standards were not available at their facilities, while 37% explained that they use other standards. In fact, the IEEE survey underscores the fact that ISO/IEC standards are often used in organizations, rather than the IEEE standards. The IEEE survey highlighted the difficulties associated with the use of IEEE standards reported by the respondents. The two main difficulties were a lack of understanding of the benefits (28%) and a lack of useful examples (25%). The survey also revealed how IEEE standards are used in organizations, most of the organizations claiming to use them for internal plan elaboration. The IEEE survey gathered several new requirements for IEEE standards being requested by the respondents. These were principally examples and templates of deliverables, support for metrics and measurement, help on life cycle process definition, a training course, and support for small, rapid application development efforts. The WG24 survey was developed to question VSEs about their use of ISO/SC7 standards and to collect data to identify problems and potential solutions to help them apply standards and become more competitive. It was intended to validate some of the hypotheses regarding VSEs drawn up by WG24 at the very start of the survey process, such as the following:    

The VSE context requires light and well-focused life cycle profiles. Particular business contexts require particular profiles. VSEs are limited in both time and resources, which may lead to a lack of understanding of how they can use the standards for their benefit. Benefits for VSEs may include recognition through assessment or audit by an accredited body.

The survey questionnaire and an introductory text were developed by WG24 and translated into 9 languages: English, French, German, Korean, Portuguese, Thai, Turkish, Russian, and Spanish. It is made up of 20 questions structured in 5 parts: General information, Information about standards utilization in VSEs, Information about implementation and assessment problems in VSEs, Information about VSE needs, and Information about justification for compliance to standard(s). Over 392 responses were collected from 29 countries. 4.1 Categorization of the sample according to the size criterion

Computing Professionals 2010, April 21-23 2010, Montréal, Canada

Of the 392 responders, 228 are enterprises with 0 to 25 employees (58%), as illustrated in Figure 1. Note that responders which are small organizations (