An Empirical Study: Understanding Factors and ...

An Empirical Study: Understanding Factors and Barriers for Implementing Agile Methods in Malaysia Ani Liza Asnawi

Andrew M Gravell

Gary B Wills

School of Electronics & Computer Science University of Southampton SO17 1BJ, United Kingdom



[email protected]

[email protected]

[email protected]

ABSTRACT This paper focuses on empirical work for a doctoral research. We investigate factors that can bring about the adoption or rejection of agile methods in Malaysia. As part of this research our initial work involved the identification of the current levels of usage and perception of the methodology in the country. A pilot study using a questionnaire was conducted in two languages, English and Malay. Our initial results indicate that of the 79 respondents more than 50% do not have an awareness of agile methods. Despite their unawareness, most of the respondents believe in agile values. The awareness of agile methods is also found to have a significant association with the language respondents used and two aspects of organisational structure asked from the questionnaire. This finding provides a promising environment for the application of agile methods in the country. It was also identified that respondents perceived both aspects (social and technical) to be equally important in delivering the benefits of agile methods. The contribution from this study is used to provide guidelines relating to the factors concerning, and barriers facing, the adoption of agile in the country, where little knowledge and studies regarding the methods can be found. The results are also used to fill the research gap of inconclusive evidence found from previous studies.

Categories and Subject Descriptors

reported that agile methods can deliver quality software [2]. Despite the many benefits that agile can deliver (e.g. [3, 4]), little work has been published regarding its current usage in the Southeast Asia region [5]. If the methods are to be used, we need to understand the penetration, perception, factors and barriers related to the adoption of agile methods in this region. This is important because in order for agile methods to deliver good quality, they still depend on the people and community to practice them [1]. We have chosen Malaysia as our case study because Malaysia is a multinational country that consists of three different ethnics groups (Malay, Chinese and Indian) [6]. Therefore, it is considered essential to conduct this study as the result may differ from those previous studies regarding agile adoption. Begel and Nagappan [4] conducted a study on the perception of agile methods worldwide, however in terms of adoption; there was no discussion on the cultural comparison. Many studies have been conducted in terms of agile methods adoption [7-11] but some of them were not consistent (eg. [3, 12] and no results were identified [7]. For example, results in terms of organisational culture and the environment were not consistent (examples in [3, 13]) and were not tested [14]. The inconsistency, the lack of published studies in this region, and the unique identity of the country has motivated us to fill in the gaps to validate the results of factors suitable for agile. Section 1.1 describes problems and subsequent motivation for the study.

Management, Measurement, Human Factors.

The rest of the paper reports objectives and expected contribution of the study, followed by relevant works, methodology and results we have obtained so far. We then describe works in progress, conclusion and finally provide directions for the future work. In this paper, term of ‘agile methods’, ‘agile method’ and ‘agile’ are used interchangeably.

Keywords

1.1 Problems and Motivation

K.6.3[Management of Computing and Information Systems]: Software Management - software development, software process software selection.

General Terms

Agile methods, perception, adoption factors, survey, structural equation modeling (SEM).

1. INTRODUCTION Agile methods are sets of approaches to develop software which place emphasis on people and their creativity [1]. It has been Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. IDoESE’10, Sept, 2010, Bolzano, Bozen, Italy. Copyright 2010 ACM 1-58113-000-0/00/0004…$5.00.

The people factor is important in agile [1, 15] and so is the organisational structure [13]. Both of them are considered as an important factor before the adoption of agile methods [16]. “Organisational culture, environment and people are all influence each other”[1]. One study has confirmed that environmental factors (for example under the categories of organisation, projects and people) have a role in delivering the advantages of agile [17]. While [12, 13, 18] found organisational structure and culture to be important however one investigation did not find a significant result [3] and some studies were not tested [14, 19]. We have elaborated on the inconsistencies in the literature review section. This demonstrates that more empirical evidence is needed to further illuminate the factors for agile suitability. These factors are essential in ensuring its effective and successful usage.

If people and community are the main concern in agile methods, we believe that different regions will have different ways in practicing agile. However studies that have been reported were commonly confined to only specific companies and environment [20]. Although many studies have been conducted in the usage of software development process, most of these mainly focused on the case in the UK, the US and other parts near to them [7, 11, 16, 21] but not Southeast Asia region. It has been described that Southeast Asia region (where Malaysia is one of the countries) [5] has became the top largest IT offshore services for development nonetheless few studies can be found in that region [5]. This significant fact is another reason for our decision to conduct this study with Malaysia as the case under investigation.

1.1.1 Malaysia The software industry has had an impact on the economic growth in Malaysia. Since 1990, Malaysia’s economic growth has been outstanding with a double GDP per capita achieved1. Realising the important contribution of the software industry to economic growth and how it can help the community, Malaysia has started focusing on it. This is important in order to transform the country into a developed and technology country. To achieve this objective, Multimedia Super Corridor (MSC) was designed in 1996 after the Silicon Valley. Receiving full support from the government, MSC is an effort to support the information communication technology (ICT) sectors in the country. It has become increasingly important as under the new economic drive, Malaysia has focused on transforming the country into a knowledge based society which can be achieved through the deployment of ICT. MSC provides facilities to both local and multinational companies particularly in the field of ICT2. As a result, software development in Malaysia has experienced increasing growth due to the support from the MSC. Currently, looking to the importance of information and communication technology and how the industry helps in the growth of the economy and to the community as a whole, ICT has become one of the key elements under the 10th Malaysia Plan (2011-2015)3. Therefore, in order to adopt good practice in software development and successful deployment within the communication technology industry as a result, the need to understand the software process is critical. Nevertheless, problems found from one study shows that Malaysia is still lacking into the usage of good software processes [22]. Problems in terms of delivering quality products, late delivery and excessive budget have also been identified [22]. The unawareness of good development practice has been identified as the source of the problem of quality in the software development in Malaysia [22]. One investigation described organisational factor, including the people factor, as the main barriers to implementation of Software Process Improvement (SPI) in Malaysia [23]. Although the study was not focused on agile methods, it can however help us to identify some of the factors that are important for software methods adoption; in particular, agile methods. Through understanding the barriers, the factors influencing the adoption can be identified too. In

We assume the problems (such as in quality, delivery, budget, people, organisational factor and having requirements) can be minimised with the application of agile methods. Proven guidelines for, and tested model for software process adoption are indeed critical in ensuring the smooth implementation of software development. The economic contribution of software development to the country and the problems found in the industries has motivated us to choose Malaysia as a location for investigating the adoption factors using agile methods.

1.2 Objectives and Expected Contribution The main objective of this research is to identify factors and barriers to implementation of good software processes that can be used by the practitioners in Malaysia and the surrounding region before they start to introduce agile in their organisations. Organisational, people and technical aspects are our main concern for the adoption. We expect to fill the research gaps within the inconclusive evidence found from previous studies. As this work has progressed, we started to identify the current usage and perception of agile methods in this country. This will help us to understand how they are perceived before identifying related factors and barriers. The contribution of our study is to reduce the chasm within the adoption factors in the country and the region where little study can be found. The knowledge can help practitioners to further understand and compare the suitability of agile methods to their organisation and people. Another contribution is to provide practitioners and software industries in Malaysia with the knowledge and perception of the key factors involved in agile adoption. From this study, we expect to minimise problems identified from software development industries in Malaysia (for example in [22-24]) through the effective implementation of agile methods. All in all, the contribution of this study is to add and validate evidence for factors supporting agile adoption.

1.3 Description of Issues In this doctoral paper, we would like to get the advice on our works which include: 1. 2. 3.

Are the proposed methods approriate for this study? Is it better to use component analysis (such as SPSS) or Structural Equation Modeling (SEM) tool to analyse the results? Which is the best way to combine findings from questionnaire and interviews?

We hope to get the most advice about the issues described above.

2. LITERATURE REVIEW

1

www.pikomps3.com.my/images/idc_paper.pdf

2

www.mscmalaysia.my/topic/12073059198422

3

addition, recent investigation has also discovered that companies in Malaysia do not have a clear methodology against which to illustrate their requirements [24]. From here, we know that identifying the needs of a software process that can help to solve the problems is important as it describes the way requirements are obtained. Agile methods can help in producing what is really needed by the customer and at the same time focus on the delivery of quality software [2].

www.bernama.com/bernama/v5/newsbusiness.php?id=504844 (accessed 18 June 2010)

Agile methods are sets of new approaches for developing software [3] which are based on practitioners’ experience [25]. The intention of the agile manifesto is to solve the issues and problems that were found in software development [26, 27]. Agile practices are adapted from existing techniques and ideas

found in the software development information field, where its usage depends on the suitability and the appropriateness of the team, organisation and project. Unlike previous methods, agile has four main values [25]: Value 1: Individuals and interactions over processes and tools. Value 2: Working software over comprehensive documentation. Value 3: Customer collaboration over contract negotiation. Value 4: Responding to change over following a plan. A systematic review of empirical studies of agile up to 2005 was conducted [28]. Although agile methods are considered suitable for many environments, there is however lack of empirical studies available to validate the suitability [28].

2.1 Perception of Agile Methods In terms of perception, a number of studies have identified the perception of developers [29, 30], customers [20, 31] and students [32, 33] using agile methods. Unfortunately, the studies were mainly carried out in western countries such as Canada, European region [30, 33] and the United States [34]. To our knowledge, very few studies on the perception of agile methods from South East Asia region exist. Mixed results about the benefits agile can deliver were reported in a study [35]. After 2005, there was also an investigation into the perception of agile methods in the Microsoft Corporation worldwide [4]. Although the study referred to [4] was conducted on a worldwide scale, there were no discussions and comparisons in respect to the organisational culture or structure among the organisations located in different countries. In addition, the organisational aspects may have had similar characteristics because the study only focused on the Microsoft organization. Another study also investigated the software practices in five ASEAN countries which included Malaysia and mentioned – ‘while it is true that there have been “global” or “worldwide” surveys of software practices, the respondents in these global surveys were mainly from the U.S, Europe and Japan’ [5]. Similar scenarios can be identified in a number of other studies [36, 37]. Although one [38] have similar approach to our study, however, they focused only on XP method, using university setting and did not include cultural comparison in their investigation. Therefore, our investigation intends to find out what criteria from the perception that will lead to the adoption of agile methods in Malaysia. This can help to provide knowledge about agile methods to the practitioners in Malaysia.

2.2 Adoption of Agile Methods In this section we will discuss the factors that have been found to help in the adoption of agile methods. One study mentioned that agile is suitable only for small and co-located teams; but at the same time provided suggestion for large organisations to adopt Agile Methods [39]. In contrast studies to date have not found a significant correlation between the successful usage of agile and the size of the team [12, 40]. A co-located working area is also considered important for the practice of agile methods. This is because the nature of agile methods requires the members of the team concerned to be located closed to one another for the collaborative way to be practiced. The open plan office is mostly preferred in agile methods [41]. However some researchers [3] fail to support for agile works facility (such as open plan office) and the co-location factor of the team [12, 40].

An important aspect under the organisational factor is to consider the characteristics of projects and team in the organisation. It is also important to understand the features and environment within which the development is taking place. One study stated that these characteristics can also be linked to the organisational factors [11]. As mentioned by the Cockburn and Highsmith [1] good communication is important to implement agile methods. However others [12] found insignificant results from the communication aspect. Agile is only suitable for specific types of software projects [42] while one study found no significant result for the suitability of agile to certain project types [3]. Having identified the factors related to the people and social aspects concerned with the successful usage of agile methods, several studies have started to investigate factors related to the organizational culture [13, 14, 18, 43]. One study found that organisational culture is an important factor in the adoption of agile methods [13]. Eight of the projects were chosen from New Zealand and one was from the United Kingdom [13]. Although New Zealand is closer geographically to the Southeast Asia region, its culture is more similar to that of the UK (western country). Another study found no significant findings in terms of organisational culture with the adoption of agile methods [3]. People are part of the agile ecosystem, which is why their talents, skills, experience and communication have become the primary concerns for the success of the adoption. It is not surprising that many studies on effectiveness and adoption of agile methods focus on people [30]. One study also found the ‘people’ aspect as the key success factor in agile methods [3]. In addition, from its early introduction, the method has been described as a set of approaches that concentrate on people and social aspects [1]. Having described people and the organisational factor, we can see that the success of agile methods is largely due to the social aspects within its environment. However, in one study, technical factors were reported to be the key success factors in the success of agile, and have resulted to be at the top three success factors and significant for agile implementation [3]. The importance of technical aspects found in Tsun [3] lies not in the tools but on the delivery strategy and agile software practice. Nonetheless, both these aspects still depend on the people who practice agile methods. Insignificant results in technical competency have also been found [12]. As described at the beginning, agile methods do not rely on heavy documentation as they depend on the tacit knowledge of the team. Yet people always question how the team can adopt the method if it does not have any documentation relating to the works they are doing. This is one issue that may be questioned by the early adopters of agile methods particularly for a country where less study about the method can be found. Agile methods use documents only when necessary, and these are kept as simple as possible. One study described the creation of knowledge as a factor in the adoption of Extreme Programming, one of the agile methods practices [30]. However it is not an easy task for the team to share and rely only on the knowledge among themselves. One study suggested that the team should also posses the skills and they should be motivated [44]. The capability of the team is critical for the success of agile [3]. On the other hand, different evidence shows that capability, professionalism and experience have acted as a hindrance in the adoption of agile methods [11]. Training [12], management

involvement and access to external resources were among factors identified to have impact on the implementation of agile software methodology [40]. These can be considered as elements of the organisational factor. From the factors of adoption, organisational and people factor are considered important. Conversely a study also found significant result in terms of technical factors [3]. We need to further investigate the relationship of technical aspects to the factors (such as people and organization) of agile methods adoption. In our study, we will focus on the organisational and people factors including the technical aspects. The association between the factors will be investigated.

3. RESEARCH METHODOLOGY Before we identify the factors and barrier to adoption, our initial study aimed to investigate the perception of software practitioners in Malaysia towards agile methods. A quantitative approach using a questionnaire is adopted to understand the background and the perception of practitioners in Malaysia towards software process and agile methods. The broad and conclusive coverage of a questionnaire is the reason we chose this method [45]. Surveys in software engineering “usually poll a set of data from an event that has occurred to determine how the population reacted to a particular method, tool, or technique and it is also help to determine the trends or relationships” [46]. Figure 1 shows the flow of our initial study. In this report, we are concentrating on our questionnaire results; the results from the interviews are not included in this paper.

3.1 Research Setting The research is conducted in Malaysia where limited studies about agile methods can be found. Malaysia is one of the countries included in the Southeast region. It is a country having multinational and multicultural features which are derived from the main ethnic groups as described in the introduction section. The organisations were chosen from three locations in Malaysia; Kuala Lumpur, Selangor and Pulau Pinang. Since Kuala Lumpur is the central part of Malaysia and Selangor is near to Kuala Lumpur, most software companies are located in these two areas [47]. We consider both companies from both sectors; public and private. By taking into account the respondents from those two sectors, therefore Kuala Lumpur and Selangor are considered to be the best choice as most of them are located within these two locations. There was also a similar assumption made by one study [48].

3.2 Participants We performed a multistage (clustering) procedure for the sampling purpose [49]. In relation to this, we first identified the groups and the organisations. The organisations or firms were chosen from the Multimedia Super Corridor (MSC)4 directory which provides information about ICT companies. Within these groups, an individual from each organisation was contacted. Emails with the link to answer the questionnaire were sent to the organisations (identified from the clustering procedure mentioned above). We also asked participants to forward the link for the questionnaire to their colleagues. The questionnaire had different sections for agile and non-agile users as the questions differed for each group.

4

http://www.mscmalaysia.my/topic/Company+Directory

Figure 1. Flow Chart for the Initial Study

3.3 Research Design We need to know the current state of agile methods usage in Malaysia; therefore a questionnaire can serve for this purpose. In addition, the questionnaire has been chosen because of the great distance between the researchers and the location of the respondent. The questionnaire was posted on the web for the duration of two months (October to December 2009). Prior to conducting the study, ethics approval was granted from the School’s ethics committee.

3.3.1 Questionnaire Design The questionnaire has 77 questions and is divided into four parts. Part 1 consists of general knowledge on software development, part 2 asked questions about agile methods, part 3 covers organisational structure and part 4 includes demographic questions. The questionnaire consists of multiple choice questions and Likert scale questions (point 1 to 5). Questions in part 2 were only meant for agile respondents. The questions in part 3 were adopted and modified from another study [50]. From our questionnaire, in order to identify agile users, we asked question whether they know about agile methods (“Do you know about agile methods?”). In addition to that, to confirm that they are really using agile or have an experience in agile, a question asking duration of using the method is asked in the next question (“If yes, how long you have been using agile methods?”). If the answer from this question is ‘none’ therefore the respondent is directed to another set of questions meant for

non-agile users. If the respondent chose the duration (for example: more than 3 years or 2 to 3 years or less than 2 years or just started) therefore the respondent is directed to section specifically containing questions about agile methods. From here we can differentiate the answer given from the two groups; agile and non-agile users. Our pilot study has received a total of 79 respondents from different organizations. Since English and Malay are the standard language used in Malaysia [51-53], we designed our questionnaire in these two languages. It is important for the respondents to properly understand the meaning of the questions.

3.4 Research Questions In this paper we focus on our initial work; thus understanding the perception and organisational aspects as understood by the practitioners is our main concern in this paper. The perception of agile methods, agile values and the organisational structure are our dependent variables. The main research question is: What are the factors that can bring about the adoption or rejection of agile methods in Malaysia? As this is a series of PhD works, the pilot study was conducted to answer part of the following questions: 1) 2) 3) 4) 5) 6)

What is the perception of practitioners in Malaysia towards agile methods? Are practitioners in Malaysia aware about agile methods? Do practitioners in Malaysia agree with the four values of agile methods? Is there any difference for group of aware and not aware of agile methods? Are technical aspects important? Are organisational aspects important?

3.4.1 Hypotheses In this pilot study, we only discuss awareness of agile methods, organisational aspects and the resepondents’ perceptions of agile methods. Thus, our hypotheses include:

3.5 Results from the Pilot Study In our analysis, we used a statistical package, SPSS version 17. Our results show that more than 50 % of respondents were not aware about agile methods. We found a significant difference between group of awareness (the group are those aware and not aware about agile methods) and language they use. A chi square test for independence indicated there is an association between language used and the awareness of agile 2 (1, n=57) = 4.347, p=0.037 < 0.05. From the results, we found 46.4% of respondents who answered in English were aware about agile methods while 53.6% were not. Of those answering in Malay, only 17.2% were aware about agile methods but 82.8% were not. In total, the percentage of the samples as a whole on the awareness of agile is 31.6% and on the other hand, 68.4% were not aware. In terms of how agile awareness contributes to the perception of agile values (the four values mentioned in section 2), multivariate analysis was performed to investigate whether there is any significant difference that exists between group of awareness (aware about agile and not aware about agile) and their overall values of agile. Our finding shows a statistically significant difference between those aware and those not aware in terms of their overall agile values (Pillai Trace =0.188, p=0.026