Refactoring Planning and Practice in Agile Software Development: An Empirical Study 1
Jie Chen1,2, Junchao Xiao1,3, Qing Wang1,3, Leon J. Osterweil4, Mingshu Li1,3 Laboratory for Internet Software Technologies, Institute of Software, Chinese Academy of Sciences, Beijing, China 2 University of Chinese Academy of Sciences, Beijing, China 3 State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing, China 4 Department of Computer Science University of Massachusetts, Amherst, MA, USA
{chenjie,xiaojunchao,wq}@itechs.iscas.ac.cn,
[email protected],
[email protected]
ABSTRACT
opposed to only the final delivery of a completed product [1]. The continual reworking of partial implementations often causes software to become more complex and brittle, and to deviate from its original design, lowering the quality of the software, making it harder to adapt as needed in response to market forces. Agile methods focus on breaking larger tasks into small increments that typically last from one to four weeks. But that may cause them to lose sufficient focus on long-term planning. An agile development increment typically starts with a planning meeting and ends with a review and retrospective meeting. The planning meeting marks the start of each development iteration. During this meeting, the objectives of the iteration are discussed and established, and are broken down into a set of tasks and time estimations that are provided to the task owners [2] [3].
Agile software engineering increasingly seeks to incorporate design modification and continuous refactoring in order to maintain code quality even in highly dynamic environments. However, there does not currently appear to be an industry-wide consensus on how to do this and research in this area expresses conflicting opinions. This paper presents an empirical study based upon an industry survey aimed at understanding the different ways that refactoring is thought of by the different people carrying out different roles in agile processes and how these different people weigh the importance of refactoring versus other kinds of tasks in the process. The study found good support for the importance of refactoring, but most respondents agreed that deferred refactoring impacts the agility of their process. Thus there was no universally agreedupon strategy for planning refactoring. The survey findings also indicated that different roles have different perspectives on the different kinds of tasks in an agile process although all seem to want to increase the priority given to refactoring during planning for the iterations in agile development. Analysis of the survey raised many interesting questions suggesting the need for a considerable amount of future research.
In general, iteration planning has to address multiple goals, such as: creating new features, fixing bugs, and making code more extensible, flexible, etc. It is inevitable, however, that some of these diverse goals and priorities may be unrealistic, and some may come into conflict with others. Our focus in this paper is on the conflicts that are set up by the need to perform refactoring, what planning approaches may help address these conflicts. For example, business stakeholders may have concerns about “big bang” software development and may therefore advocate scheduling numerous bug-fixing tasks aimed at delivering incremental product releases. But mandating tight scheduling constraints, may put off or prevent major redesigns, thus also putting off or preventing the creation of a reliable, stable, product with the flexibility needed to respond to rapid market changes [4]. All too often, when decision makers pursue quick responses to the market, needed refactoring is deferred, code quality is reduced, and the refactoring that is eventually needed becomes more difficult and costly [5]. The emphasis of this approach is to focus on near-term goals, while still attempting not to ignore longerterm issues. Thus, for example, the product owner may wish to focus on short-term goal to increase speed of software product [6].
Categories and Subject Descriptors K.6.3 [Management of Computing and Information Systems]: Software Management
General Terms Management
Keywords Refactoring; Project Management; Agile; Iteration Planning
1. INTRODUCTION Accommodation of change is one of the key requirements in software planning and project management. Agile software development (ASD) is a well-established approach in software engineering that is aimed at supporting the rapid change needed by quick-to-market applications. ASD advocates principles such as frequent delivery of partially implemented working systems, as
ASD processes typically also incorporate simple design modification and continuous refactoring as approaches to maintaining code quality even in highly dynamic environments. Refactoring, in particular, is key in adding flexibility and extensibility while also introducing new functionality [22]. But how best to integrate refactoring into the incremental plans in ASD is less obvious. Indeed, there is conflicting evidence about the effectiveness of various strategies for planning and performing refactoring. One approach pursued in industry is to set aside an extended period of time devoted specifically to refactoring. On the other hand eXtreme Programming (XP) advocates refactoring throughout the entire project life cycle, interspersed with other routine development activities, such as adding new functionality
Permission to make digital or hard copies of all or part of this work for personal or Permission togranted make digital copies all or of this for classroom use is without or feehard provided thatof copies arepart not made or work distributed personal classroom use is granted without feethis provided that are for profit or or commercial advantage and that copies bear notice and thecopies full citation onnot the made first page. components this work owned by others and than ACM or Copyrights distributedforfor profit orofcommercial advantage that must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, copies bear this notice and the full citation on the first page. To copy to post on servers or to redistribute to lists, requires prior specific permission and/or a otherwise, or republish, to post on servers or to redistribute to lists, fee. Request permissions from
[email protected].
requires prior specific permission and/or a fee. ICSSP’14, May 26–28, 2014, Nanjing, China ICSSP’14, May 26-28, 2014, Nanjing, China. Copyright 2014 ACM 978-1-4503-2754-1/14/05...$15.00 Copyright 2014 ACM 978-1-4503-2754-1/14/05... $15.00. http://dx.doi.org/10.1145/2600821.2600829
55
and fixing bugs. XP proponents claim that refactoring both saves development cost and also improves software quality [2]. But, on the other hand, there is some research that shows that this interspersed refactoring may often be ineffective and error-prone [8], especially when it is undertaken without sufficient time and resources to run the regression tests [8] that are needed to ensure the correctness of refactoring edits [9].
RQ 4: How is refactoring actually practiced in the process?
The project described in this paper undertook some interviews aimed at gaining better understanding of refactoring planning and execution in iterative processes, and identifying the strengths and weaknesses of various approaches to planning and performing refactoring. These interviews suggest that projects that have a business goal to deliver capabilities rapidly must deal with a natural tension between getting functionality out quickly and maintaining a high quality code base. Instead of undertaking proactive refactoring aimed at preserving code flexibility and quality, some teams prefer to wait for a business-driven opportunity to try to work on refactoring, or simply to wait until team performance drops to a point that is unacceptable to the business side.
2. BACKGROUND AND RELATED WORK
The rest of the paper is organized as follows: in section 2, we review related work. In section 3, we describe our research method. In section 4, we summarize the statistical analyses of our survey results. We discuss these results in section 5. And finally we present our overall research findings in section 6 and conclude the paper in section 7. ASD is being seen increasingly widely as an effective approach to meeting the needs of rapid response to market forces. Accordingly much research is focusing on how to plan the iterations that are a central feature of ASD. A key theme of much of this research seems to be maximizing the value of user stories (e.g. [17][18]). Recently, however, there has been a growing emphasis on also addressing issues of speed and stability in ASD processes as well. Bachmann [21] observes that today’s organizations must strive for both agility and sound design. Three tactics are presented to support reasoning about architectural tradeoffs between agility and stability. Dyba [19] found that lack of focus on architecture in ASD could lead to suboptimal design decisions. Similarly, losing sight of the big picture is also on the list of the top six concerns with agile development [20]. However, these papers do not consider refactoring as part of agile processes. Recently, Bellomo [4] explored how agile projects deal with the pressure to rapidly deliver high-value capabilities and found that incorporating refactoring with other development practices can be useful in being able to respond quickly to unanticipated stability problems.
It has been known for decades that techniques such as code refactoring can be used to improve quality and enhance project agility [10][11]. However, although the costs of quality optimizing activities such as refactoring are significant and immediate [12], their benefits are often hard to see, except perhaps in the long term [13] [14] [15]. Because of this, it can be difficult for decision makers to justify the high immediate cost of refactoring by the difficult-to-quantify potential cost savings from a proposed refactoring. More specifically, it is hard for a decision maker to know, with confidence, by how much the quality of a project declines without refactoring, and how much more it will cost to maintain and extend project quality through refactoring. Some previous work has been aimed at supporting the identification of so-called bad smells that might be used to suggest when to take program refactoring into consideration as a quality enhancement method [5]. And there are several theoretical and practical results that could be useful to decision makers attempting to do cost-benefits analysis of refactoring [16]. But this existing work does not seem to us take sufficiently strongly into account the need to consider the different ways that refactoring is regarded by the different people carrying out different roles in an agile process and the different ways in which these different people weigh the importance of refactoring versus other kinds of tasks in the process.
There have also been a number of empirical studies of refactoring. Moser [23] conducted a case study of the use of ASD in an industrial setting and found that refactoring enhances quality and reusability related metrics. Xing and Stroulia found that 70% of structural changes in Eclipse’s evolution history are due to refactoring [24]. Dig et al. studied the role of refactoring in API evolution, and found that 80% of the changes that broke client applications were due to API-level refactoring [25]. For more detail, Negara applied a continuous change analysis algorithm to the code evolution data from 23 developers working in their natural environment for 1,520 hours, and found that up to 42% of some kinds of refactoring activities were not recorded as such in the Version Control System [8]. Murphy-Hill et al. found that developers often interleave refactoring edits with other behaviormodifying edits and apply refactoring manually without using automated refactoring engines [29]. There have been a number of studies of the impact of refactoring. Ratzinger et al.[26] found that as the number of refactoring edits increases the number of defects decreases. On the other hand, it has been found that a high ratio of refactoring edits is often followed by an increasing ratio of bug reports [24][27] and that incomplete or incorrect refactoring may cause bugs [28][22]. In other work Reddy describes a refactoring ripple effect using dependency oriented complexity metrics [32].
In order to explore these issues, we have collected data through industry surveys. Specifically, we have decomposed our main research question as follows: RQ 1: How does a development team plan for each kind of task during the process of planning for a development iteration? RQ 2: In what ways do those who have different roles in software development have different perspectives and preferences for the different kinds of tasks in an agile process?
A number of other studies have addressed the problem of identifying when refactoring should be undertaken. Simon provides a distance based metric to identify where to apply which kind of refactoring [30]. Marinescu provides a list of metric based detection strategies for capturing object oriented design flaws that suggest the need to refactor [31].
RQ 2.1: Do different roles have different levels of understanding of the workloads required to perform each kind of task? RQ 2.2: Do different roles place different priorities on each kind of task during the planning of a development iteration? RQ 2.3: Do different roles have different perceptions of the priorities that other roles place upon each kind of task?
None of these papers, however, considers sufficiently carefully the different ways in which refactoring is viewed by different
RQ 3: How does refactoring affect the process?
56
roles in agile processes and how planning and performance of refactoring should be balanced with other development tasks.
required that, 1) the respondent had been a part of a team that had adopted ASD practices for at least one year, and 2) the respondent had knowledge about development and refactoring.
3. RESEARCH METHOD We addressed the goal of exploring and providing insight into planning for refactoring in agile software development by undertaking a study of how iteration planning is practiced in industry. Broadly, our research was carried out in three phases: Phase 1: Designing a survey, Phase 2: Administering the survey, Phase 3: Analyzing the data.
The survey was administered using a web-based questionnaire, which can be found at http://www.wenjuan.com/s/eA3Qn2/. We received 126 responses, of which 105 were eligible responses. The data that were gathered were then statistically analyzed using data analysis techniques as described below.
3.1 Designing the Survey
In this section, we first summarize the responses from respondents. Then, in the following subsections, qualitative analysis is done to address the research questions in section 1.
4. SURVEY RESULTS
As a first step, we performed a small-scale pilot study to solidify our understandings of the central research issues and define the goals of the survey. The survey was designed to focus on: •
Planning practice: Exploring each of the different types of tasks considered during iteration planning, and determining if different roles have different insights into those types of tasks.
•
Refactoring practice: Exploring how refactoring is currently practiced within industry, how projects are impacted by deferred refactoring, and how different project roles differ in their views about refactoring practice.
4.1 Backgrounds of Respondents and Teams First we gathered descriptive statistics to summarize the backgrounds of the respondents. The questionnaire requested background data on both the respondent and the agile team to which he/she belonged. This data is summarized in Table 1. The respondents came from all over the world, suggesting good international coverage, but the majority of the respondents came from either China (35.2%) or North America (30.5%). Around 61.9% of the respondents came from large companies (1000+ employees), which we expected to be well organized and to do a good job of process management. Most of the responses (about 62.90%) came from respondents who belong to teams having 1-10 employees, and 76.1% of the respondents were working on projects that were decomposed into short iterations (shorter than one month), making them suitable for the application of agile development principles. As indicated in Table 1, most of the products were owned by the team instead of the by customers or others (e.g. non-team members of the team’s organization). We allowed respondents to indicate that they have more than one role, but categorized the respondents by the role that they claimed as their main responsibility in their project. There was good response from both developers (50.5%) and non-developers, experienced (47.6%) and non-experienced respondents.
The questionnaire comprised four parts: Background information. We gathered information about the respondents, including both personal information and information about the agile team to which the respondent belonged. ASD team planning practices. We presented closed-ended questions [33] asking respondents to describe: 1) the percentage of time a developer spends on each kind of task in a typical iteration, 2) the authority that each role has in iteration planning, 3) the priority that each kind of task is given in iteration planning, and 4) the strategy used when planning each kind of task. ASD team Refactoring practices. We presented questions indicating the level of agreement about various experiences in performing refactoring. Answers were to be given on a Likert Scale ranging from strongly disagree (1) to strongly agree (5).
We were particularly interested in acquiring more detailed information about developers as they are the main population of our survey and they are responsible for creating the product and executing iteration planning. Approximately 39.6% of developers were more experienced (5+ years) and 67.9% of respondents have been working in an agile development context for more than one year. We also invited participation from respondents whose roles were related to decision making during iteration planning, being particularly interested in comparing their opinions to those of the developers. These roles included project manager (responsible for ensuring the successful completion of work-in-progress), product manager (represents customers’ interests), architect (sets the technical direction of the overall system), tester (charged with preventing defects from entering the system), UED (responsible for assuring system usability), operator (responsible for keeping products running during production), and agile expert (the scrum master/agile coach). All of the agile experts and operators considered themselves experienced, and most of the project managers (78.9%) and architects (71.4%) also considered themselves to be experienced. But only 33.3% of the UED, 40% of the testers and 50% of the product managers considered themselves experienced.
Personal opinions. We used open-ended [33] questions to allow respondents to provide additional information, insights and perspectives not captured by the previous questions.
3.2 Administering the Survey The questionnaire was pre-tested for validity to clarify ambiguities, identify readability problems, and establish an effective ordering of the questions. Pre-testing was done by ten well-qualified software development professionals who had different experience levels and work functions and worked in both local and global corporations in China. We collected data by distributing the questionnaire online to practitioners from different IT companies. We recruited our participants by distributing our survey with an email invitation and by making it available to professionals through online social networks such as LinkedIn. Special interest groups within those networks, especially those that deal with agile and software engineering, provide a platform for the discussion of relevant topics by professionals, and so seemed to be logical places in which to seek particularly informed responses to our questionnaire. We did not restrict our surveying to any particular industry type, sector, or size. The only requirement we had was that the respondent had practiced ASD. More specifically, we
In the following subsections, we analyze how the responses that we received shed light upon our stated research questions.
57
Table 1. Characteristics of the survey sample Individual Experience 5 years
the task to be done; S3. Plan that a needed task be done as part of another task; S4. Make a simple plan for the critical tasks; S5. Make a specific detailed plan for all tasks.
Team Team Size 12.40% 20.00% 20.00% 47.60%
Experience in Agile 5 years 16.20% Role Project Manager 18.10% Product Manager 7.60% Architect 20.00% Development Engineer 50.50% Test Engineer 21.00% UED/Operator 5.80% Expert 9.50% Organization Size 1000 61.90%
1-5 6-10 11-20 21-40 >40
26.70% 36.20% 16.20% 6.70% 12.40%
Two more answer options we also provided: S6. I don’t know, and S7. Others. Table 2 summarizes the results as percentages of responses. Multiple-selection was allowed for this question. Our survey showed that a large proportion (around 98.1%) of the respondents said that tasks relating to implementing new features are carefully planned (the respondents selected either S4 or S5).
Product Owner Customer 26.30% Team 51.50% Others 21.20% Location China North America South America Europe Oceania Africa Asia (besides China) Iteration Length One week 2-4 weeks 1-3 months 3-6 months 6-12 months More than one year Uncertain
And indeed none of the respondents selected the “I don’t know” choice for NF. Among the 1.9% who selected strategy 1 for NF, one was from an organization with less 50 employees and the respondent indicated that they just do the best they can. Others selected other strategies in addition to S1, suggesting that S1 is chosen depending upon the nature of the feature. Similarly, the great majority (about 68.6%) of respondents indicated that bug fixing was also carefully planned (the respondents selected either S4 or S5), further reinforcing the view that there is uniformly careful attention paid to planning for these two types of tasks, and corroborating conventional wisdom that creating new features and fixing bugs are regarded as the most important development tasks.
35.20% 30.50% 1.00% 14.30% 2.90% 1.90% 14.30% 17.10% 59.00% 14.30% 1.90% 1.90% 1.00% 4.80%
The responses addressing the type of planning done for both kinds of refactoring were far more evenly divided. Smaller proportions of respondents claimed that they have applied careful planning for either root-canal (42.6%) or floss refactoring (43.8%) tasks. Further, for both kinds of refactoring, about 40% of the respondents claimed that they do not do careful planning at all for this kind of task (they selected either S1 or S2) and about 10% of the respondents said that they had no idea. In addition, 6.6% of the respondents selected “Other strategy” and one respondent noted that they do root-canal refactoring only upon a request from either the customer or top company management. Another respondent observed that the team does not plan for root-canal refactoring as a separate task but deals with it as a new feature aimed at increasing customer satisfaction with product performance. The lack of focus on root canal refactoring was further reinforced by noting that the total percentage of approaches to doing RR tasks aggregated 113.2% (recall that multiple responses were allowed), indicating a relative lack of consensus on how to address these kinds of tasks.
Table 2. Planning strategies for each kind of tasks S1 S2 S3 S4 S5 S6 S7 Total
NF 1.90% 2.90% 0.00% 44.80% 54.30% 0.00% 0.00% 103.90%
BF 6.70% 21.90% 11.40% 45.70% 22.90% 1.00% 0.00% 109.60%
FR 21.90% 19.00% 13.30% 27.60% 16.20% 7.60% 0.00% 105.60%
RR 16.20% 16.20% 25.00% 17.60% 25.00% 11.80% 1.50% 113.20%
4.2 Planning Strategies RQ1: How does a development team plan for each kind of task during the process of planning for a development iteration?
We summarize what we infer about research question RQ1 from our survey as follows: Agile teams pay close attention during iteration planning to tasks involved in creating new features and bug fixing, but do not seem to have a unified view of how to approach planning for refactoring.
In an attempt to understand various planning strategies, the questionnaire asked whether each of the following four kinds of development process tasks is considered during iteration planning: •
NF: tasks aimed at creating new features
•
BF: tasks aimed at fixing bugs
•
FR: tasks to support floss refactoring
•
RR: tasks to support root-canal refactoring
4.3 Different Preferences and Awarenesses RQ2: In what ways do those who have different roles in software development have different perspectives and preferences for the different kinds of tasks in an agile process?
4.3.1 Awareness of Workload
The two kinds of refactoring were so-named previously in [29]. In floss refactoring, the programmer intersperses refactoring with other kinds of program changes to keep code healthy. Root-canal refactoring, in contrast, is used to correct deteriorated code and is a protracted process consisting exclusively of refactoring.
RQ2.1: Do different roles have different levels of understanding of the workloads required to perform each kind of task? To investigate if the respondent’s role impacts their perception of development workload, we use the Chi-Square Goodness-of-Fit Test. For this study, we divided the respondents into two groups: Developers (50.5%) and Non-developers (49.5%). Respondents from both groups responded to the question “what proportion of workload does a development engineer spend for each kind of
We also asked which of the following planning strategies pursued: S1. Do not make a plan to accomplish a task, but leave a needed task to be done when someone has spare time; S2. Do not make a plan to accomplish a task, but leave some slack time to allow for
58
Table 3. Workload percentage for each kind of tasks
task”. And for each kind of task, we provided five answers: 50%. For each kind of task, we applied the Chi-Square Goodness-of-Fit Test to examine the hypothesis that proportions of the selections made by the nondevelopers match the proportions of selections made by the developers.
50%. This suggests that developers actually spend a significant amount of time doing work that is other than creating new features, but may do so in ways that may be unobservable or invisible to other roles.
0.0% 0.0% 3.8% 1.4 0.0% -1.4 15.4% -0.5 18.9% 0.5 13.5% -1.2 25.0% 1.2
1%10% 5.8% -1.3 13.2% 1.3 17.3% -2 34.0% 2 36.5% 0.1 35.8% -0.1 48.6% 0.4 44.4% -0.4
11%30% 25.0% 2.1 9.4% -2.1 46.2% 1.1 35.8% -1.1 36.5% 1.6 22.6% -1.6 27.0% 0.8 19.4% -0.8
31%50% 23.1% -3.3 54.7% 3.3 26.9% 0.5 22.6% -0.5 3.8% -2.2 17.0% 2.2 2.7% -1.1 8.3% 1.1
>50% 46.2% 2.5 22.6% -2.5 5.8% -0.4 7.5% 0.4 1.9% 0 1.9% 0 0.0% 0 0.0% 0
I don't know 0.0% 0.0% 0.0% 0.0% 5.8% 0.5 3.8% -0.5 8.1% 1 2.8% -1
Sig.
0.001**
0.209
0.265
0.437
** Correlation is significant at the 0.01 level (2-tailed), Critical value=2.58. * Correlation is significant at the 0.05 level (2-tailed), Critical value=1.96. AR represent: Adjusted Residual
The respondents rated the priority for each kind of planning task on a scale of 0 to 5 (0 means the task is not considered, 5 means the task if given the highest priority). On average, NF and BF tasks have relatively higher priorities. Further, the results reveal significant evidence of negative skewing and excess kurtosis for tasks involved in creating new features, indicating these tasks get the highest priority. In contrast, the priorities for tasks involved in both kinds of refactoring are close to the normal distribution with a mean value around 2.25, suggesting notably lower priority. In addition, we used correlation analysis to help us understand the relationship between role authority in planning and the priority for each kind of task. We used correlation analysis to help us understand which relationships were positively correlated, which were negatively correlated, and the strength of these relationships. We computed two sets of Spearmans’s coefficients of correlation and significance values at levels 0.05 and 0.01. Table 5 shows the summarized correlation results for items with non-missing values.
Results obtained for other kinds of tasks (BF, FR, RR), provide supporting evidence for the above observation. For bug fixing tasks, 46.2% of the non-developers responded that the workload is 11-30% while the developers selected the three choices: 1%-10%, 11%-30%, 31%-50% equally frequently. About 18.9% of the developers said that they spend more than 30% of their workload on floss refactoring while only 5.7% of the non-developers thought that was the level of workload entailed. There was a relative consensus about the workload for root-canal refactoring. Most respondents from both groups claimed 1%-10% of the workload. There was a small difference between the two groups in that more non-developers selected a workload of more than 10% and more developers selected a workload of less than 1%. For both kinds of refactoring, more non-developers answered, “I don’t know”.
The data shows a significantly positive correlation between the authority in iteration planning of developers and testers, meaning that the authority of the developers and testers rises or falls pretty much in unison within a single iteration planning. Similarly, the authority of the project manager is positively correlated with both the product manager and testers. We also found a significantly positive correlation between the priorities of the two kinds of refactoring tasks, and also between bug fixing priority and floss refactoring priority. However, we found a negative correlation between the tasks involved in creating new features and tasks involved in both kinds of refactoring, and significant negative correlation between NF tasks and RR tasks. These results corroborate our prior intuition that when a team is focused on adding new features, refactoring tasks are reduced to low priority.
We summarize what we infer about research question RQ2.1 from our survey as follows: Most non-developers thought more than 50% of the development workload is spent creating new features in each iteration, while most developers thought only 3150% of their workload was spent creating new features. This suggests that developers spend more (unplanned) time and effort on tasks such as refactoring than is realized by non-developers.
4.3.2 Practice of Priority RQ 2.2: Do different roles place different priorities on each kind of task during the planning of a development iteration? The questionnaire categorized respondents into four roles in iteration planning as shown in Table 4. An assessment of the authority of each role in iteration planning was rated on a scale of 0 to 5, where 0 means the role does not participate at all in planning decisions, while 5 means that the role has the ultimate authority for all decisions in iteration planning. On average, the product manager had the highest authority level among the four roles with a mean value of 3.65. This was followed closely by the project manager with a mean value of 3.50. The distribution of the authority for project manager and product manager was negatively skewed meaning that most of them had high levels of authority. At the opposite end, testers had the lowest authority level.
It is inevitable that people having different roles will have different responsibilities and thus that their goals may be different. Accordingly, we wanted to explore if people having different roles prefer to see different kinds of tasks pursued. It is easy to see that the authority of product manager is statistically significantly correlated with the priority of NF in iteration planning. This supports the hypothesis that when the product manager has higher decision-making authority the priority for NF is correspondingly higher. Similarly, testers and project managers place a higher priority on bug fixing, which also reflects their responsibilities.
59
Table 4. Summarized descriptive statistics Role
Label Mean Std. Dev Skewness
Project Manager Product Manager Developer Engineer Test Engineer
R1 R2 R3 R4
Task
3.50 3.65 3.45 2.49
1.527 1.532 1.083 1.370
-0.835 -1.120 -0.418 -0.191
Label Mean Std. Dev Skewness
Creating New Feature Bug Fixing Floss Refactoring Root-canal Refactoring
NF BF FR RR
4.10 3.80 2.24 2.26
0.950 1.152 1.179 1.390
-1.164 -0.642 -0.124 -0.124
The figure shows that all respondents wanted to attach a higher priority to both kinds of refactoring. Respondents wanted to see the priority of root-canal refactoring increased by more than 10%. Product managers seemed content with the exiting priority given to floss refactoring, while experts (scrum masters/agile coaches) wanted to see a substantial increase in this priority. It seems to make sense that, because floss refactoring is usually not required for implementation of a new feature, the product manager would see little value in it. Architect, testers and experts all wanted a higher priority for bug fixing, but the other roles seemed to have little desire for any changes in priority. The most unexpected finding was that product managers seem to want a lower priority placed on creating new features. This may be because product managers are as concerned about product quality as about meeting the customers’ demands for new features. In order to explore these hypotheses, we used the Wilcoxon signed-rank test, a nonparametric statistical hypothesis test, to determine if the differences just noted are significant. This test is used to compare two related samples, matched samples, or repeated measurements on a single sample to assess whether their populations mean ranks differ. The hypothesis we used was: The median of differences between real priority and desired priority is zero.
Kurto sis -0.307 0.331 0.061 -0.565 Kurto sis 1.245 -0.339 -0.113 -0.797
Table 5. Summarized Spearman correlations table R1 R1 R2 R3 R4 NF BF FR RR
R2
R3
R4
NF
BF
FR
RR
1 0.269** 0.132 0.065 0.140 0.071
1 0.173 1 0.217* 0.023 1 0.100 -0.123 0.201* 1 0.013 -0.235* 0.136 0.589**
1 0.246* -0.073 0.213* 0.082 0.315** 0.050 0.093
1 0.093 0.072 0.283* 0.134 0.030 0.128
The Wilcoxon signed rank test for pairs of priorities (desired priority versus the actual priority used by the team), rejects the hypothesis for floss refactoring tasks (with p=0.00
