Decision Support for Extreme Programming Introduction and Practice

Decision Support for Extreme Programming Introduction and Practice Selection Daniel Karlström and Per Runeson Dept. Communication Systems, Lund University, Box 118, SE-221 00 Lund, Sweden

[Daniel.Karlstrom, Per.Runeson]@telecom.lth.se

ABSTRACT This paper presents an investigation concerning the introduction of Extreme Programming (XP) in software development organisations. More specifically the concept of using a decision support method known as the Analytical Hierarchy Process (AHP) is evaluated by a group of students and a group of developers and the outcome is compared to experiences from an XP case study. The results provide an indication that different practices are thought to be easier and more effective to implement in the two groups. A company considering implementing only a few practices can use this as help for deciding which practices to implement. Companies introducing all practices can use the results of this kind of method to see where more attention might be needed after or during the introduction of XP.

1

INTRODUCTION

Extreme programming (XP) [1,2] has come into focus recently as an agile approach to software development. Extreme programming consists of a set of practices, which can be implemented separately or in combination. When introducing XP, it is in many circumstances impossible to start with all the practices involved. This paper discusses different strategies for introducing XP in various organisations and how to support the strategy decision. Software process improvement in general suffer from that there are different viewpoints of which process is used, and which process should be used. Development processes are documented in official documents, but as they are interpreted differently by different individuals, different versions appear depending on the perception of the process. Bandinelli et al. describe these different perceptions of the process and name these as the official, perceived and desired process [3]. The viewpoints are typically different depending upon the individual’s role in the organisation [4]. The same situation is expected with respect to opinions on the different practices in XP.

In order to support the strategy decision regarding XP introduction, a decision support methodology called the Analytical Hierarchy Process (AHP) [5] is used in order to investigate the issues involved in selecting XP practices. A case study is performed with two different groups prioritising XP practices. The outcome is validated by comparing it to experiences gained from a case study of introducing XP [6]. The paper is structured as follows. In Section 2, the practices of XP and the possible strategies for introducing these are presented. Section 3 presents the methodology for the empirical study, while Section 4 presents the outcome of the investigation. Section 5 interprets the data and finally Section 6 presents a summary.

2

XP INTRODUCTION STRATEGIES

Two distinct strategies are possible when introducing XP into an organisation [1]. Either (1) all practices are introduced at once or (2) a fewer number practices are selected for introduction. In the first case, the methods described in this paper can be used to investigate where more introduction support might be needed to avoid difficulties, or where the XP framework might need tailoring to suit the particular organisation in the future. In the second case, the method might be used to select the practice that is thought to be the most effective. An introduction of the complete XP framework in a large organisation provides greater challenges than in a small one. Consideration needs to be taken to differences throughout the organisation, communication between different groups in the organisation and also differences in attitudes towards XP or a change in methodology at all. It is common to limit initial introduction to a pilot group to study the effects before a total introduction is performed in a large organisation. Before a change in methodology, the use of decision support tools can give help in making crucial decisions such as full or partial introduction, or which practices to implement.

2.1

XP practices

XP is composed of a few fundamental values, principles and activities, which are implemented by 12 practices. The fundamental values are communication, simplicity, feedback and courage. The fundamental principles are: rapid feedback, assume simplicity, incremental change, embracing change and quality work. The basic activities are coding, testing, listening and designing.

The 12 practices that are intended to realise all this are described in the following list [1]. • Planning Game Quickly determine the scope of the next release by combining business priorities and technical estimates. As reality overtakes the plan, update the plan. • Small Releases Put a simple system into production quickly, and then release new versions on a very short cycle. • Metaphor Guide all development with a simple shared story of how the whole system works. • Simple Design The system should be designed as simply as possible at any given moment. Extra complexity is removed as soon as it is discovered. • Testing Programmers continually write unit tests, which must run flawlessly for development to continue. Customers write tests demonstrating that features are finished. • Refactoring Programmers restructure the system without changing its behaviour to remove duplication, improve communication, simplify, or add flexibility. • Pair Programming All production code, i.e. code that is actually used in the final product, is written with two programmers at one machine. • Collective Ownership Anyone can change any code anywhere at any time. • Continuous Integration Integrate and build the system many times a day, every time a task is implemented. • 40-hour Week Work no more than 40 hours a week as a rule. Never work overtime a second week in a row. • On-site Customer Include a real, live user on the team, available full-time to answer questions. • Coding Standard Programmers write all code in accordance with a set of rules emphasizing communication through code. The practices can be introduced independently or as a whole depending on the situation in the development organisation. The practices are intended as a starting point for a development team. The team should start using the practices as they are described in XP and then through continuously adapting and evolving the methodology used, optimise them towards the team’s own preferred method of working under the current circumstances.

2.2

XP roles

An XP project works best if certain roles are assigned to the team members so that they each have different responsibilities regarding the previously described practices. The roles do not necessarily need to represent individual persons. One role can be assumed by several people and conversely one person can assume several roles if need be. The roles are briefly described in the following list [1].

• Programmer The programmer is at the heart of XP. The programmer creates the product based on the story cards written by the customer. • Customer The customer provides the functionality to be implemented in the form of story cards. The customer also creates, sometimes with the help of the tester, functional tests to ensure that the product does what it is supposed to do. • Tester The tester role in XP is focused towards the customer to ensure that the desired functionality is implemented. The tester runs the tests regularly and broadcasts the results suitably. • Tracker The tracker role is to keep an eye on the estimates and compare these to the performance of the team. The tracker should keep an eye out for the big picture and remind the team, without disrupting it, when they are off track. • Coach The coach’s role in XP is to be responsible for the process as a whole, bringing deviations to the team’s notice and providing process solutions. • Boss The boss’s role in XP is basically to provide for the needs of the team to ensure it get the job done.

3 3.1

METHODOLOGY Decision support method

The decision support method is basically composed of four steps:

A: Present the XP practices to the subjects. B: Allow the subjects to rate the factors anonymously by pairwise comparisons, based on their personal viewpoints. C: Analyse the data using AHP. Aggregate and analyse the rating results and identify discrepancies. D: Strategy decision. Decide strategy based on the outcome of the analysis The rating of the practices should be performed by a suitable sample of all people involved in the development organisation that are active in the development. In a small group, a complete sample can be used, but in a large organisation this would not be practically possible. In such a case, the sample should be designed to ensure that the results are affected as little as possible by the sampling. The method used for rating the factors is the Analytic Hierarchy Process (AHP). The AHP is a method for evaluating alternatives in a choice situation [5]. It uses pairwise comparisons and measures each alternative’s relative contribution to the rating. Each subjects’ individual set of comparisons is put into a judgement matrix. From this matrix the relative weights for the compared items can be calculated for each individual. These relative weights are known as priorities. The process also gives the possibility of calculating how consistent the performed ratings are for each individual

as the pair-wise comparisons imply that the alternatives are indirectly compared several times. This was one of the main reasons for selecting the method. The AHP process has been implemented for applications outside the process improvement domain. Karlsson et al. [7], for example, use the process to select between customer requirements. This method of comparing requirements has been further developed into a commercial tool developed by FocalPoint AB [8]. The XP practices are rated with respect to their expected effect in the organisation and with respect to their expected ease of introduction. The effect is of primary interest when selecting a subset of practices while the ease is of primary interest when preparing for an introduction of all the practices. However, the selection of a subset will always be a trade-off between ease and effect. These factors are used in a general sense, providing a simple concept for the rating of the practices rather than being based on exact definitions. It is thought that the members of the project would capture the group’s characteristics and needs and rate the factors according to this. If certain other more specific factors have been identified as important, these factors could be added adjacent to ease and effect in order to further investigate the perceptions of the group and provide further information on which to base decisions. The outcome of the rating is a vector of relative priorities for each rated item, which in turn is the input to the analysis step. When using the AHP method with several subjects, there are some options to be chosen among for the aggregation of the rating results. When aggregating results from an AHP comparison process we must decide whether we wish to aggregate the judgements of each individual or the priorities calculated for each individual. This is referred to as aggregation of individual judgements (AIJ) and aggregation of individual priorities (AIP) respectively. The decision is determined by whether the group is assumed to act as a unit or as separate individuals [9]. Secondly it must be decided whether to use the arithmetic or geometric mean for the aggregation. For AIJ the geometric mean must be used. For AIP any of the two may be used, but the arithmetic mean has the advantage of being comparable to the original values. We have hence chosen to use AIP aggregation with arithmetical means. The outcome from the AHP analysis is then analysed using descriptive statistics and statistical analyses. The statistical methods described in this section are examples of appropriate methods for analysis of the data. There are several other methods that are appropriate and could be used. An overall view of the data can be given by using bar plots of the means for each group. Further analysis of the ratings can be performed using an ANOVA test [10], if the data is normally distributed and the variances are equal. Significant rating differences can then be established within each group using Fisher’s PLSD test [10]. The normal properties of the data are checked using a KolmogorovSmirnov test for normal distribution. If the assumptions for ANOVA are not met by the data, a Kruskal-Wallis nonparametric test can be performed instead [11].

High support

High pos. effect

Even profile

analysis

Complete Low support

Low pos. effect

Un-even profile

and cost

AHP High pos. effect

Discrepancy

Adapted practices

evaluation

Partial Removed practices Low pos. effect

Figure 1. Decision support using the method The results from the analysis are intended to support the decisions taken prior to XP practice introduction. The choices performed prior to introduction are illustrated in Figure 1. The first choice, partial or complete introduction is often decided beforehand, but results from the method may change this. For example, if a complete introduction has been decided on beforehand and certain practices appear difficult to implement with low positive effect, a partial introduction may be considered instead. If a complete introduction is selected, the outcome of the method is used to identify practices that need more support, e.g. training courses, to ensure that they are introduced properly. Similarly, practices may be identified that need less support. If a partial introduction has been chosen, the outcome provides information on which practices to implement and which to reject. Discrepancies between people and groups provide additional information for the decision makers as well as the ease of introduction. Discrepancies between groups of persons can give vital SPI strategy information. The ratings of one factor by two groups are compared using an unpaired t-test if the ratings are normally distributed. If the ratings are not normally distributed a Mann-Whitney test is performed [11]. The final outcome is a list of which practices are given priority by each group, and a list of significant differences between the groups, if any. This information can be used in the decision process. If all XP practices shall be used, the results about ease of introduction is a support when planning for training and coaching efforts. If a subset of practices shall be used, the expected effect may guide the selection. Significant differences between groups provide certain information on that there are different expectations on the XP practices.

3.2

Evaluation Method

In order to evaluate the method, it was applied in a case study. The outcome of the case study, in turn, was validated against data from a case study based on the actual introduction of the XP practices in a development project [6]. The purpose of this first study was to investigate the introduction and use of XP in a real-world environment. The case study application of the method presented in this paper was conducted during spring 2001 as a part of a graduate course in software engineering at Lund University.

The students taking this course are in fact the subjects, referred to as “students” in the study. In addition, a group of industry people acted as subjects, referred to as “industry”. The case study applied one treatment, i.e. the decision support method, to two types of subjects, i.e. one group of industry people and one group of students. The industry people represent one viewpoint and the student group represent another viewpoint. The industry group was a team of 5 developers about to commence development using XP and the students were a group of 26 graduate students at the computer science and engineering graduate programme at Lund University. The students were working in pairs. The subjects were given an introduction to XP. The students were asked to read an introduction to extreme programming [2] and were presented with the practices in a class-room environment. The developers were presented to extreme programming through an introduction day with an introductory presentation, discussions and an extreme hour [6], a group exercise performed during one hour illustrating a simplified version of XP. After the XP introduction, the subjects were asked to rate the factors first according to their expected positive effect on development and then according to their expected ease of introduction. Both groups performed the rating of the practices via a web-based interface, and the AHP calculations were performed using the FocalPoint tool [8]. The students performed the rating in a classroom environment in pairs, while the developers performed the rating in the office environment one-by-one. The FocalPoint tool allows incomplete ratings, i.e. that all pairs are not compared, but all the subjects in the study performed ratings of all pairs of XP practices. The outcome from the ratings is a vector of ratings for each of the five industry subjects and one for each of the 13 pairs of student subjects. Furthermore, a consistency index is calculated, which indicates how consistent the subjects are in their ratings. The rating vectors are analysed according to the method and the outcome is a list of which of the practices where considered significantly more easy and more effective than others. In addition, significant differences between the subject groups are identified. Finally, the outcome of this case study is compared to the qualitative judgement of the XP practices based on the use of XP in another case study. Section 4 presents the result of this evaluation.

3.3

Validity

The purpose of the empirical study is not to come to some certain conclusion on XP practices, but to illustrate and make credible that the decision support method is useful and provides relevant information to decision makers. Below, the issues of threats to internal, conclusion, construct and external validity are briefly discussed [12]. The mostly discussed internal validity threat is related to selection of subjects. Using students as subjects tend to be criticized, but in this case study, the most important characteristics of the subject groups is that they are somewhat different with respect to the context in which they perform the method. This is, however, used to illustrate the ability of the method to identify such differences. The subjects used in the study did not have very much

experience in XP other than the brief training provided. This was intended to produce a similar situation to a company introducing XP. Before the introduction of XP the people involved are not likely to have much experience of XP in a real world situation. Threats to conclusion validity concern statistical tests. Standardized and robust tests are used. In cases of nonnormality, these are reported openly and the analyses are mainly based on non-parametric tests. Threats to construct validity concern the definition of the study itself. This may be a threat as the method is not compared to other methods. However, the validation against an observational study of the industry subjects as they complete a project using XP [6] is intended to reduce this threat. Finally, threats to external validity concerns generalization. As the study is a case study, there is not intention of finding a general result, but to illustrate the outcome in a certain context and show that the method can provide a useful result.

4 4.1

EVALUATION RESULTS Method Results

The case study was performed successfully, and all the 31 subjects completed their ratings of the XP practices. The students performed the rating in pairs due to practical reasons, making the number of student ratings 13. The consistency ratio was calculated and found to be sufficient for all except one subject in each group. The subjects having a consistency ratio of 0.2 or more were removed as outliers [7]. The limit of 0.2 is fully satisfactory in a practical application of AHP. In total there were 4 industry subjects and 12 pairs of student subjects remaining in the study after this step. The individual ratings were first aggregated using the arithmetic mean of the ratings for each factor. This data were plotted in bar plots, see Figure 2. A Kolmogorov-Smirnov test was then performed on the data to check if it was normally distributed. The next stage of analysis was to perform an ANOVA test. As we could not formally prove the normal distribution of all the factors, we also performed the nonparametric alternative, the Kruskal-Wallis test. This test gave the same result as the ANOVA test for the expected effect, but differed in one case for the expected ease. The final test that we would like to perform is the Fisher PLSD test to investigate the individual relationships in the data, but this requires that the preconditions for the ANOVA are satisfied. In the cases where the ANOVA provided the same result as the Kruskal-Wallis test, at a significance level of 0.05, the Fisher PLSD test was used nonetheless [10,13]. The bar plots (Figure 2) indicate that the industry subjects rank short releases as the most effective practice and on-site customer as the second most effective. The students rank the testing approach having the largest effect, while the collective ownership is ranked lowest. Industry subjects rank pair programming as the by far easiest practice to introduce, while the students expect 40-hour week to be the easiest. The Kolmogorov–Smirnov test was performed and the data was found to be normally distributed on a significance level >0.9 as reported in Table 1.

Table 1. Number of normally distributed cases out of the 12 practices. Effect 5 10

Both ANOVA and Kruskal-Wallis tests were run to identify whether there are any significant differences in the rankings. The p-values are reported in Table 2. It can be noted that the ANOVA test and the Kruskal-Wallis test give significant results on the same cases for effect. Hence, we can identify significant differences concerning the ranking of effects, while the differences concerning ranking of ease of introduction are not statistically significant.

Ease ANOVA 0.014* 0.19

Industry Student

Interaction Bar Plot for Practices

for Practices (c)Effect: Category Student, Effect

Inclusion criteria: effect ovo from all.svd

Inclusion criteria: effect students from all.svd

,14

,12

,12

,1

,1

Coding Standard.2 Coding Standard.2

Forty-h week.2

On-site Customer.2

Collective Ownership.2

Continuous Integration.2

Refactoring.2

On-site Customer.2

Cell Interaction Bar Plot for Practices

Pairprogramming.2

Coding Standard.2

On-site Customer.2

Forty-h week.2

Continuous Integration.2

Pairprogramming.2

Collective Ownership.2

0

Testing.2

0

Refactoring.2

,02 Metaphor.2

,02 Simple Design.2

,04

Planning Game.2

,04

Testing.2

,06

Simple Design.2

,06

,08

Metaphor.2

,08

Short Releases.2

Cell Mean

,14

Short Releases.2

Cell Interaction Bar Plot for Practices

(d)Effect: Category for Practices Student, Ease

for Practices (b)Effect: Category Industry, Ease

Inclusion criteria: cost students from all.svd

Inclusion criteria: cost ovo from all.svd

,2

,14

,17

,12

,15

,1 Cell Mean

,13 ,1 ,08

,08 ,06

Forty-h week.2

Collective Ownership.2

Cell

Figure 2. Bar graphs showing mean ratings for: (a) developers rating effect of practice, (b) developers rating ease of introduction of practice, (c) students rating effect of practice, (d) students rating ease of introduction of practice.

Continuous Integration.2

Refactoring.2

Pairprogramming.2

Testing.2

Simple Design.2

Metaphor.2

Coding Standard.2

On-site Customer.2

Forty-h week.2

Continuous Integration.2

Pairprogramming.2

Cell

Collective Ownership.2

Testing.2

Refactoring.2

Metaphor.2

0

Simple Design.2

0

Planning Game.2

,02 Short Releases.2

,03

Short Releases.2

,04

,05

Planning Game.2

Cell Mean

K-W 0.011* 0.010*

Interaction Bar Plot for Practices

for Practices (a)Effect: Category Industry, Effect

Cell Mean

Effect ANOVA 0.065* 0.007*

K-W 0.15 0.51

In the cases where there are significant differences, we use the Fisher PLSD test to identify which differences are significant. As seen in Table 1, all data is not normally distributed, but as the test are robust and ANOVA and Kruskal-Wallis provide the same result, we continue with the Fisher PLSD test.

Planning Game.2

Ease 5 8

Industry Student

Table 2. P-value of ANOVA and Kruskal-Wallis tests (* indicates significant results).

The PLSD test provides comparisons between each pair of practices. The full analysis is too extensive to report here, but we highlight a few interesting results. For the effect rankings by the industry subjects, the highest ranked practice, short releases, is significantly different from five out the remaining eleven practices. The second ranked practice, on-site customer, is significantly different from only two of the remaining eleven practices. Except for these, there is only one significant difference, i.e. between the planning game and the continuous integration practices. The case is similar for the student ratings. The highest ranked testing approach is significantly different from seven of the other practices and the lowest ranked collective ownership is significantly lower than 4 other practices. In addition, simple design is found to significantly higher ranked than two other practices. A further analysis was performed by a pair-wise comparison of the means for each practice between the student and industry group. Normality was checked by using the Kolmogorov–Smirnov test for normality on each set of means. The results rejected normality in one of the four sets, the factor ease in the group industry. The results are summarised in Table 3.

The actual ease/effect results therefore are therefore irrelevant to the reader. The factors ease and effect could in other companies be replaces by other factors considered more important to the XP introduction, for instance lead-time reduction or quality improvement

4.2

Table 4. Coding Scheme

Ease Effect

Table 3. Normality check for the means of practices within each group.

Industry Student

Ease Rejected >0.9999

Effect >0.9999 >0.9999

In the case of comparing the two groups for effect, this can be performed using the pair wise t-test [10] as both groups are normally distributed. The test shows a significant difference between the groups. As the groups are not normally distributed for the factor ease, the preconditions for the t-test are not fulfilled. A non-parametric test was attempted instead, the Wilcoxon Signed rank test. This test could not, however, show a significant difference between the groups. With this information at hand, the decision on improvement strategy can take place, see Figure 1. In the case of selecting a subset of XP practices, short releases, on-site customer and testing are ranked the highest. The two groups have different topranks, and hence the rationale behind their ranking should be investigated. In the case of introduction of all XP practices, it can be concluded that the industry group assumes that pair programming and collective ownership require little support for the introduction, while the other practices are evenly ranked. The students assume 40 hour week and on-site customer will be the easiest to introduce. The fact that a significant difference could be shown using the t-test is interesting. If the groups were within a large development organisation, a different XP implementation strategy might be considered where groups are significantly different. Note that the results from the application of the method are specific to the environment within which the experiment was performed. The intent of this paper is not to investigate a general ease and effect profile for XP but rather to provide and test a tool that can show this relationship in specific cases.

XP Application Case Study

A qualitative XP case study is conducted at the company where the industrial subjects performed the development project using XP [6]. Here the XP practices were used by 10 people during the project. The experiences on the ease of introduction and effect of applying the practice have been summarised and are presented in Table 5. The practices are judged qualitatively with respect to ease and effect using the scheme in Table 4.

-Very hard Very neg.

Hard

0 Neutral

+ Easy

Neg.

Neutral

Pos.

++ Very easy Very pos.

Table 5. Case Study Summary

XP Practice Planning Game Small Releases Metaphor Simple Design Testing Refactoring Pair Programming Collective Ownership Continuous Integration 40-hour week On-site customer Coding standard

Ease + + -+ ++ ++ + + -

Effect ++ ++ + ++ + ++ + + 0 ++ +

It can be seen in the table that some of the practices identified using the method presented in this paper before the start of the project as effective, were also actually found to be effective during the course of the project., for instance short releases and on-site customer. The students ranked testing very high, which in the case study was considered having very large effect. However, a major difference between the results of this qualitative evaluation and the systematic decision support method, is that the qualitative evaluation does not involve any systematic ranking and comparison. Hence, two practices judged on the same ease or effect level, cannot be compared. The decision support method provides complete comparisons between all pairs of practices.

5

SUMMARY

Extreme programming (XP) has been set in focus recently as the most widely spread example of agile methodologies for

software development. As XP consists of a set of practices, the introduction of XP in an organisation involves strategic decisions whether to introduce all practices, or a subset. In the latter case, decision support is needed to decide which subset to select. In the first case, information about expected problems in the introduction is valuable in the practical planning of for example training and coaching efforts. In this paper, a decision support method, based on the Analytic Hierarchy Process (AHP) is presented. The method is applied to a case study with industry and student subjects, which purpose was to rate the XP practices with respect to ease of introduction and expected effect of the practice. Furthermore, the outcome is validated by comparing the results with the qualitative evaluation of the application of XP in a project with 10 participants. The outcome of the application is that the industry group and the student group have different views of both ease and effect of the practices. This is an interesting result as such, since this indicates that measures have to be taken to motivate or train people in the process improvement process. The proposed method hence gives support for the decision process by visualising opinions in the organisation, and thereby improving the chances for good decisions, that are firmly anchored in the organisation.

6

ACKNOWLEDGEMENTS

This work was partly funded by The Swedish Agency for Innovation Systems (VINNOVA), under a grant for the Center for Applied Software Research at Lund University (LUCAS). The authors would also like to thank Martin Höst (LTH) for his contributions to the paper.

7

REFERENCES

[1] Beck, K., Extreme Programming Explained: Embrace Change, Addison Wesley, 1999. [2] Beck, K., “Embracing Change with Extreme Programming”, IEEE Computer, October 1999, pp. 70-77. [3] Bandinelli, S., Fuggetta, A., Lavazza, L., Loi, M., Picco, G. P. “Modelling and Improving an Industrial Software Process”, IEEE Transactions on software Engineering, Vol. 21, No. 5, 1995, pp. 440-454.

[4] Karlström, D. , Runeson, P., Wohlin, C., ”Aggregating Viewpoints for Strategic Software Process Improvement – A Method and a Case Study” proceedings 6th International Conference on Empirical Assessment & Evaluation in Software Engineering, EASE 2002. [5] Saaty, T. L., “The Analytic Hierarchy Process” McGraw-Hill, New York, 1980. [6] Karlström, D., “Introducing Extreme Programming – An Experience Report”, proceedings 3rd International Conference on eXtreme Programming and Agile Processes in Software Engineering, XP2002. [7] Karlsson, J., Ryan, K., ”A Cost-Value Approach forPrioritising Requirements”, IEEE Software September/October, 1997, pp. 67-74. [8] Home page of Focal Point AB, accessed 02/03/05, http://www.focalpoint.se/index_eng.html. [9] Forman, E., Peniwati, K., “Aggregating Individual Judgements and Priorities with the Analytic Hierarchy Process”, European Journal of Operational Research, 1998, pp. 165-169. [10] Montgomery, D. C., “Design and Analysis of Experiments”, John Wiley and sons, 1997. [11]

Siegel, S., Castellan Jr., N. J., “Nonparametric

Statistics”, McGraw-Hill International, 1998. [12] Wohlin, C., Runeson, P., Höst, M., Ohlsson, M., Regnell, B., Wesslén, A., Introduction to Experimentation in Software Engineering, Kluwer Academic Publishers, 2000. [13] Bratthall, L., Wohlin, C., “Understanding Some Software Quality Aspects from Architecture and Design Descriptions”, International Workshop on Program Comprehension, 2000, pp.27-36.