measurement unit for sizing a software application from the user's viewpoint, therefore called function points (FP) [18]. Because FP moves from functional.
IFPUG-COSMIC Statistical Conversion Juan J. CuadradoGallego Universidad de Alcalá, Departamento de Ciencias de la Computación, 28805 Alcalá de Henares, Madrid, Spain
Luigi Buglione Ecole de Téchnologie Superieure (ETS) – Université du Québec à Montreal (UQAM) / Engineering.it, 00148 Rome, Italy
Ricardo J. RejasMuslera
Fernando Machado-Píriz
Universidad Francisco de Vitoria, Departmento de Ingeniería, Madrid, Spain
Universidad Catolica de Uruguay, Montevideo, Uruguay
Abstract One of the main issues faced within the Functional Size Measurement (FSM) community is the convertibility issue between FSM methods. A particular attention during last years was devoted to find a mathematical function for converting IFPUG functional size units to the newer COSMIC ones. Moving from the data sets and experiences described in previous studies, some attention points about cost and quality from the data gathering process emerge. This paper analyzes the data gathering process issue and proposes a solution for overcoming such difficulties. From an application of a repeteable and verifiable procedure, performed in a university course on Software Engineering with the support of an experienced measurer, two new data sets were derived. Finally an analysis of all datasets was done, presenting a possible interval for the conversion between IFPUG-COSMIC fsu. Keywords: Software Engineering, Software Measurement, Functional Size Measurement, IFPUG, COSMIC.
1.
measure of the amount of functionalities that will be delivered to the customer.
Introduction
Measurement of the size of a software system is one of the most important figures for a proper management of a software project. In fact, knowing the the software size to be developed, it is possible to determine the amount of needed resources, both human and materials, time and approximate also how much it will cost.
The enormous existing need of the Albrecht’s work, mainly to solve problems associated with software projects management, together with the success derived from the first uses of such unit [19], were the reasons behind the birth in 1986 of the International Function Point Users Group (IFPUG) created to promote the diffusion of such unit and control its evolution as a standard. The name changed from Albrecht’s Function Points to IFPUG Function Point Analysis (FPA). Since then, several versions of IFPUG have been published [1-7], and it was also recognized from ISO as a standard [21] but excluding its adjustment factor.
The first unit defined and used to measure software size was the number of lines of source code (LOC). Although this unit was spreadly applied, LOC measures the leght of a software, not its size, whatever its type (according to a certain product attribute). Furthermore, the usage of LOC for project management purposes produced the so-called productivity paradox [22], because a growing number of LOC apparently produced lower unit costs and higher productivity figures. Again, LOC can be measured only after the Coding phase was done, while in order to plan a project, an estimator needs to know this information yet from early stages.
COSMIC Full Function Points1 are the result of the pursuit of the international group of experts in software functional size measurement, the Common Software Measurement International Consortium (COSMIC), to find a measurement unit able to be successfully applied to the largest possible number of software domains and, specially, to real-time and embedded software, where the IFPUG method was quite hard to be applied. Since its first publication in 1999 [8], the interest developed in both the academic community and the industry by the new functional sizing method (FSM) was enormous, reaching a vast diffusion and utilization in very short time, with three new versions published later [9-11], including the one which has been
The answer to this problem was provided by an IBM researcher, Alan Albrecht, defining a new measurement unit for sizing a software application from the user’s viewpoint, therefore called function points (FP) [18]. Because FP moves from functional requirements, they are estimable yet from high-level requirements from the customer and then fully countable at the end of the Design phase, obtaining a
1
1
From version 3.0, the name of the method is simply ‘COSMIC’ instead of COSMIC-FFP.
standardized by ISO [20]. This situation is the result of the contribution of three main factors: first, its wider scope of applicability; second, the clarity its concepts were exposed, making it easier to be used and be learned; last but not least, the lower cost for gathering and managing it. All these factors explain why a large number of companies worldwide yet using IFPUG from years are now starting to use COSMIC too, and this is the point from which this research moves on.
ID 1 2 3 4 5 6 7 8 9 10 11
TI 39 52 120 170 218 224 249 260 380 766 1424
TC 23 29 115 109 181 182 173 81 368 810 1662
Table 1. Data from [14] (vog03)
Therefore, one of the most important issues in the software measurement field is the conversion between units of different software FSM methods2, at the aim to save the investments done for sizing the overall software application portfolio. In this research line some of the Authors yet studied the convertibility issue [ 12;13;17].
Vogelezang & Lesterhuis referenced a previous study by Fetcke [15], where 5 applications were measured both with with IFPUG 4.1 and COSMICFFP 2.2 (Table 2). Fetcke did not propose any new conversion equation, but from his data Vogelezang & Lesterhuis obtained the following equation, with R2=0.98:
The paper is organized as follows: in Section 2, previous studies on this issue are described. Section 3 discusses the main difficulties related with the data gathering process, in particular from a cost and quality viewpoint, proposing a procedure for overcoming such constraints. In section 4, the two new data sets obtained are presented. Section 5 presents the analysis for such dataset. Finally, Section 6 presents the conclusions of this study as well as future work on this issue. 2.
C = −6. 2 + 1. 1 * I ID 1 2 3 4 5
EI 30 16 19 21 9
EO 13 4 9 9 4
EQ 13 6 7 7 6
TI 77 40 49 56 31
E 18
X W 25 10
R 28
TC 81 38 51 52 29
Table 2. Data from [15] (fet99)
2.2. Abran et al. (2005)
Previous studies on fsu convertibility
Abran, Desharnais and Aziz [16] presented a new study with data 6 data points (Table 3) comparing again IFPUG 4.1 and COSMIC-FFP 2.2 units.
Three relevant studies, presented in chronological order, have been published about experimental studies on conversion between the IFPUG and COSMIC-FFP units. Every one of them shows a conversion using linear equations, adjusted by the minimum squares method, where the independent variable is the IFPUG measure (I) and the dependent variable is the COSMIC measure (C). As a consequence, the equations look like: C = a +b*I
ILF 21 14 14 19 12
(3)
ID 1 2 3 4 5 6
TI 103 362 124 263 1146 570
TC 75 209 170 203 934 675
Table 3. Data from [1] (abr05)
(1)
The equation obtained, with R2=0.91,was: 2.1. Vogelezang & Lesterhuis (2003)
C = 18 + 0.84 * I
Vogelezang and Lesterhuis [14] published a first study with 11 data points (Table 1) where a linear equation for converting from IFPUG to COSMIC-FFP units was introduced3. The equation obtained, with R2=0.99, was: C = −86.8 + 1.2 * I 2
(4)
2.3. Desharnais et al. (2006) The last paper published about experimental studies on conversion between IFPUG 4.1 and COSMIC-FFP 2.2 units considered a set of 14 applications, produced by Desharnais, Abran and Cuadrado-Gallego [17] (Table 4), with R2=0.93. The authors obtained the following equation:
(2)
Other FSM methods become ISO standards are Mark-II FPA [23, 24] and NESMA FPA [25-30].
C = −3,22 + I
3 Legend for the tables presented in this paper: ID: Project
identification; ILF: Number of UFP per ILF; EI: Number of UFPper EI; EO: Number of UFP per EO; EQ: Number of UFP per EQ; TI: total number of UFP; E: Number of CFP per E; X: Number of CFP per X; W: Number of CFP per W; R: Number of CFP per R; TC: Total number of CFP.
2
(5)
ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14
ILF 112 217 98 61 77 53 56 70 96 105 105 49 26 105
EI 31 98 104 64 94 22 24 94 202 83 55 103 42 157
EO 145 162 127 55 135 29 21 51 54 128 88 49 35 115
EQ 95 168 71 25 66 22 10 72 148 28 69 57 10 70
TI 383 647 400 205 372 126 111 287 500 344 317 258 113 447
E 63 96 59 39 52 20 11 45 78 54 49 50 19 67
X 155 233 125 66 158 37 41 103 110 114 119 86 23 149
W 26 45 68 28 65 7 16 46 193 31 28 38 33 84
R 120 91 146 55 173 24 47 104 198 92 98 78 39 167
It is important to note that the case reported in [14] seems to be different, since the authors belong to a consultancy firm. Consequently, a commercial agreement is in place, with corresponding costs, the company provides data, and the duplicated measurements performed by the authors is part of the internal research of the consultancy firm and not part of the commercial agreement. Nevertheless, it is out of scope in this paper to demonstrate the correctness of the hypothesis of the economical implications of the organization owning the data, and in fact the disappearance of the problem is really very welcome. On data heterogeneity, it is assumed the likeness of the measurements performed by a single measurer and by two very experienced measurers, working together for a long time.
TC 364 565 398 188 448 88 115 298 579 291 294 252 114 467
Table 4. Data from [17] (des06)
3.
Data gathering
3.1 Data gathering constraints
3.3 Data gathering methodology
One of the main problems, if not the topmost, of this kind of researches relies on data gathering, in terms of cost and quality, needing a deeper and rigorous analysis on them.
The data gathering procedure proposed and used in this study is very different from those ones used in previous studies. The solutions chosen to solve the aforementioned constraints was: (a) for data gathering cost issue, the second available solution was chosen, and data was collected within the frame of a final assignment in a university measurement course on IFPUG and COSMIC methods; (b) for the data quality issue, since the lack of experience and heterogeneity in countings of measurers, a rigorous selection of candidate measurers with more knowledge in the matter and better grades was done, jointly with a review for all counts done by an experienced measurer.
Firstly, the data gathering costs: the wages of for specialists and/or certified counters for these methods are usually quite high, up to 200 € per hour in Europe. Therefore, software companies yet adopting a certain measurement unit for sizing their projects often do not see which could be their return of investment (ROI) in repeating the counting with a different functional sizing unit (fsu),, taking care to the costs involved. As a consequence, data gathering in industrial environments is practically impossible. In order to solve this constraint there are basically two possible solutions: (a) researchers, coming for other institutions, usually a university or a research center, do the measures with different units by themselves, using set of applications provided by companies under agreements subscribed between both organizations; (b) groups of students, once they finalize their courses on software measurement studying these units, are asked to perform the required measurements with the different units studied. In both cases the problem of the cost for measuring will be significantly reduced.
The following four steps were performed:
Secondly, the quality of the data obtained in the measurements. Two possible issues that can have an impact on quality are: (a) incorrect measurements, due to inappropriate measurers training; (b) the creation of a heterogeneous database, due to the inclusion of measures done from different measurers without making a proper validation about the inconsistencies and deviations. 3.2 Data used in previous studies Analyzing data from [15][14][1][17] and taking care of the aforementioned constraints, a possible solution can be the participation of a unique, single measurer, coming from a different institution, in almost all cases a university, for doing the counts. 3
1.
Teaching of the IFPUG and COSMIC-FFP measurement processes. The intended targets are undergraduate students for ‘Systems Planning and Management’ courses, in the frame of a B.Sc. on Technical Engineering on Management Informatics. Students receive 20 hours of theoretical lectures on the IFPUG 4.1 and COSMIC-FFP 2.2, 10 hours on each one, provided 2 hours a class. In two courses IFPUG was taught first and COSMIC later, and in other two on the contrary, to neutralize the effects of the order in learning these measurement methods. The versions used are the standardized versions by ISO [20-21].
2.
Selection of the students participating in the measurements. In order to participate in the measurement process it was required to candidate measurers: (a) to have been present in at least 90% of the lessons, therefore, just a one single absence; (b) have a grade better than 8/10 in a written test of knowledge performed on the 11th week of classes. This test was applied to all students, even if they do not participate to the measurement process.
3.
4.
4.
4.2 Second new data sample(jjcg07)
Measurement of a real world application with IFPUG and COSMIC-FFP units. Each selected student received the specifications of a real software application, different to each of them, previously developed by the University of Alcalá (UAH) during the implementation of software in a collaborative project with external software companies,, whose copyrights are owned/shared by UAH Each student should then proceed to measure the assigned application with both FSM methods. Being novice counter, they had three weeks for doing the counts.
Moving from a total of 77 students, divided into 4 groups, 3 of them with 20 students per group and one with 17 students. The final number of students attending at least 90% of the lessons was 64 (83%). From these 64 students enabled to participate in the measurement process, 51 approved the assignment, with the following distribution: 19 approved, 25 notable, 6 with notable and more than 8/10 points, and 8 with excellent. . As a consequence, 14 students were selected to participate in the measurement process (27% of the initially selected ones, those ones achieving excellent and notable with more than 8/10 points). Table 6 lists the first new dataset, those details are reported in [17].
Correction and uniformization of the measures performed by students. The couple of fsu obtained by the selected measureres were reviewed by one of the authors of this paper
ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14
First results: new datasets
These previous steps were applied during two consecutive years (courses), producing the following results: 4.1 First new data sample( jjcg06 ) Moving from a total of 83 students, divided into 4 groups, 3 of them with 20 students per group and one with 23 students. The final number of students attending at least 90% of lessons was 71 out 83 (85%). From these 71 students enabled to participate in the measurement process, 63 approved the assignment, with the following distribution: 18 approved, 34 notable, 10 with notable and more than 8/10 points, and 11 with excellent. As a consequence, 21 students were selected to participate in the measurement process (30% of the initially selected ones, those ones achieving excellent and notable with more than 8/10 points). Table 5 lists the first new dataset, those details are reported in [12]. ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
ILF 101 92 98 49 35 131 82 93 63 70 96 98 103 92 123 63 101 84 77 77 133
EI 141 162 164 72 54 189 114 110 78 111 159 202 96 118 169 25 110 128 86 101 59
EO 54 66 55 82 20 64 64 58 92 72 91 56 68 81 49 69 90 29 45 82 37
EQ 44 66 43 84 0 54 51 52 42 65 0 39 15 33 27 140 6 64 97 39 41
TI 340 386 360 284 109 438 311 313 275 318 346 395 282 324 368 397 307 305 305 299 270
E 74 49 62 21 21 97 39 75 51 68 73 84 72 70 114 22 43 47 37 72 29
X 67 112 86 105 23 79 95 71 60 68 83 70 61 65 70 62 42 61 57 93 29
W 45 36 38 16 15 123 38 33 25 23 50 98 41 27 35 110 42 40 33 20 21
R 48 72 24 86 6 97 28 55 22 110 43 55 39 62 29 42 137 26 68 82 65
ILF EI 103 141 21 24 42 29 42 32 49 48 47 27 28 67 34 12 28 39 42 64 42 54 49 42 35 36 42 29
EO 48 12 23 20 23 48 28 4 8 4 18 36 12 23
EQ 59 9 0 12 21 11 18 21 6 44 12 12 0 0
TI 351 66 94 106 134 133 137 71 81 154 126 139 83 94
E 75 11 20 16 25 41 33 2 32 32 38 56 20 21
X 58 13 26 38 14 30 33 25 41 55 23 60 25 24
W 66 8 10 8 13 34 18 1 12 15 19 16 17 14
R 102 14 11 11 29 6 33 24 30 23 34 36 12 33
TC 301 46 67 54 81 111 117 52 115 125 114 168 74 92
Table 6. Sample jjcg07 [17]
5.
Data analysis
5.1 Looking for linear equations The first step towards the analysis of the gathered data was to determine the linear equations allowing the conversion between IFPUG and COSMIC fsu. The equations found are presented below: 1.
TC 234 269 210 228 65 396 200 234 158 269 249 307 213 224 248 236 264 174 195 267 144
Sample jjcg06. The linear equation obtained using minimum squares adjustment on the 21 data samples, with R2=0.7 is: C = −36.61 + 0.83 * I
2.
(6)
Sample jjcg07. The linear equation obtained using minimum squares adjustment on the 14 data samples, with R2=0.86, is: C = 0.19 + 0.85 * I
(7)
5.2 Linear equations analysis The first analysis on the statistical indicator of the three equations shows all of them between the boundaries considered acceptable, although those from the simple jjcg06 are slightly out of the limits, the distances obtained are not large enough to discard the results obtained.
Table 5. Sample jjcg06 [12]
4
Study
a
b
fet99
-6.2
1.1
vog03
-86.8
1.2
abr05
18
0.8
des06
-3.2
1
jjcg06
-36.6
0.8
jjcg07
0.2
0.9
jjcg0607
-4.5
0.7
7.
[1] IFPUG, “Function points counting practices manual 2.0”, International Function Points Users Group, 1988 [2] IFPUG, “Function points counting practices manual 3.0”, International Function Points Users Group, 1990 [3] IFPUG, “Function points counting practices manual 4.0”, International Function Points Users Group, 1994 [4] IFPUG, “Function points counting practices manual 4.1”, International Function Points Users Group, 1999 [5] IFPUG, “Function points counting practices manual 4.1.1”, International Function Points Users Group, 2000 [6] IFPUG, “Function points counting practices manual 4.2”, International Function Points Users Group, 2001 [7] IFPUG, “Function points counting practices manual 4.2.1”, International Function Points Users Group, 2004 [8] COSMICON, “COSMIC-FFP Measurement Manual 2.0”, Common Software Measurement International Consortium,1999 [9] COSMICON, “COSMIC-FFP Measurement Manual 2.1”, Common Software Measurement International Consortium,2001 [10] COSMICON, “COSMIC-FFP Measurement Manual 2.2”, Common Software Measurement International Consortium,2003 [11] COSMICON, “COSMIC-FFP Measurement Manual 3.0”, Common Software Measurement International Consortium,2007 [12] Cuadrado-Gallego, J.J., Rodríguez, D., Machado, F., Abran, A., "Convertibility between IFPUG and COSMIC Functional Size Measurements," in Proceedings of International Conference Product Focused Software Development and Process Improvement (PROFES) International Conference, Lecture Notes in Computer Science (LNCS), 4589, 273-283, July 2007. Springer. [13] Cuadrado-Gallego, J.J., Machado-Píriz, F., Aroba, J., "On the conversion between IFPUG and COSMIC software functional size units: A theoretical and empirical study,” Journal of Systems and Software, 5 (81), 661-672, June 2008. Elsevier Science Inc, N.Y, USA. [14] Vogelezang, F., Lesterhuis, A., “Applicability of COSMIC Full Function Points in an administrative environment: Experiences of an early adopter,” en: Proceedings of the 13th International Workshop on Software Measurement, Shaker-Verlag, Montréal, Canada, Sept. 22-25, 2003, pp. 232-243. [15] Fetcke, T., “The warehouse software portfolio, a case study in functional size measurement,” Technical report no. 1999-20, Département d’informatique, Université du Quebec à Montréal, Canada, 1999 [16] Abran, A., Desharnais, J.-M., Azziz, F., “Measurement Convertibility: From Function Points to COSMIC-FFP,” Proceedings of the 15th International Workshop on Software Measurement, Shaker-Verlag, Montréal, Canada, Sept. 12-14, 2005, pp. 227-240. [17] Desharnais, J.M., Abran, A. and Cuadrado-Gallego, J.J. “Convertibility of Function Points to COSMIC: Identification and analysis of functional outliers,” Proccedings of the 17th International Workshop on Software Measurement, ShakerVerlag, Palma de Mallorca, Illes Balears, España, Nov. 5-8, 2007, pp. 130-146. [18] A. J. Albrecht, "Measuring application development productivity," en Proc. Joint SHARE, GUIDE, and IBM Application Development Symp., IBM, pp. 83-92. [19] A. J. Albrecht and J. E. Gaffney, "Software function, source lines of code, and development effort prediction: A software science validation," IEEE Trans. Software Eng., vol. 9, no. 6, pp. 639-647. [20] ISO/IEC19761:2003, Software Engineering COSMIC-FFP. A Functional Size Measurement Method, International Standardization Organization, ISO, Genève, 2003. [21] ISO/IEC 20926: 2003, Software engineering IFPUG 4.1 Unadjusted functional size measurement Method. Counting practices manual International Standardization Organization, ISO, Genève, 2003.
Table 7. Parameters of all the equations
Comparing the parameters of equations (6) and (7) with those ones from previous studies, although they present relevant differences, all ones show a similar behavior in the slop of the straight line, around 1, and if the intersection of the most probable intervals for the straight line’ slopes, the interval (0.7, 0.8) is obtained. This is very significative, and it can be seen in previous studies also, where there are too much difference in the independent terms, but a coincidence in the straight line’ slopes around 1, with a most probable interval of (1.1, 1.2).. 6.
References
Conclusions and future work
The main objective of this research was to find a mathematical function allowing the covertibility between the functional size units of a software systems expressed in the IFPUG FPA unit to the equivalent expressed through the COSMIC-FFP unit. For this purpose, the starting point was the analysis of previous conversion studies performed by other authors (some of them performed also by one of the authors of this paper), In such studies different linear equations for conversion were proposed, obtained by the statistical method of the minimum squares adjustment, applied on a data sample gathered by the authors in each case, of by other authors not looking for a conversion equation.. Moving from the observation of some constraints in terms of cost and quality for the data gathering process, a repeatable process has been proposed in order to obtain more reliabla data and in a cheaper manner using B.Sc. students as FSM measurers. The procedure has been verified also by repetition in two subsequent academical years. Some next future works on this research subject are about: § Verification of the results obtained in this study, reducing the confidence interval for the conversion factor. § Comparison of results obtained in studies like this one, based on empirical data on software measurement unit conversion, with those derived from qualitative analysis.
5
[22] Jones C., What are Function Points?, Software Productivity Research Inc., 1997, URL: www.spr.com/library/0functmet.htm [23] C. Symons, “Function Point Analysis: Difficulties and Improvements,” IEEE Transactions on Software Engineering, vol. 14, no. 1, pp 2-11. [24] ISO/IEC 20968: 2002, Software engineering Mk II Function Point Analysis. Counting Practices Manual, International Standardization Organization, ISO, Genève, 2002. [25] Nederlandse Software Metrieken Associatie, “Definitions and counting guidelines for the application of function points analysis. A practical manual 1.0”, 1989 [26] Nederlandse Software Metrieken Associatie, “Definitions and counting guidelines for the application of function points analysis. A practical manual 1.1”, 1990 [27] Nederlandse Software Metrieken Associatie, “Definitions and counting guidelines for the application of function points analysis. A practical manual 2.0”, 1996 [28] Nederlandse Software Metrieken Associatie, “Definitions and counting guidelines for the application of function points analysis. A practical manual 2.1”, 2001 [29] Nederlandse Software Metrieken Associatie, “Definitions and counting guidelines for the application of function points analysis. A practical manual 2.2”, 2003
[30] ISO/IEC 24750: 2005, Software engineering NESMA functional size measurement method version 2.1, Definitions and counting guidelines for the application of Function Point Analysis, International Standardization Organization, ISO , Genève, 2005.
6
