Short Running Title: Value-based renewal processes in software maintenance Contact Author:
Giuseppe Visaggio
Address:
University of Bari Department of Informatics Via E. Orabona, 4 70126 Bari – Italy
Phone:
+39 80 5443270
Fax:
+39 80 5443270
e-mail:
[email protected]
Value-based decision model for renewal processes in software maintenance
Giuseppe Visaggio Department of Informatics University of Bari Via E. Orabona, 4 70126 Bari – Italy
[email protected]
ABSTRACT
This work is based on a retrospective analysis of a renewal process applied to a very aged, highly degraded software system. Some parts of the results are generalized to define a method for determining the technical and economic qualities of software system components. The work also presents a decision model for identifying the most suitable renewal process to be applied, based on the quality of the components and the aims of the renewal process. With the model presented, decision-making on the renewal process is specialized to each component of the software system, thus ensuring greater benefits from the process as a whole. The same model can be used to monitor software system quality decay and thus avoid the need to use the most costly renewal processes.
ii
1. Introduction
The semantics underlying some key words used in this work need to be established right from the start. A software system is described as a legacy system if it is one of an organization’s assets and works to support some of its business functions. A legacy system is termed aging when its economic returns or quality are lower than expected; thus the concept of aging has nothing to do with the chronological age of the system.
To rejuvenate an aging system, one or more types of renewal process are used. For instance: reverse engineering analyzes a subject system to identify its components with their interrelationships, and to create a representation of the system in another form or at a higher level of abstraction; restructuring improves the structure of a program automatically, without taking the program purpose into account; restoration improves the structure of programs, and of data according to their meaning in the programs; reengineering examines and alters a subject system to reconstitute it in a new form with improved quality. This may include modifications with respect to new requirements not met by the original system; rehosting refers to migration of the system to a different architecture; migration involves changing the software environment the legacy system runs on Chifosky [1990].
On the basis of the above definitions, software system aging has both economic and qualitative explanations. Assessment of a legacy system must therefore focus on both aspects and must support decisions on what type of renewal process is best able to attain the improvement targets that have been set.
1
Owing to their importance in software maintenance environments, the above problems have been much studied and an exhaustive list of references on this topic would take up too much space. We shall therefore confine mention to Sneed [1995], Ransom et al. [1998], Favaro and Pfleeger [1997], works produced in communities with different characteristics and nearest in approach to the present work.
These works represent the components of a legacy system on a plane with two axes: technical quality and economic value of the components. Each axis has a threshold separating acceptable values from unacceptable ones for each of the two parameters. The two thresholds divide the plane into four quadrants (figure 2.2): I, low quality, low value; II, low quality, high value; III high quality, high value; IV, high quality, low value. All authors conclude that it is possible to decide whether to subject each component of the legacy system to one of the renewal processes or else to rewrite it, according to which quadrant it belongs to.
Apart from Sneed [1995], who provides more detail on how to score the software components, the others leave this decision to the decision-maker.
In particular, in Sneed [1995] some
business value components (market value, contribution to profit, data significance) are identified, which must each be given a score in each program by an expert.
All these works leave the definition of the thresholds on each of the axes entirely up to the decision-maker. For the authors, the decision is unrelated to the business organization, and is the same for all the components in each quadrant.
In this work, on the other hand: 1. Guidelines are provided for calculating the quality and economic scores for each component; 2
These can be reused in other projects, although they can and must also be continually refined with use; 2. A model for determining the thresholds on each axis is defined; the model depends on the quality and economics policy adopted by the organization intending to renew the legacy system; 3. A decision process is included, that helps to establish which renewal process should be carried out for each component; this process may differ for components belonging to the same quadrant and depends on the targets the organization intends to attain with the renewal process.
The present work is based on generalization of the results of a retrospective analysis of the data obtained with a real renewal project carried out on a banking software system [Visaggio 1997, 1999b] with 653 programs for 1.5 million instructions, observed over about 2 years. It includes a description of the model for determining the thresholds on the two axes, an example of application of the model to the real system studied, and use of the model to assess the benefits that can be obtained by applying it to an aging system (section 2); discussion, which contrasts the model with related works (section 3); and the final conclusions, that point out the problems still left open and indicate the author’s future work directions (section 4).
2. Aims of renewal processes
There are many different metrics, with different meanings, for assessing the quality and the economic value of a software system; a summary indicator of the aging of a system is obtained with a Score associated with each system component, calculated according to the relative 3
metrics.
2.1 Model for calculating quality scores
A software system S is composed of components. The definition of a component is, in this case, at the discretion of the software engineer and is the elementary unit he/she will take as the basis for quality assessment and for the decision on the renewal process to be activated: S={C1, C2,…, Ci,…,Cn}
Each component is associated with a set of measurements, one for each metric in the quality assessment plan. Component i th is associated with set
Q i = {m1i , m2i , … , mji ,…., mki }.
where generic mji is the value of Mi for component Cj .
Each metric Mi has a baseline Bqi > 0 that represents the limit value considered satisfactory for this metric, i.e.: if mik < Bqi then Ck has a good quality level for metric Mi . The baselines express the quantitative quality targets of the organization that established them. They are, therefore, specific to each organization and may change with time if the quality targets change. The values of the baselines are a form of experience that can be generated by the organization adopting them or imported from other organizations. In any case, the baselines should gradually be improved as the organization attains greater maturity in their use. 4
After establishing the Bqi the Score Sqi can be defined for each component Ci:
[1]
Sqi = ∑j ((Bqj / mji )* wqj ) ;
if mji = 0 then Sqi =∞; this value cannot be represented on the score axis and so Bqj / mji =10*Bqj whereby Sqi will be very large but still representable on the score axis; where:
wqj is the weight assigned to metric Mj in assessing the quality of a component, and must be such that:
[2]
∑j wqj = Tq;
and Tq is the quality Threshold that separates poor quality components (Sq i < or =Tq) from good quality components ( Sq i > Tq ).
The weights wqi express the importance of a metric and, therefore, the organization’s quality policy. The higher wqj , the greater will be the importance of metric Mi in the metric quality definition plan and, therefore, the software engineer’s need for a better control of Mi. In fact, as the value of wqi increases, Sq becomes more sensitive to values mi.
If the organization has no policy or does not wish to express any policy, the weights wqi can all be attributed the same value.
The value of Tq details the resolving power of the Sqi scale. The higher Tq, the better the 5
Scores for each component can be distinguished.
2.2 Characteristics of the quality scores
From [1] and [2] it can be seen that if all the metrics have lower values for Ci than Bqi , the Sqi value will be higher than Tq. Thus, if a component has a score below Tq, it has at least one metric with a value above the threshold.
When a component has a score below Tq it needs quality improvement. For Sqi to be higher than Tq, one or more metrics with values below the corresponding baseline must be improved; it is best to improve metrics with a higher wq.
To improve a metric it must be understood, and this will suggest what renewal process should be executed. Table 2.1 is an excerpt of the decision table aiding comprehension of the quality metrics, obtained by means of generalizing the results of a retrospective analysis of a real case, described in Visaggio [1997].
Using the above decision table to reduce the number of Anomalous Files, i.e. files that are not created in the application but are read, modified and cancelled, for instance: it is necessary to execute Reverse Engineering if the program only has access functions to the anomalous files. If, on the other hand, the program has other functions then Restoration must be executed to isolate the functions accessing the files from the other functions implemented in the application and cancel them. The exact way these files are treated by the application must be defined if they have been created by other applications and, finally, the right requisites must be implemented by 6
means of reengineering.
One example may be a real legacy system whose components are programs. Table 2.1 shows the baselines and relative weights for the metrics detailed. The weights indicate that the metrics considered most important in the example are: number of IF and PERFORM range overlay. Semantic Data, on the other hand, has zero weight, which means that this metric characterizes the system but does not help to assess the degree of aging. Figure 2.1 shows the scores for the programs. Most of them fall under the threshold, set at a value of 50; the only 2 outliers (Sq>300) have been eliminated. The distribution of the components in figure 2.1 shows that the system suffers from quality decay and must be renewed. The points representing each program can be seen to be more densely clumped towards the bottom, so it would be necessary to increase the threshold value in order to distinguish the individual points associated with each program.
Programs number
201
1
0
100
200
300
Sq
Figure 2.1 – Distribution of the Sq in the Legacy System, before the renewal process.
7
METRICS
BASE
WEIGHT ACTIVITIES
PROCESS
1
1. Reengineering
LINE Anomalous
1
1. Eliminate Programs Accessing An.F.
2. Eliminate Programs Parts Accessing 2. Restoration
Files (An.F.)
An.F.
3. R.E.
3. Define the correct Requirements for 4. Reengineering managing An.F. 4. Implement the previous Requirements Dead Data
1
1
(D.D.)
5. Eliminate Dead Data
5. R .E.
6. Eliminate Instruction Creating D.D.
6. Restoration
7. Define Requirement for Improving 7. R .E. Flexibility of Db
8. Reengineering
8. Implement previous Requirement Semantic
-
0
Redundant
9. Eliminate Synonyms with the same 9. R .E. attributes
Data
10. Reengineering
10. Eliminate Synonyms with different Attributes
IF Number
11
9
11. Classify IF
11. R .E.
12. Extract Procedural IF
12. Restoration
13. Redefine Specification and Implement 13. Reengineering Decisional Modules PERFORM
1
5
14. Eliminate the overlay
Range
14. Restructuring/ Restoration
Overlaid Table 2.1 - Excerpt of the decision table Metrics- Processes. 8
2.3 Model for calculating the economic score
The economic score is calculated on the basis of the business value of each function implemented in the programs. It therefore identifies each program’s contribution to the implementation of the business function, with reference to its size. The value calculated for each program is distributed over its size unit, thus making the values for each program congruent and independent of size. Each component C i has a size Szi . As stated above, to obtain a model that provides comparable values regardless of the size of the component being assessed, all the values are calculated per size unit. There are different metrics for calculating the size of a software component; the organization can choose the one to use in the calculation model. In this work, the model for calculating the economic benefits obtainable is independent of the metrics used to express the size and then therefore be used whatever the metrics chosen to measure the size.
Each component implements a Business Function Bf i or a part of it. Let:
[3]
Bfi = {Sz1i , Sz2i ,..., Szki ,…, Szni }
where Szki is the size of Ck that contributes to implement Bfi . We have already observed that the size of a component can be measured in different ways, e.g. number of lines of code or of function points. Szi = Σk Szki
is the total size of the implementation of Bfi .
Each Bfi can be associated with a Business Value Bvi on an annual basis. This value must 9
express the aim of the investment made when the software system being assessed was created or maintained: achievement of price leadership, reduction of costs; raised production revenue or product differentiation [Rasmussen 1996]. A method often used to calculate the annual rate of the business value is the Net Present Value. Sometimes, the value of a business function cannot be quantified directly, in which case a method must be found for indirect quantification. For example, if the value of the business function is in the organization image, it could be quantified by answering the question: how much is the organization willing to invest on software that improves this image?
Assessment of a business function could include a combination of many parameters. In any case, the organization’s business management must define this method, normally established at the time when it decides to invest in the purchase of the software system.
For [3] every Ci can play a part in many business functions. We can say that Ci contributes to realize {Bf1,…, Bfj ,…, Bfm } and that generic Bfj contributes size Szij . To calculate its Business Value the procedure is as follows: [4]
Bv(Ci ) = ∑j (Bvj / Szj) * Szij where
Bvj is the business value of Bfj Szj is the full size of Bfj Szij is the size of Ci ’s contribution to Bfj
It is sometimes difficult to obtain all the information necessary for calculating [4]. One example is when each component’s internal coherence is low and there is strong coupling among the components. In these cases, the Business Value per component can be spread over the whole software system according to its size with respect to the whole system: 10
[5]
Bv(Ci) = (∑k Bvk / ∑i Szi ) * Szi
In [5], the first summation is extended to all the Bf of the software system, the second to all the components of the system.
E.g.: Let us suppose we have 3 business functions (Bf1, Bf2, Bf3) and two programs (C1, C2) that contribute to these business functions, as in table 2.2.
Business Functions
Sz1j (N. Locs)
Sz2j (N. Locs)
Business Value
Bf1
100
---
30
Bf2
50
30
40
Bf3
---
80
30
Total
150
110
100
Table 2.2 - Business value detailed per component.
Then, by applying [4]: Bv(C1) = 30/100*100 + 40/80*50 Bv(C2) = 40/80*30 + 30/80*80
Instead, let us suppose we do not know the details of how the business functions are implemented in the two components. Then, for [5] :
Bv(C1) = 100/260*150 11
Bv(C2) = 100/260*110
The business value considered up to now is based on purely economic benefits. Unfortunately, for the organization using the software system being assessed, the Economic Value (Ev) does not correspond to Bv. In fact, this has to be purged of the costs for management and maintenance of the system.
These costs are not always evident and often entirely hidden. For example, if a system is badly documented, it is difficult to transfer knowledge of it to maintainers, users or administrator. Apart from the risk of blockage of some programs in the absence of those who know them, this involves an increase in the contractual power of the single resource who knows the application; thus he often costs more than the effort value required from each resource: the overcharge is hidden. Another example of a hidden cost may be the distance between the organization using the system and the supplier of management or maintenance services of the software system: the real costs include expenses for transferring technical staff.
There is a vast variety of cost sources, some of which, as we have seen from the previous examples, are due to the condition of the software system or the organization location. In short, many cost sources are specific to the organization. Hence, a possible list of costs that must be expunged from the business value to obtain the real economic value of the software system is included in this work. The reader must bear in mind, however, that the list may be incomplete and should be refined by experience within each organization, which will become increasingly mature as time goes on. •
Csass :
Cost price of the software system. This cost may have been cancelled out in the
case of aged systems; 12
•
Csman : cost of maintenance; this includes costs for human resources and the relative logistics for ordinary maintenance of the software system; the costs for execution of the renewal process must not be considered in this count, as they come under the heading of extraordinary maintenance;
•
Csadm: cost of operation of the system; this must include all the costs for human resources and the relative logistics involved in managing the software system;
•
Csast: cost of assistance in the use of the system; this cost must include all the human resources, informatic experts and others, involved in assisting the operators that use the system;
•
Csdif : cost of diffusion of knowledge of the package to maintainers, managers, assistants and users; these include the mean yearly costs for the human resources and the relative logistics involved in diffusing knowledge of the package;
any costs for extraordinary
diffusion operations should be excluded, such as those following renewal of the software system or acquisition of new components for the system; •
Csplt: cost price and management costs of the Hw and Sw platforms necessary for using, managing and maintaining the software system;
•
Csorg: cost of coping with bad or incomplete functioning of the software system; for instance, if owing to defects that cannot be eliminated by ordinary maintenance, the system risks erroneously validating data in one or more files, the organization must provide tools and human resources for monitoring files at risk; the costs for the tools and the human resources and relative logistics involved must be considered in this cost source.
All these costs can be subdivided per component only if the organization that sustains them has an adequate analytical accounting system. In the absence of analytical accounting for cost destinations, the components are subdivided according to their size in comparison with the total 13
size of the system, to align all the values on the basis of the same size unit.
In validating the Business Value and the costs for each of the components, there is a risk of assessing them incorrectly or incompletely. These risks are well known to economic assessors and must be managed with techniques used in economics, that are outside the scope of this work.
When the above values are known, the economic value Ev of each component can be calculated
[6] Ev (Ci ) = Bv (Ci ) – (Csass i + Csman i + Csadm i + Csast i + Csdif i + Csplt i + Csorg i )
For the Economic Value, too, it is well to define a score Se. For this purpose, the organization defines the set {Rass , Rman, Radm, Rast, Rdif, Rplt, Rorg}. Each of these represents the economic percentage value the organization is willing to spend on the corresponding cost. For each type of cost, Csx ( where x = ass/man/adm/ast/dif/plt/org ) defines a baseline Bex [7] Bex = Rx * ( ∑k BvK / ∑j Szj )
where the first summation is extended to all the business functions and the second to all the software system components. The baseline must be such that Bex > 0, whatever the value of “x”. If Csxk < Bex then Ck has a good Csx type cost. The baselines quantitatively express the cost targets the organization has established. Their specificity to the organization depends on its ability to accumulate experience, for Be as for Bq.
With these premises, the Score Sei for each component Ci can be defined:
14
[8]
Sei = ∑ x (Bex / Csx i * wex ); if Csxj = 0 and Bex / Csxj = 10 * Bex
where wex is the weight assigned to cost Cx in assessing the economic Score of a component and must be such that
∑x wex = Te
where Te is the economic Threshold that divides the uneconomic components (Se i < or =Te) from the economic ones ( Se i > Te ). Te may be different from Tq, but for reasons of symmetry it is easier if they are the same.
The relations between wex, Cx, Bex, and Te are exactly the same as those between wq, M, Bq and Tq, described in the previous section.
2.4 Meaning of the economic scores
From [8] it appears that all the components with lower values than the relative baselines, for all the cost types, are above the threshold. Some components may be above the threshold but have some costs that exceed their baseline. This means that other costs for these components are so much lower than the baseline that they compensate the uneconomic ones. These compensation mechanisms are possible also thanks to the weights associated with each cost type.
When a component is below the threshold, it has at least one type of cost above the 15
organization’s expectations, that is not compensated by the cost types that balance out uneconomic ones. This means that the legacy system must be improved, by applying the renewal processes that affect the cost types causing this uneconomic score.
The organization will need to accumulate experience to be able to determine the best relations between the costs to be reduced and the processes to be activated to attain the economic improvement required. In the analysis described in Visaggio [1999b], the improvements that can be produced by the renewal processes using the following characteristics have been measured: • Changes in the organization requiring knowledge transfer; in fact, when a resource has to start work in a new office, he/she has to learn how to use and manage the software functions that support the activities carried out in the new office; the speed of learning of these new functions is a quality factor (Transferability); • The number of user Requests For Service (RFS) by the maintainers that each developer can satisfy during the given time unit (Productivity); • The tendency toward increase in unsatisfied RFS (Proneness to backlog); • The tendency to waste available man hours, that cannot be used because the knowledge of the programs is distributed; the maintainer free to work at a given time cannot reduce the backlog RFS because he/she does not know the programs that need to be changed to satisfy those particular RFS (Proneness to waste); • The software capacity to evolve, to adapt to changes within the organization or in the applicative domain (Adaptability) • The frequency of user requests for assistance on how to use the software functions, that may be attributed to unsatisfactory documentation (Assistance Requirements); • The frequency of user requests for new reports because the users are unaware that the information is already made available by the software system (Reporting Requirements) 16
• The degree to which the software functions conform to the users’ expectations (Effectiveness); • The frequency with which a software function generates failures (Reliability).
The above characteristics can be weighed up against the costs with a rationale that at present is only qualitative:
Adaptability, Transferability, Productivity, Proneness to waste are all closely related to the cost for maintenance because the first parameter makes the software easier to maintain, while the second makes it possible to distribute the work better among the maintainers and therefore exploit all the man time
available for maintenance.
Transferability also enables better
comprehension of the software and so increases the productivity of each maintainer. Finally, transferability of the system makes it possible to increase the pool of potential maintainers of the software system, thereby reducing their contractual power and eliminating any overcharge. It should be noted that the transferability referred to is generated by the technical documentation of the software system. Transferability due to documentation of the software management results in less running errors and makes management of the information system easier.
Reliability reduces the number of times that execution of the software ends in an anomalous fashion or ends correctly but gives wrong results. Both events require intervention by the system managers to recuperate the condition prior to the event that generated the malfunction, and then re-execution of the system with the necessary safeguards against recurrence of the same malfunction.
System Transferability by means of user documentation increases comprehension of the system and reduces the number of final user’s requests for assistance. This diminishes the value for 17
Assistance Requirements. Moreover, good quality user documentation enables on-line assistance, obviating the need to transfer the assistant and thereby reducing Csast. Improved comprehension implies better exploitation of the information produced by the system; this reduces the value for Reporting Requirements requested for extemporaneous needs.
Transferability through the technical documentation reduces the costs of knowledge transference to the maintainers. Through documentation of how the system functions, it reduces the cost of knowledge transference to the system administrator. Finally, transferability through the user documentation reduces the cost of transference to the final users.
Effectiveness makes the behaviour of the function conform to that expected by the organization so that the automation of the Business Procedures becomes more complete and requires less organizational support, i.e. less cost. Moreover, a lower Proneness to Backlog increases the organization’s satisfaction, in other words, satisfies its expectations from automation, and therefore reduces the need for organizational support.
On the basis of the considerations and the experience on the field acquired, described in Visaggio [1999b], the author proposes table 2.3 to support decisions on the processes to be activated to improve the economic value of a software system.
The processes that can affect Csadm include Migration and Rehosting but these are not justified by any characteristic measured in the case study in Visaggio [1999b]. However, it is clear that if a system is migrated to a more modern platform, characterized by lower administrative costs of the operative system, there will be a drop in this type of costs. In particular, Csass and Csplt are costs that can be improved only by replacing the working application with one having a cheaper 18
cost price or platform. Generally, Migration to a new application has more effect on Csass and rehosting more effect on Csplt.
COST TO BE
BASELINE
WEIGHT IMPROVABLE
REDUCED Csass
CHARACTERISTIC 4.53
0
ACTIVATABLE PROCESSES Migration (H), Rehosting (F)
Csman
Csadm
Csast
67.88
45.25
9.05
20
4
10
Csdif
4.53
4
Csplt
45.25
2
Adaptability
R.E. (P), Restoration (H)
Transferability
R.E. (P), Restoration (H)
Productivity
R.E. (N), Restoration (F)
Proneness to waste
R.E. (P), Restoration (H)
Transferability
R.E. (P), Restoration (H)
Reliability
R.E. (N), Restoration (H)
Transferability
R.E. (P), Restoration (H)
Assistance Requirements
R.E. (F), Restoration (H)
Reporting Requirements
R.E. (F), Restoration (H)
Transferability
R.E. (P), Restoration (H) Migration (F),Rehosting (H)
Csorg
4.53
10
Effectiveness
R.E. (N), Restoration (F)
Proneness to backlog
R.E. (N), Restoration (F)
Table 2.3 - Types of Cost decision table – Efficacy of the Processes. The symbols used have the following meanings: N = no efficacy; P = poor, when most parts of the problem remained unsolved; F = fair, when only few parts remained unsolved; H = high, when all the essential parts of the problem were solved. When one type of cost should be reduced, e.g. Csdif, table 2.3 suggests that it is best to improve 19
transferability with the renewal processes.
This aim can be attained by executing reverse
engineering to improve the system documentation.
In the case study presented in Visaggio [1997] and Visaggio [1999b] the components of the Legacy System were very highly coupled owing to the number of shared files, so the business functions were distributed among the components. Moreover, the historical data on the costs of maintenance were not analytical, so that to calculate Se, equidivision among all the sizes involved had to be performed. Hence, for each component, the size unit (100 LOCs) had the same Se , equal to 12.23. This is very much lower than Te (50); in short, the system had a very low, indeed negative, economic value, as shown in figure 2.3.
2.5 Benefits of renewal processes
With the assessment model presented here, it is possible to calculate the benefits obtained for each component subjected to a renewal process. The model examines the results from two perspectives:
quality and economic value.
To analyze both, each component should be
positioned on a graph with the two scores as coordinates. On the graph, the two thresholds individuate four quadrants (cfr. figure 2.2).
The position of a component on the graph in the figure may vary over time even if it is not subjected to a renewal process. For example, if a software system has a working platform that is amortized over time, when amortization is complete its Csass will equal zero and therefore its economic score will be increased.
20
Se 1
II
III 2
∆Se Te IV 3
I Tq ∆Sq
Sq
Figure 2.2 – System for classifying the components.
Hence, the component shifts towards direction 1 in figure 2.2, and without changing in quality, increases in economic value. Conversely, a component could shift towards direction 2 in figure 2.2, due to obsolescence of some Business Function. In this case, the value of some components could decline because Bv would drop, even without any quality modification.
A component could also move in direction 3 in Figure 2.2, if one or more qualities of the system without any economic influence were to improve. For example, the ease of comprehension of the data could be improved without this resulting in appreciable advantages for maintainers and users.
Clearly, the desired direction of movement of the positions of the components is towards quadrant III. In fact, a legacy system with no aging symptoms should have all its components 21
positioned in this quadrant. To obtain greater benefits, it is necessary to regulate the angle of direction of the component’s movement so that the gain on the economic score (∆Se in Figure 2.2) must be as high as possible for the least increase in quality score (∆Sq in Figure 2.2). In fact, from Visaggio [1997] it can be seen that the costs of the renewal process are anchored to the quality improvement aimed at; on the other hand, the benefits obtained, which justify the investment, are of an economic nature.
To obtain a good angle of movement for the component, the two decision tables described in the previous section can be used. Supposing that the parameter to be improved is Csman, the table 2.3 shows that this improvement will regard the characteristics: Adaptability, Transferability, Productivity and Proneness to Waste. Apart from Transferability, all the other characteristics can be much improved with restoration; this also appreciably improves transferability. None of these characteristics will be markedly improved by reverse engineering.
With reference to the quality models in Fenton and Pfleeger [1993], Grady [1992], Fenton et al. [1995], for example, when these have been opportunely specialized and improved with experience, it can be established which parameters must be modified to improve the characterics individuated previously. Using decision Table 2.1, the efficacy of the renewal processes can be forecast. In our example, among the metrics in Table 2.1: dead-data, IF Number and PERFORM range overlay need modifying. Table 2.1 shows that many problems connected with the presence of IF will remain even after the renewal process because a complete solution to this problem would require reengineering. Moreover, it is clear that the restoration process is proportionally more efficacious the more the effort made available [Visaggio 1997].
The ordering of the activities depends on the specific system and the quality policy adopted. In 22
fact, the quality parameters to be improved should be dealt with in decreasing order of their contribution to Sq, i.e. according to their value and weight. This ensures that the renewal process uses the effort at its disposal first for those parameters that will most improve Sq and then gradually proceeding towards less significant parameters as regards Sq. This method improves the efficacy of the process.
2.6 The use of both scores
The system described in the case study presented the classification shown in Figure 2.3 before the renewal process. This figure shows the same outliers as Figure 2.1. 60
50
Se
40
30
20
10
0
0
100
200
300
Sq
Figure 2.3 – Classification of the components before the renewal process.
In the case study, the decision to subject the system to a renewal process was, of course, taken without the support of the decision tables described here and aimed to improve as many quality metrics as possible. Checking on the economic returns was performed by trimestral monitoring 23
of the improvement of the process and the maintenance costs. 60
50
Se
40
30
20
10
0
0
40
80
120
160
200
240
280
Sq
Figure 2.4 – Classification of the components after execution of the renewal process.
The effect of execution of the renewal processes is shown in Figure 2.4. After the renewal rocess, the reference component became the module, because the programs, which were generally monoliths, had been modularized. Nevertheless, to facilitate comparison of the state of the system before and after the renewal process, again in figure 2.4, each dot represents a program. All the program quality metrics are the result of the mean of the corresponding metrics for the modules belonging to the program.
Figure 2.3 shows that most of the programs fall in quadrant I and quadrant IV. According to Sneed [1995], all the programs must be reengineered, while according to Ransom et al. [1998], Stevens and Pooley, [1998], the programs in quadrant I must be discarded and those belonging to quadrant IV reengineered. In the case study, no program was discarded, no program was reengineered, some were subjected only to reverse engineering and others also to restoration
24
[Visaggio 1997]. Figure 2.4 shows that the quality and economic value of the programs were markedly improved but that they still fall in quadrants I and IV. The economic balance (in EURO) is shown in Table 2.4. Before Renewal
After Renewal
B. V.
6,800,000
6,800,000
Cost
9,888,000
7,165,955
Ev
-3,088,000
- 365,955
Gain
2,722,045
Table 2.4 Economic balance.
The economic value after restoration was still, although only slightly, negative. This negative result could also be due to poor assessment of the Bv. Nevertheless, the renewal process reduced annual costs by 2,722,045 EURO. The entire renewal process required an effort of 2,497 man days at a mean cost of 500 EURO/Day, for an overall cost of 1,248,668 EURO [Visaggio 1997].
The gain was obtained through the reduction of Csman, Csadm, Csast, Csdif and led to important advantages for the management, the maintainers, the system administrators and the users [Visaggio 1999b]. It was evident in the case study that to further improve the system, reengineering would be necessary.
From this case study, we can generalize the following lessons learnt: •
The renewal processes to be executed must be decided for each component on the basis of its Sq and Se;
on the contrary, the decision models in literature require decisions with such
great granularity that they might hide the possible advantages of the renewal processes; 25
•
even when the components do not shift towards quadrant III, there might still be a considerable cost-benefit balance, especially if the system is very aged;
•
the decision model on the processes to be executed must be specialized to the company and adapted to the targets set each time a renewal project is planned; thus the assessment and decision models must be continually improved with growing experience in their use.
3. Discussion
To analyze the significance of the model presented in this work, its sensitivity to some of the more important parameters as regards the economic and technical value of legacy systems quoted in the literature should be examined. Uses of the model that are collateral to the targets described up to now could also be considered.
The company’s business goals [Ransom et al. 1998] are expressed in the Bv, provided that the management has quantified these adequately. This value can also express the expectation of change of some functional requirement or the life expectancy of some functions [Ransom et al. 1998]. In fact, the values of the business functions whose implementation or usefulness to the company are expected to change are kept low. The system’s critical importance for the organization [Ransom et al. 1998] is attributed values in Bv and any inadequacies are expressed by Csorg, which will be proportionally higher, the less suited the system is to the organization using it. In fact, any deficiencies in the software system have to be overcome by modifying procedures within the organization. Deficiencies in resources within the organization [Ransom et al. 1998] can also be expressed by Csorg, in which case other resources must be recruited from outside and their cost will be higher than the industrial cost for internal resources. For the 26
system to be long-lived [Ransom et al. 1998] Csman must be heavily weighted, as also the quality metrics most correlated with maintenance, so that Se and Sq are sensitive to software maintenance factors.
The organization’s level of difficulty in adapting to software system
changes [Ransom et al. 1998] is expressed by Csast and its level of adaptation by Csdif. The quality of the hardware, basic software and environment, and the availability on the market of maintenance services [Ransom et al. 1998], are expressed by Csplt. The ability to find resources able to use these is covered by Csam, which will be proportionally higher the less the resources available on the market. The ability to train the resources needed to manage the platform is shown by Csdif.
In Favaro and Pfleeger [1997] it is stated that 60% of the functions implemented in a software system have no business value. The model described here is sensitive to such a possibility, thanks to the Bv of the components that implement these functions, that would have zero value, so that Se would be drastically low.
The model proposed is also a useful tool for monitoring a software system during its lifetime. In fact, it is helpful if a change occurs in the scenario that its efficiency assessment is based on. For example, if the business scenario changed, the Bv of the functions would change and, regardless of the quality values, the components would be positioned in different quadrants.
More
specifically, a drop in customer satisfaction would render some functions useless and require the construction of new functions. This would be highlighted in the model by the shift of all the components implementing these functions to other quadrants.
Another scenario that could change over time is the architecture. If the maintainer were to change the architecture of the system he would alter the distribution of the business value for the 27
components in [4] and [5]. Consequently, the association between Se and Sq would change for the software system components. In this case, too, the components’ positions in the quadrants would change.
Finally, but no less important, even if no reference scenario changed, during maintenance of a renewed, reengineered or new system, any quality degradation would be revealed by Sq, i.e. by a greater population in quadrants I and II. Moreover, on the basis of the above findings, use of the decision tables after each quality check could show whether the system needed renewal and, if so, what processes should be used. Thus even an excellent or completely new system could need renewal during maintenance to repair the quality degradations that are systematically generated by maintenance [Visaggio 1999a]. In this case the decision model presented can be used to check that the system’s quality does not degenerate so much as to require renewal processes like restoration, that are highly costly. In short, the model can help to prevent system aging.
Thus, every working software system should be monitored periodically from both the economic point of view, Se, and the qualitative, technical points of view, Sq. To support such monitoring, the model would immediately reveal changes in the relationship between Se and Sq and would graphically show how much the change had affected the total Economic Value of the system [6]. The model also provides the manager with information on the difficulties in recuperating quality, to prevent the legacy system from becoming an aged system. This support will be particularly efficacious if the weighting system of the Bv and Mi is valid and truly reflects the real problems.
28
4. Conclusions
This work describes a model for assessing a software system, resulting from the generalization of results obtained from retrospective analysis of the data on a renewal process executed on an operative but very aged legacy system. The assessment model is the basis for a decision process that aims to establish the most suitable renewal processes for improving the state of the legacy system, according to the improvement targets set by the company involved.
The model proposed, and the decision process that uses it, determine the most effective renewal process for each component according to these targets. This is an innovative procedure with respect to previous approaches described by other authors who have dealt with this problem, such as Sneed [1995], Stevens and Pooley [1998], Ransom et al.[1998]. These authors treat the quadrants defined previously as equivalence classes for all the components, according to the renewal processes required.
The model and the decision process described here can be reused for other renewal projects but this work has shown that they also represent an experience package that must be continually perfected with use. Like all experiences in software engineering, both the model and the process must be continually improved, extended and specialized to the organization that uses them and the projects they serve to implement. An obvious example of these needs is the weights to be attributed to the types of cost and of metrics. The greater the organization’s experience in expressing its business and quality policy through these weights, the better these will be specified. Moreover, weighting may be different for different projects or may change over time if the organization’s policy is different for different products or has changed in the course of the years. 29
This work highlights the fact that renewal processes do not have time-limited usefulness but can serve to recuperate quality decay generated by maintenance. This consideration justifies the terminology used in this work whereby a legacy system is a working system belonging to company assets, whereas a legacy system that needs improvement is an aging system. Aging does not, of course, depend on the chronological age of the system.
Two specific problems are left open. One is external validation of the results of this case study in other renewal processes. The other is research on econometric models for estimating the cost of execution of renewal processes a priori. In fact, forecasting the costs of execution of the renewal processes would enable a full cost- benefit analysis to be made, before executing the renewal process.
The author is developing new research to probe both these problems. For the first, the renewal model is being applied to another large old software system. For the second problem, work is being carried out on an econometric model derived from the retrospective analysis of data from the same project that enabled the generalization of the results described in this work.
Acknowledgements
I would like to thank Ms. Mary V. Pragnell, B.A. for her contribution as technical-writer and Ms. Claudia Capurso for her careful and accurate typing of the text. I would also like to express my appreciation of the referees’ painstaking, constructive comments that have resulted in many improvements. This work has been partially supported by the funds of the Italian MURST (60%) 30
under the project headed “Empirical evaluation of the software deterioration”
References
Chifosky, E.J. and J.H. Cross (1990), “Reverse engineering and design recovery: a taxonomy”, IEEE Software 7, 1, 13-17 Favaro, J. and S.L. Pfleeger, (1997), “Making software development investment decisions”, Technical Report 97-01, Howard University’s Centre for Research in Evaluating Software Technology, Washington, DC Fenton, N.E. and S.L. Pfleeger Ed., (1993), Software metrics: a rigorous and practical approach, International Thomson Computer Press, London, UK Fenton, N.E., R. Whitty and Y. Iizuka, (1995), Software quality assurance and measurement: a worldwide perspective, International Thomson Computer Press, London, UK Grady, R.B., (1992), Practical software metrics for project management and process improvement, Prentice Hall, NJ Ransom, J., I. Sommerville and I. Warren, (1998), “A method for assessing legacy systems for evolution”, In Proceedings of the IEEE Conference on Software Re-engineering, IEEE Computer Society Press, Florence, Italy, pp. 84-92 Rasmussen, G., (1996), “Measuring the value of information systems investments”, Technical Report n. 96/03, Caesar, School of Information Systems, University of New South Wales, Sydney, Australia Sneed, H., (1995), “Planning the reengineering of legacy systems”, IEEE Software 12, 1, 24-34 Stevens, P and R. Pooley, (1998), “Systems reengineering patterns”, Software Engineering Notes 23, 6, 17-23
31
Visaggio, G., (1997), “Comprehending aged legacy systems to improve their qualities with a renewal process”, Technical Report n. 97-26, ISERN (International Software Engineering Research Network), University of Kaiserlautern, Germany Visaggio, G., (1999a) “Assessing the maintenance process through replicated, controlled experiments”, The Journal of Systems and Software 44, 3, 187-197 Visaggio, G., (1999b), “Assessment of a renewal process experimented in the field”, The Journal of Systems and Software 45, 1, 3-17
32
