Dynamic Models Of Maintenance Behaviour Martin Shepperd Empirical Software Engineering Research Group School of Design, Engineering and Computing Bournemouth University Talbot Campus Poole, BH12 5BB, UK
[email protected] Abstract This position paper considers the problem of understanding the dynamic behaviour of software systems and in particular from a maintenance perspective. One approach is to consider the dynamics, in particular feedback loops, of such systems. We briefly report on the limited work in the area to date [1, 2] and then describe a case study derived from an information systems for a multi-national organisation. We examine change documents (CD) data over a period of four years. This enables us to treat the data as a time series and search for trends. Other researchers have investigated patterns in the evolution of systems over time and in particular evidence of feedback processes. We noted similar evidence and generated a model to simulate system behaviour over time. The model incorporates two levels of feedback. We compare actual with simulated behaviour and note a good correspondence. From this research, we argue that the analysis of the dynamic behaviour of systems may lead to a better understanding of the behaviour and management of large software systems over time. KEYWORDS: software maintenance, metrics, system dynamics, case study
1. A System Dynamics Perspective. Project scheduling and estimation, not least for maintenance situations, continue to be difficult, prone to error and inaccuracy. We need, therefore, to better understand such processes before we can control them. A major source of understanding derives from empirical study. A number of researchers have suggested the merit of addressing the dynamic behaviour of systems based upon a branch of study known as system dynamics. System dynamics was introduced by Forrester [3] as a means of modelling the dynamic behaviour of complex systems. Of major interest is the feedback mechanism, at least if the aim is to understand what processes effect the behaviour of the system. In a system dynamics model, feedback loops represent the relationship(s) between at least one stock (thing, resource, constraint) and at least one flow (activity, change). The loops generate goal seeking behaviour, although goals are not necessarily achieved, particularly when they conflict with other goals. Two types of feedback loops exist, counteracting feedback loops, which attempt to maintain the necessary conditions to achieve a goal, and reinforcing feedback loops, which compound change, but as a consequence drive the goal further away [6]. The interaction of these two types of loop provides the dynamic behaviour. Abdel-Hamid and Madnick [7-9] and Cooper and Mullen [10] are among those who have applied system dynamics (and thus systems thinking) to software project management. Lehman has focussed on feedback in his long running examination of software evolution, which has led to the FEAST projects [11]. The first stage of the project, FEAST/1 set out to provide evidence on the impact of feedback on the software development process, and show how this could be utilised in the management and improvement of the process. This project concentrates on the evolution of what Lehman terms E-type software. E-type software systems are developed to address a real world problem, and for these applications the notion of "correctness" is meaningless. Instead the acceptability of the software is determined by user satisfaction. Lehman has postulated a number of "Laws of Software Evolution"; those relevant to E-type systems and FEAST can be found in [1], where it is stated that E-type systems are multi level, multi loop, multi agent feedback systems. This suggests that the other characteristics of E-type systems, such as the evolution of complexity and the growth pattern, can be explained in terms of feedback loops.
In the remainder of this position paper we describe a case study of the maintenance behaviour of a large information system over a period of approximately four years. We attempt to simulate our data using an inverse square model but find that an alternative model based upon two feedback loops is more successful. We conclude with discussion of the potential significance of this approach to modelling system behaviour and suggestions for further research.
2. Case study. The system under study is a very large in-house development by a multi-national company. The system consists of 450 C programs/binary files, 1580 database tables and 4000 SQL scripts, on a single platform. The project began in 1989 to integrate the core business activities of the organisation in a common, European application, able to support the various businesses making up the organisation. This case study is based upon change and release data to version 3, released in 1994. Post release data was collected between 1994-1997. Towards the end of this period, in July 1997, responsibility, including for the maintenance of the product, was out-sourced to a specialist software development company. The majority of staff involved transferred to the new company. Clients are allowed to make requests for changes to be made to the software. If it is decided to make the change, it will be included in the next release after completion. Release intervals are fortnightly with the original developers and four-weekly with the software house. The information obtained from these various stages were the date a CD was created (i.e. entered the system), along with a unique identifier. The dates that it entered various stages, allowed us to work out the time the CD spent in development and testing, as well as time spent waiting. The data can be shown as a time series, showing the number of changes incorporated into each release, in Figure 1. A median smoother has been applied. The cycles in the data indicate at least two feedback loops. The points above the peak of the fourth cycle correspond to the approximate time that the intention to outsource the product was officially announced. It can be seen that this period was one of apparently great activity in terms of output, possibly due to pressure to sign off as many CDs as possible prior to change over. The downturn in the fifth mini-hump approximates to the actual change over. Again there is some evidence of increased activity prior to this event. Outsourcing commenced after release number 88.
120 C o u n t
80 40
20
40
60
80
ReleaseNumber Figure 1:Count of Changes (CDs) per Release By Release Number When the cumulative total is plotted in Figure 2, we can see four corresponding curves in the line. The period covered by each curve or cycle is approximately 20 releases.
2
3000 c u m ∑ C n t
2000 1000
20
40
60
80
ReleaseNumber Figure 2: Cumulative Growth of System by Release Number
3. Simulating System Behaviour. As we have already indicated, the notion of feedback is a central one to system dynamics. It is also central to Lehman’s ideas [1] regarding the evolution of software systems over time and in particular that the growth of systems is self-limiting. It has also been proposed that this can be modelled using an inverse square law [12]. In this section we consider how these ideas can be applied to our case study. Recall that Figure 2 indicated the cumulative growth of the management information system over a period of about four years. We note that whilst the growth is monotonically increasing — as one might expect from a cumulative function of change requests — the growth is not exactly linear. Indeed there appear to be four quite pronounced segments of curves, each of which shows a tendency for the rate of growth to diminish. Lehman has argued that this is an example of a feedback system; the software grows in size and age and so changes become successively more difficult to make as evidenced by the flattening in the growth rate. It may be, however, that our case study exhibits a second feedback loop such that once the rate of change becomes so low the system maintainers are unable to provide the required level of functionality in a changing world. If this becomes the case the system owners have a variety of choices: the system could be abandoned; a significant level of additional resources could be made available; major architectural change could be made to the system or a more efficient maintenance process introduced perhaps using newer technology. In our model we discount the first option. The first stage of our investigation was to apply the single feedback model suggested by Turski [12] based upon an inverse square law. Unfortunately, when this particular formulation of a self-limiting system was applied to our dataset we observed an extremely poor fit, essentially very rapid growth followed by no growth whatsoever. This may in part be explained by the fact that our time series commences from zero, whereas the systems they studied had quite considerable initial sizes. We decided instead to use a simpler exponential model: Si = Si-1 + Ri; Ri = Ri-1e subject to the inequality 0
