Test Case Development During OO Life-Cycle and Evolution - CiteSeerX

Test Case Development During OO Life-Cycle and Evolution Ishbel Duncan, Malcolm Munro and Dave Robson Testing Research Group, Computer Science, University of Durham, DH1 3LE, UK

(Computer Science Technical Report 4/96)

Keywords: OO - Life-cycle - Testing - Test Data Abstract Testing is considered as integral to every phase of the OO life-cycle to ensure a well developed test suite covering requirements, design and structure. Release criteria between the phases of development incorporate test suite addition and documentation allowing traceability from requirements through coding to speci c test cases. Software engineers can pool their knowledge via the test suite and this expertise can be used for regression testing after system alteration or for re-use considerations.

1 Introduction Testing is a major problem in Object Oriented (OO) Code development. Much research eort has gone into design and requirements analysis but little has been expended on the mechanics of testing for product release, regression or re-use of code. This paper addresses a method of development which encompasses testing at all phases of a system life-cycle. If testing is performed as an integral part to development, test data can be created that is traceable to particular phases and can be used eciently for regression and acceptance testing. The next section outlines the current status of OO testing and is followed by a strategy for incorporating testing within the OO development life-cycle. An example of life-cycle based test case development is then given.

2 OO Testing Object Oriented (OO) systems have been promoted for ease of design, coding and re-use. Much research eort has been focused on design methods and process metrics [Henderson-Sellers, 1996, Rumbaugh et al., 1991, Booch, 1986] yet little has been expended on testing and maintenance aspects. It appears that OO is promoted for its ease of design and use implying that testing and maintenance are easier or cheaper than in traditional, 3GL systems. Now that industry have large scale OO systems comprising thousands of classes it is now apparent that the original claims of OO promoters are in doubt. It is debatable whether OO testing is or is not dierent from traditional testing in the main, what is not debatable is the perception of OO testing being much more dicult. The presence of inheritance and polymorphism will alter testing necessity and focus but whether traditional unit testing is dierent from class testing, or whether function integration and object integration testing are comparable, is the subject of other research [Duncan et al., 1996b]. However dierent the testing requirements or focus of application may be, it is true to state that the OO software industry is now facing major testing problems. This is the rst step to discovering problems with corrective and adaptive maintenance and regression testing. Industry has debated the worth of OO testing emphasis being placed at the unit level (classes) or at the system level [Graham, 1995]. There appears not to be any consensus as to which level achieves the better results in the form of error discovery. However, the lack of any recognised taxonomy does not help the process of error detection and assessment [Duncan et al., 1996a] The nature of OO systems, being formed from small interacting clusters of classes (components), appears to aid understanding but makes testing more dicult. This may be due to the dispersal of information across several classes and therefore makes control and data tracing more complex. It is possibly the eect of having many active objects at any one time as opposed to a function call graph, which also creates several active functions, that is confusing to coders and testers. We postulate that the domain boundaries between classes are likely to be of a higher dimension, once classes are integrated, and therefore perhaps more likely to engender domain and range `holes' where faults can occur [Beizer, 1990]. Hatton hints at this when he states that it is better to have fewer functions of moderate size than many of a smaller size [Hatton, 1995] and the postulate also explains why small routines are generally found to have many faults. There is certainly no standard methodology for the testing of OO code and any method must 2

incorporate strategies for class hierarchy testing and polymorphism. The hierarchical testing strategy of Harrold, McGregor and Fitzpatrick [Harrold et al., 1992] indicates the necessity of testing from a root class down the derivation tree to a leaf class. However, this strategy does not indicate what testing has to be performed for completeness. Similarly other papers in the eld indicate that testing must be thorough and complete but there is no consensus at to what this actually entails. Class hierarchies are an an important part of OO code, but some data shows that classes with a hierarchy greater than one exist for only 10% of the classes in a system. If this sample data is mirrored in most large scale OO code then it shows that although class hierarchy is important it is perhaps not the most expensive part of testing. Statement coverage, the most basic testing strategy that exists, is no more dicult than with traditional code and may be easier due to the small size of many class methods (this depends on the language and methodology used, but usually methods (functions) and much smaller than in 3GL systems). However, statement coverage is minimal and non sucient and it has been shown that many faults exist in the interactions between classes, i.e. at object integration level. These faults are unlikely to be detected at statement coverage level and so it is important to examine object to object calls, eectively thread testing. To assume that a class is safe [McCabe et al., 1994] because it has had a set number of test cases applied to it is a misleading test metric. Several test cases may nd no errors but a single, well developed test case can discover many faults. It is not the quantity of the test cases that are important but the quality of them. Consequently, a check on the adequacy of the test suite must be made. That is, the test suite should be analysed for its fault nding capabilities. It is assumed that OO system testing is no dierent from traditional system testing [Graham et al., 1993]. What is apparent from the literature is the fractured approach to testing OO systems and the lack of methodology. There is also no formalism for measuring the adequacy of the test suite in respect of its fault nding abilities. This however, is similar to traditional systems. The next section outlines a mechanism for involving testing at all stages of the OO life-cycle to emphasize the role played by testing in quality, re-use and maintenance.

3 Testing Within the Development Life-cycle For testing to be structured and focused, test cases should be developed alongside requirements, design and coding. Testing should not be envisaged as a distinct phase of the life-cycle but as an integral part to the design and build process. As requirements are laid out, test cases simulating common use cases or scenarios [Jacobsen et al., 1994] can be noted and referenced via those use cases. That is, there should exist traceability between the requirements and the test cases. As design progresses towards a lower level where classes, hierarchies, methods and components are in skeleton form, again test cases can be formed to mirror those design structures and analyse problematic boundary conditions or method calls. Also test cases should be developed to analyse the ordering of method and class initiation. If pre and post conditions are to be used then test cases should be developed to fall within the scope of the conditions and to fall outwith the scope. As with requirements centred test cases, design centred test cases should be marked and made traceable to particular design layers or components (small cluster of co-operating classes). By the coding stage, a programmer should have access to a suite of test cases outlining the functionality and to some extent, the structure of the desired system. A pre-prepared suite allows a programmer, and ultimately a tester and maintainer, to work eciently without having to generate a new bank of test data. Any tests developed can be added to the test suite and again marked as 3

associated with particular classes or clusters. By forming a test suite throughout system development, each specialist contributes to the testing phase and ultimately to the quality of the system. Test data is traceable and therefore modi cations allow a software engineer to determine which test cases are primarily aected. (There will be complications here because of data and class relationships.) As specialists are not recreating test cases for each phase of the development time can be better spent on analysing the system rather than checking it functionality. `Big Bang' testing is thus avoided by integrating testing throughout the development process. Testing occurs at dierent levels of granularity and faults in pre-code stages are discovered earlier than in traditional testing models. Sections of code that deal with the environment of the system can have associated test cases that are environment dependent. These will be duplicated for other environments the system may be ported to and would constitute a portability test suite for use in acceptance tests by the customer.

4 An Example of Developing with Integrated Testing To show how this strategy would work an example of a small scale development is demonstrated. On a very simplistic level, say the requirements for a small prototype banking system were given as A small banking system has 2 types of accounts - cheque and savings. Each account is subjected to a daily withdrawal of $250. The savings account has a daily interest accrued dependent on balance. This rate is dierent for balances below $1000 and over that amount. The cheque account may have an overdraft allocation of up to $200 if authorised by the bank.

4.1 Requirements Test Cases A rst inspection of these requirements would lead a designer to consider a base class of account with modi cations for interest accrued or overdraft allocation in derived classes of savings and cheque account. However, before the system can be passed into design, the requirements engineer should describe applicable test cases, either by using use cases or by some other technique. For example, use case number 1 may describe the creation of a cheque account with N pounds deposited and then N-1 pounds withdrawn before the account is closed. See Table 1. If N-1 is less than 250 then this should be a valid sequence of events. The next use case may describe a similar scenario where N-1 > 250 and indicate that the system should respond in such a way that the withdrawal is shown to be invalid. Both the above test cases should be marked as requirements pro ling and associated with cheque accounts. See rst two columns of Table 1. In a similar fashion, test cases can be developed for other scenarios. These will help indicate to the requirements engineer what responses the system must output to given inputs and conditions. In eect the requirements engineer does not change their function but is asked to provide test cases outlining expected scenarios and responses. It is up to the requirements engineer, together with the customer, to discuss what reponses should be expected when the sequences of events are altered when, for example, the accounts are subjected to closure before another deposit is requested or a cheque account is opened with a negative balance within the range of an allowed overdraft. Eectively the 4

Table 1: Requirements Test Cases

Phase & Case # Class/Component Req1

cheque withdraw limits

Req2

cheque withdraw limits

Req3

cheque overdraft vacility

Test Case

Result

create(account) Valid deposit(N), 0 < N  250 withdraw(N-1) close(account) create(acc) Invalid deposit(N), N = 251 > 250 withdraw(N) withdrawn close(acc) readin(acc(N)), N = 50 Valid withdraw(N+M), 0  M  200 if od alloc for 0  M  200 else Invalid

skills of the tester and requirements engineer are pooled early on in the system development and problem scenarios are possibly detected earlier in the life-cycle.

4.2 Design Test Cases As development progresses, the design specialist takes up the proposed system. It should be enforced that this does not happen until there are outline test cases. It is thus ensured that the designer has more detail to work with. As the design progresses towards a lower level in which classes are outlined with methods and attributes, more speci c, structural test cases may be added to the requirements test suite. These test cases should be marked as being derived from the design stage and may be more concerned with the mechanics of class interaction. For example, if contracts exist between two or more classes, test cases should be developed to describe expectation. In the banking example it may be that the savings account balance variable (attribute) must be non-negative. In this situation a pre-condition must exist on the savings account class that indicates that the balance attribute be greater than zero. At least three test cases must then be derived; one for a non-negative balance, one for a negative amount and one for zero. What happens when the balance is zero must be discussed as it may be harmless to calculate the interest on a zero balance (depending on the accuracy of your calculations or processor) but it is safer to trap this condition early. See Table 2. The way the system responds to the latter cases must be ensured to be acceptable. It is easier, and ultimately cheaper, to adapt the design at such an early stage of development than perform later maintenance on working code. In a similar fashion, test cases can be developed for other scenarios. A portability issue here may be the size of integers or oats associated with the account numbers, balances and interest accrued amounts. If the development system has oats of 6 Bytes then test cases should again indicate what is expected to happen if the oat attribute suers over ow. Other issues may be concerned with whether variables are integers or oats such as the amount for withdrawal. Is the withdrawal amount always an integer as in British ATMs or is the variable a oat as in Australian ATMs? The latter may allow for future extensions to the system wherein transfers of money or payments of bills may be 5

Table 2: Design Test Cases

Phase & Case # Class/Component Des1 Des2 Des3

Test Case init(acc(N)), N > 0

savings interest application savings interest application savings interest application

Result

Valid interest-accrued(acc(N)) add interest init(acc(N)), N < 0 Invalid interest-accrued(acc(N)) msg to user init(acc(N)), N = 0 Invalid interest-accrued(acc(N)) but acceptable

made. Designing with extensions in mind may also indicate test cases which cover the current status of the system and future options. With this example, two test cases should be derived which allow for integer withdrawal (always valid) and oat withdrawal (valid only when system extended).

4.3 Code Level Test Cases As each class is developed, the coder should associate a series of test cases, again labelled for reference by the class name. For example, on the bank account class the variable for withdrawal-today should be checked to make sure it is zeroed every day. Similarly, boundary conditions on the variables and predicates that deal with checking the withdrawal or overdraft limits should be analysed. In some cases, these test cases will duplicate some developed at design stage but it is important to keep both sets or mark a test case as being applicable to both design and coding because of future removal for redundancy. The coding level tests should include checks on the creation and destruction of objects. How many objects can be created? Does the system check for every object's destruction (by maintaining a simple count, for example)? What happens if an attempt is made to destroy an uncreated object? Class method ordering should also be analysed. Using regular expressions to describe the actions of a class [Beizer, 1990, Kirani and Tsai, 1994], test cases can be derived to cover a variety of situations. For example, the following test cases describe valid scenarios (where N and M are integers).   

create.open.deposit(N).withdraw(N-1).close.destroy create.open.deposit(N).deposit(M).close.destroy create.open.withdraw(0).close.destroy

The next cases describe scenarios which demonstrate the robustness of the system   

create.create.destroy create.open.destroy create.open.withdraw(N).close.withdraw(N).destroy 6

The test cases for coding should ensure 100% statement coverage as a minimum plus a random number of tests on the order of invocation of class methods. Tests should also be performed on the parameters passed to methods, again checking that minimal and maximal numerics, null strings etc. are catered for. These latter test cases are necessary for the next stage of system development and also for future regression tests. It is at this stage that metrics can be used to indicate how well developed the test suite is in terms of its coverage of statements, structures (loops, switches), segments (Linear Code Sequence and Jumps [Hennell et al., 1982]), equivalence partitions, data states etc.. The metrics chosen must re ect the strength of testing required. It has already been mentioned that to ensure statement coverage by way of applying a number of test cases equal to the cyclomatic complexity is non sucient. However, it should be considered a minimal requirement to have all statements executed. More rigorous testing comes at a price and it must be left to the developers to determine the criteria by which the code is released. If inheritance exists then unit level testing should incorporate the hierarchical strategy for class testing [Harrold et al., 1992]. Each class of the inheritance tree should be analysed for coverage and for random method invocation and parameter partitions. A measure of adequacy on the class based test suite should include statement coverage, structure coverage, some partitioning on parameters to methods and tests on inherited classes and polymorphic code.

5 Object Integration Testing At this stage of development, objects should be analysed for their ability to interact safely. Just as in object testing, method order invocation should be checked for correct processing both for valid scenarios (call orders) and for non-valid, either by requirements or design documents. Although other researchers have described the necessity of object integration testing based on call order of methods [Graham et al., 1993], it is equally important to analyse the parameters of those methods. It is because of parameter and data manipulation that statement coverage is considered non sucient. If objects are analysed via their states, the attributes or variables of the base class, then those states achieved under single object testing (class or unit testing) must also be achievable under multiple object testing. If states are unachievable under object integration then these states should be documented as dead code. It is theoretically possible to have 100% statement coverage under class testing but still have dead code under object integration testing. Strategies for the testing of clusters of objects are well known in the testing literature. Commonly a class which has no server classes is analysed prior to a class (object) which calls it. Problems arise when classes reference each other. In these cases, test stubs may have to be developed. Inheritance of classes and polymorphism also play a part in object integration testing. If methods exist with the same name but dier through inheritance or polymorphism, then again each must be checked for coverage and parameter partitions. As in non-polymorphic method invocation, it is possible that polymorphic methods may be uncallable due to data circumstances. Whether this is considered a feature, a bug or simply coding for future extensibility must be left to the system engineers. What is important, is that dead code is documented. As the system develops under successive iterations of the life-cycle, it should be expected that object integration testing is performed many times. It is thus essential that test cases are marked 7

correctly for redundancy removal and for regression testing. If a system is designed to be composed of many components, each made up of a small cluster of co-operating classes, then, the test suite may be physically large, but each sub-group applicable to components should be of a manageable size. A coder or tester then has a repository of test cases covering requirements, design and coding levels to work with. When classes or clusters are marked for re-use, the associated test cases can be placed in the re-use library alongside the code. Test cases should be documented to show features within the code, such as those pertaining to portability or reliability issues. Methods or attributes that have to be adapted for generalisation or extension should be marked as pertaining to maintainability. Marking features allows cross reference to quality issues such as those outlined in ISO/IEC 9126 the standard for quality characteristics [ISO9126, 1991]. Documentation is a necessary part of the test suite development process and should form part of the release criteria between phases of the life-cycle. That is, movement should not occur between say, requirements and design phases, until a test suite is built and documented. Metrics to manage the size and complexity of the test suite is a separate issue and is the subject of ongoing research.

6 Conclusion Testing should not be considered a single phase of the software engineering life-cycle, even within an interative development process. It is argued that testing should be integrated within each phase and test suite creation should form part of the release criteria between phases of system development. There is no current method(ology) to indicate how to apply testing strategies to OO systems and where to focus a test application. By developing test cases alongside requirements, design and coding the system features are essentially documented and analysed during development. The tester and other software engineering specialists can thus share essential information through the test suite medium. Design and requirements aws may be found earlier in the life-cycle and because of the parallel development of test cases, each class is analysed before integration into the system. Therefore, class testing is performed as part of the system development allowing testers to focus on object integration analysis, commonly known to be a source of many faults. As testing is performed methodically throughout the system development the testing bottleneck now found in industry will be reduced. Arguments as to whether testing should be focused on unit (class) or system testing can now be avoided by insisting on constant test application throughout the life-cycle. Integrated testing contributes to maintenance, re-use and evolution issues by pooling specialist information throughout the design and build process and forcing documentation of system features. By enforcing release criteria between the development phases to incorporate test suite creation, addition and documentation, a system can be tested more eciently throughout the build process.

References [Beizer, 1990] Beizer, B. (1990). Software Testing Techniques. Van Nostrand Reinhold, 2nd edition. [Booch, 1986] Booch, G. (1986). Object-Oriented Development. IEEE New York. [Duncan et al., 1996a] Duncan, I., Robson, D., and Munro, M. (1996a). Defect Detection in Code. Computer Science Technical Report 2/96, University of Durham. [Duncan et al., 1996b] Duncan, I., Robson, D., and Munro, M. (1996b). Dierences and Similarities between Testing OO and Procedural Code. Computer Science Technical Report 5/96, University of Durham. [Graham, 1995] Graham, D. (1995). Testing Object Oriented Systems. UNICOM Seminar Series.

8

[Graham et al., 1993] Graham, J., Drakeford, A., and C.D.Turner (1993). The veri cation, validation and testing of object-oriented systems. BT technology Journal, pages 79 { 88. [Harrold et al., 1992] Harrold, M. J., McGregor, J., and Fitzpatrick, K. (1992). Incremental Testing of ObjectOriented Class Structure. In 14th Int. Conference on Software Engineering. [Hatton, 1995] Hatton, L. (1995). Bugs: avoiding the avoidable and living with the rest. In Procs. of Durham 95, Ninth European Workshop on Software Maintenance. [Henderson-Sellers, 1996] Henderson-Sellers, B. (1996). Object Oriented Metrics. Prentice Hall. [Hennell et al., 1982] Hennell, M., Hedley, D., and Woodward, M. R. (1982). The path implications of a hierarchy of coverage measures on program testing. Technical report, Dept of Computational Science, University of Liverpool. [ISO9126, 1991] ISO9126 (1991). ISO/IEC 9126 Information Technology - Software product evaluation - Quality characteristics and guidelines for their use. [Jacobsen et al., 1994] Jacobsen, I., Christerson, M., Jonsson, P., and Overgaard, G. (1994). Object Oriented Software Engineering - A Use Case Driven Approach. Addison Wesley, 4th edition. [Kirani and Tsai, 1994] Kirani, S. and Tsai, W. (1994). Speci cation and Veri cation of Object Oriented Programs. Technical report, Computer Science, University of Minnesota, University of Minnesota, Minneapolis, MN 55455, USA. [McCabe et al., 1994] McCabe, T., Dreyer, L., Dunn, A., and Watson, A. (1994). Testing an Object-Oriented Application. The Quality Journal, pages 21 { 27. [Rumbaugh et al., 1991] Rumbaugh, J., M.Blaha, Premerlani, W., Eddy, F., and Lorenson, W. (1991). Object Oriented Modelling and Design. Prentice Hall.

9