2014 Fourth International Conference on Communication Systems and Network Technologies
Test Case Generation for Object-Oriented Systems: A Review Rajvir Singh Department of Computer Science & Engineering, Deenbandhu Chhotu Ram University of Science and Technology, Murthal, Sonipat, Haryana(India):131039 e-mail:
[email protected] ¾ Performance requirements are required as it is specified in the specification document. ¾ Software reliability and software quality based on the data collected during testing. The various advantages of testing are as follows: ¾ Increased accountability and control ¾ Cost reduction ¾ Time reduction ¾ Defeat reduction ¾ Increasing productivity of software developers
Abstract— Testing is as good as its test cases. It means testing is highly dependent upon faults detection ability of test cases. Test case generation plays a vital role in testing process and is main area of research in the field of software testing. The good test cases decrease the chances of failure of the system and ensure the quality of the system. This paper presents the survey of the various test case generation techniques / methods for object-oriented systems in more comprehensive way. Keywords - Software Testing, Object-Oriented System, Test Case, Test case classes, Sequence diagram, UML diagram, Use case diagram, object-oriented net.
I.
Object-Oriented Technology and Software Testing: Object Oriented paradigms provide a number of ways to permanently alter the software engineering field, capturing it into the realm of true elegant design. Object oriented paradigms have taken the software evolution as a means of managing divergent complexities of development. The challenges through OO modeling/ programming incorporating design paradigms are making head way for developing robust, reliable and maintainable software. The program code fragment can be written after that can be subjected under test and it can also be modeled for reusability as a design process. The dynamic behavioral modeling implementing the state models using OOAD and UML are most popular now with wider acceptance. It is widely accepted that object-oriented (O-O) paradigm significantly increase the software reusability, extendibility, inter-operability and reliability. This is also true for high assurance system engineering, provided that the system are tested adequately. Object-oriented system testing (OOST) is an important software quality assurance activity to ensure that the benefits of object-oriented(O-O) programming will be realized. Object-Oriented Software Testing(OOST) deals with problems introduced by Object-Oriented features e.g. encapsulation, polymorphism, inheritance and dynamic binding. It is the discussion about different levels of testing associated with object-oriented programs[22].
INTRODUCTION
Software testing plays a vital role in system development life cycle. It is an investigation that is to be conducted for providing information about the quality of the software product. According to IEEE testing is, “The process of exercising or evaluating system or system components by manual or automated means to verify that it satisfies specified requirements”. Software testing is an attempt to dry out a program with well designed input data with the intent of finding failures. In other words, “Testing is the process of executing a program with the intent of finding errors”. Testing identifies faults, whose removal increases the software quality by increasing the software’s potential reliability. Testing also measures the software quality in terms of its capability for achieving correctness, reliability, usability, maintainability, reusability and testability. Generally the testing process consists of three things: (i) Test Case Generation, (ii) Test Case Execution and (iii) Test Case Evaluation. “The test case generation process plays the vital role among the three cases.” The various objectives of testing are as follows: ¾ Testing is a process of executing a program with intent of finding an error. ¾ A good test is one that has a high probability of finding an as-yet-undiscovered error. ¾ A successful test is one that uncovers an as-yetundiscovered error. ¾ Testing should aim at suggesting changes or modification if required, thus adding value to entire process. ¾ The objective is to design tests that systematically uncover different classes of errors with a minimum amount of effort and time. 978-1-4799-3070-8/14 $31.00 © 2014 IEEE DOI 10.1109/CSNT.2014.201
Intra-method testing: Tests designed for individual methods. This is equivalent to unit testing of conventional programs. Inter-method testing: Tests are constructed for pairs of method within the same class. In other words, tests are designed to test interactions of methods. Intra-class testing: The Intra-class tests are designed and constructed for a single whole class, usually as sequences of calls to methods within the class by different methods of that class. 981
their later requirement. The designing and generation of test cases requires: (a) Domain knowledge (b) Technology knowledge and (c) Testing best practices. Based on the literature survey and software testing best practices, there are thirteen classes of test cases as given:
Inter-class testing: It is meant to test more than one class at the same time. It is equivalent to integration testing. The above mentioned first three variations are of unit and module testing type, but inter-class testing follows the integration testing approach. The overall strategy for objectoriented software testing is identical to the one applied for conventional software testing but differs in the approach it uses. Testers begin testing in small and work towards testing in the large. As classes are integrated into an OO architecture, the whole system is tested to ensure that errors in the requirements are uncovered. Test Case: It is a set of test inputs, execution conditions and expected results developed for their objectives. A test case is a single test instance in terms of input data, test execution conditions and expected outcome. It determines whether the feature of an application is working correctly. Test cases also reference test objectives such as verifying compliance with a particular requirement or execution of particular program path [26, 27]. So Test Cases should be written in enough detail that they could be given to a team member who would be able to quickly start execution and find defects.
Sl. No. 01 02 03 04 05 06 07 08 09 10 11 12 13
Test Case Areas Aesthetic/Visual Validation Navigation Usability Data integrity/ Data base Modes(Add/Edit/Delete/Read-only) General Data field Numeric Field Alpha Field Checks Validation testing on every screen (Add, View, Modify, Delete, Cancel) Security Coverage (Web Session, Cross Site Scripting) Performance Coverage
Table II
Table I
Test Case Classes [25]
One important aspect of software testing life cycle is the generation/creation and management of test cases repository which is an independent area of work irrespective of different Quality Assurance methodologies followed by the testing team. Such a repository is at C-DAC centre, Noida, U.P.(India) [25]. Test Adequacy Criterion: It is well known fact that testing makes the software error free and increases the reliability. But it is not known when to stop the testing process or what constitutes the adequacy of a test. Goodenough and Gerhart [28] made an early stuck to this. The test adequacy criterion is nothing but an essential and important predicate, which shows the adequacy. In other words it is a stopping condition to stop the testing process, when all the defined criteria have been met [29]. Branch coverage, path coverage, transition coverage, activity coverage are few of such test adequacy criteria. Different types of coverage criterion for testing objectoriented systems are [23]: (i) State Coverage Criterion: The value of state coverage is the ration between the covered states and all the states in the statechart diagram. It requires that all the state nodes in a state chart graph to be covered at least once. (ii) Action Coverage: To generate test cases with respect to action coverage we construct as many marked specifications as there are actions within the specification. (iii) Transition coverage: Given a graph and a Test Suite TS, s is said to achieve transition coverage, if it caused each transition t of the state graph to be exercised at least
Test Case Template[56]
The effective test case writing is gradually learning process and hence it is a skill and which can be achieved by experience and in depth study of the application on which test cases are being written. Like domain testing or riskbased testing, test cases are created according to certain test styles. Based on different domain/OOS needs, instead of writing separate test cases, it’s better to collate the test cases for different classes in order to modify them according to
982
year 1991, adequate test suites containing redundant tests and compared the fault finding of reduced sets to the full test sets. In their experiment, it is found that the fault– detection capability was compromised significantly when the test-sets were reduced. There was little correlation observed between the test-set size and fault finding capability. Michael Scheetz, Anneliese von Mayrhauser, Robert France, Eric Dahlman & Adele E. Howe [36] presented an approach to generate black-box test cases from conceptual models of the system. The models they discussed are different from traditional design and requirements models. In these models they include information relevant to test case generation. They also provide a procedure to obtain test objectives from these models. Y. G. Kim, H. S. Hong, S.M. Cho, D.H. Bae and S.D. Cha[1] introduced a method for generating test sequences using UML state chart diagrams. They transformed the state chart diagram into a flattened structure of states called EFSMs and testing trees. From the testing trees, they listed possible event sequences which they considered as test sequences. The testing criterion they used to guide the generation of test sequence is the coverage of states and transitions of testing trees. For object-oriented systems, they also proposed a method of generating test cases for class testing using UML state chart diagrams. They transformed state charts to extended FSMs (EFSMs) to derive test cases. The hierarchical and concurrent structure of states is flattened and broadcast communications are removed in the resulting EFSMs. Then data flow is identified by converting the EFSMs into flow graphs and to these flow graphs the conventional data flow analysis techniques are applied. MIAO Huaikou and LIU Ling[2] proposed a test class framework using concept of object-oriented approach for generating test cases from Z specifications. They presented Test Case Generation system (TCGS). The future scope include the analysis of effectiveness of the tool and corresponding improvement. Offut and Abdurazik [33] and [34] proposed a technique for generating test cases from UML state diagrams. They have highlighted several useful test coverage criteria for UML state charts such as: (1) full predicate coverage, (2) transition coverage etc. Vincenzo Martena, Alessandro Orso & Mauro Pezze[3], proposed a new inter class testing technique of OO software. They used data flow analysis technique for deriving a set of test case specifications for inter-class testing. In this, the effectiveness of the technique developed is measured by driving test cases for a microscope controller developed for European Space Laboratory of the Columbus Orbital Facility. This technique is comprised of two phases; first generation of test case specification through symbolic execution and automated deduction. The requirement of the code is the limitation of this technique as it uses structural testing approach. Although the technique developed is powerful enough but still the efficiency can be increased by considering more factors in to consideration of study. Paolo Tonnela[4] proposed the method of generating test cases using the GA and chromosome syntax for Java source
once. The value of transition coverage is ratio between the checked transitions and all the transitions in the state chart graph. (iv) Transition Path Coverage: Given a graph and a Test Suite TS, s is said to achieve transition path coverage , if it causes each elementary transition path p of the state graph to be exercised at least once. The value of path coverage is ratio between the traversed paths and all the paths in the graph. (v) Condition coverage: a single condition is covered, if it evaluates to both true and false at some point during test execution. Decision coverage has also been called branch coverage or predicate coverage. This means that 100% condition/decision coverage is achieved if all condition evaluate to true and to false during the test execution. A decision consists of conditions separated by logical operators (e.g. and. or ). A test adequacy criterion helps in defining test objectives or goals that are to be achieved while performing a specific software testing. For example, branch coverage requires that every branch in a program under test is to be exercised by at least one test case. During test case generation the test adequacy criterion is important factor to consider for generating test cases. The components of UML sate chart diagram are: State, Initial State, Final State, and Transition[23]. II. LITRATURE REVIEW There are various research papers / reports which are presented by many authors for test case generation and fault detecting capability of test set for object-oriented systems as discussed below. W. Eric Wong, Joseph R. Horgan, Aditya P. Mathur and Alberto Pasquini[24] presented the question of the effect on fault detection of reducing the size of a test set keeping coverage constant. They randomly generated large collection of test sets that achieved block and all-uses data flow coverage of each subject program. They created a minimal subset for each test set that preserved the coverage of original set and then the fault finding capability of the reduced test-set has been compared to that of original test set. The data they used shows that the minimization keeping coverage constant results in no reduction in its fault detection effectiveness or little reduction in fault detection effectiveness. This observation leads to the conclusion that test cases that do not contribute to additional coverage are likely to be effective in detecting additional faults. Thierry Jeron, Jean-Marc Jezequel, Alain Le Guennec[35] described the framework of a continuous validation process that can be set up to move from the OO analysis to the OO implementation of a validated distributed system. Further their work can be extended to present validation framework to deal with higher level interactions between objects in distributed systems. G. Rothermal, M. Harrold, J. Ostrin and C. Hong [30] performed a similar experiment using seven sets of C programs with manually seeded faults to establish the correctness or disprove the results in the Wong study. For their experiment they used edge-coverage [31] given in the
983
different authors can test and compare their techniques which make it difficult to evaluate the performance of a novel technique against existing ones, apart from random search usually there are no common comparisons between different optimization algorithms, there is no sufficient theoretical work for OOS, all articles are of empirical in nature. Jaturat J. and Taratip S.[12] proposed an approach to find the test order for classes under test using class slicing technique. For improvement in the algorithm use of message sequence diagram is proposed in concluding remarks. Monalisa Sarma & Rajib Mall[10] and Monalisa Sarma, Debasish Kundu & Rajib Mall[11] proposed a novel approach for generating test cases from UML design diagrams. They considered use-case and sequence diagram in test case generation scheme. In this approach they converted SUT(System Under Test) into System Test Graph(STG) which is an integration of Use-case Diagram Graph (UDG), sequence diagram(SD) into sequence diagram graph(SDG) and then integrate UDG and SDG to form STG. Information necessary for test case generation is pre-stored into this graph. This information is retrieved from extended use-case diagram also called use-case template class diagram and data dictionary expressed in the form of object constraint language (OCL) which are associated with the UML diagrams. The graph obtained is traversed to generate test cases automatically based on coverage criteria and a fault model. They have used two test criteria: first, all use-case and all use-case dependency relations criterion and second, all sequence diagram message path sequence coverage criterion. Zhe Li & Tom Maibaum[37] presented a novel approach for testing of O-O programs at the integration level. They used UML sequence and class diagrams for test case generation and to implement these test cases the concept of coordination contracts is used. The future scope includes improvement in efficiency in generating test cases and checking the applicability of the approach for larger systems in the present scenario of the software industry. Hang Zhou, Zhiqiu Huang & Yi Zhu[13] proposed an approach to formalize polymorphism sequence diagrams and generate test data. They presented the three case studies in their work: (i)Supermarket Cashier System (SCS), (ii) Billing System of Grant Wall Broad Band Network (BSGWBN) and (iii) Company Business Management system (CBMS). In their study the path based coverage criteria are used. Future scope includes optimized test case generation for large OOS like these systems. Alessandro Oliveira Arantes, Nandamudi Lankalapalli Vijaykumar, Valdivino Alexandre de Santiago Junior & Danielle Guimaraes[38] proposed modeling issues in using state charts to represent the specification of space application software from which test sequence can be generated. The future scope includes studying integration between Web-Perform Charts and Tools for automatic test execution to improve test automation process. Nirmal Kumar and Dr. Mukesh Kumar [39] proposed an approach using Genetic Algorithm for the generation of test cases for OO programs. The encoding and decoding of test
code of CUT (Class Under Test). The tool used is eTOC (evolutionary Testing of Classes). The future work includes investigation of alternate coverage criterion and the extension of the proposed methodology as described in [39]. Mats P.E. Heimdahl & Devaraj George[5] proposed model checkers for finding test cases by formulating a test criterion as a verification condition for model checker. The tools used are NIMBUS tool-set which is an education and analysis environment for RSML-e and is used to translate to the input language model. The model and trap properties are given as input to the tool NuSMV. Test case minimization with the goal to find the minimal test suite satisfying the requirements is in general a NP problem, but often greedy heuristic suffice to generate significantly reduced test-suites. The future scope includes more robust techniques can be used for the above test-suite generation process. Li Dan, & Bernhard K. Aichernig[6] proposed combining of algebraic and model-based test case generation techniques, in this RSL tool is used for the purpose. Future scope includes practical tools development for test case generation using this hybrid technique and to improve its efficiency. Philip Samuel, Rajib Mall & Sandeep Sahoo[7] proposed a method to generate test cases from UML sequence diagram using slicing technique, Test case generation based on other UML models and their hybrid techniques can be investigated for the applicability and the effectiveness of the system. Clemuntine Nebut, Fraank Fleurey, Yves Le Traon & Jean-Mark Jezequel[8] proposed an automatic test case generation by using UML use-case model driven approach, they used statement coverage criterion for generating tests from a formalization of the requirements of the OO embedded software system. Their main goal in this paper is to generate test cases automatically which in turn reduces the human effort. The automatic test case generation is the hot topic of research these days and various researchers are trying to achieve this goal with more accuracy and effectiveness. Philip Samuel, Rajib Mall & Pratyush Kanth[9] presented the method for automatic test case generation from UML communication diagrams. In this method they generated cluster level test cases based on UML communication diagrams; the post order traversal is used to carry out for the constructed tree for selecting conditional predicates from communication diagrams. The message path coverage and boundary coverage criteria are achieved by test cases generated in this method. UTG-UML test case generator tool is used for generating test cases for swing component of Java. UTG stands for UML behavioral Test case Generator. The future Scope of this includes hybrid techniques for investigation for other test case criteria based on domain specific and other UML models. A. Arcuri and Xin Yao [44] proposed the state of art for test data generation for OO systems and discuss issues in OOS testing, These issues are: little work is done using optimization algorithms, empirical tests have always been done on very small clusters which reduces the reliability of the results, there is no common benchmark cluster on which
984
generalization. There are to consider other relationships also between classes e.g. association and aggregation the extended Petri nets can be used in future. Hojun Jaygarl, Kai Shin Lu & Carl K. Chang [16] proposed GenRed:A Tool for generating and reducing OO test cases that utilizes three approaches : input on demand creation and coverage based method selection techniques that enhance Randoop, a state of art feedback directed random testing technique, and a sequence based reduction technique that removes redundant test cases without executing them. Nagendra Pratap Singh, Rishi Mishra, Mrinal Kanti Debbarma & Sanjay Sachan[17] proposed the review: Life Cycle of Object-Oriented Software Testing (LOOST), it considers the following OO testing techniques: Boundary Value Testing, Class Testing, Class Integration Testing, Coverage Testing (path coverage and code coverage), Regression Testing, Stress Testing, Black-Box Testing, White Box Testing, Usage Scenario Testing and User interface testing. As Life Cycle is a continuous process so in future further review may be needed to enhance the life cycle of product to make it more reliable. M. Prasanna, K.R. Chandran, & K. Thiruvenkadam[32] proposed a technique for generating test cases using collaboration diagrams. They converted the diagram into an intermediate graph and from the graph, by applying Prim’s and Dijkstra’s algorithm, generated a set of test cases. Ranjita Kumari Swain, Prafula Kumar Behera & Durga Prasad Mohapatra[23] discussed the relationship among coverage criteria, test suite size, and fault finding capability that clearly needs more study. First they are not studying testing of traditional programs / procedural programs. They generate test cases for Object Oriented software. Yury Pavlov & Gordon Fraser[18] proposed semiautomatic search based test generation technique to achieve high code coverage using the search based tool EvoSuit[45]. As EvoSuit currently supports branch coverage and mutation testing as test objectives, the future scope includes further criteria such as criteria based on data flow and more readable test cases generation. Pietro Braione, Giovanni Denaro, Andrea Mattavelli, Mattia Vivanti & Ali Muhammad[19] proposed an industrial case study of the effectiveness of test generators as [44] presented this issue in OO software testing. Automatic test generators assume some type for systematic coverage of the program code or heuristic sampling of the program inputs. Strengths and weaknesses of test generators are evaluated by evaluation of assumption made by various authors in literature. Mathias Landhauber & Walter F. Tichy[20] proposed a method for automated test case generation by cloning technique. Test cases are often similar, a preliminary study of eight open-source projects found that on average at least 8% of all test cases are clones; the maximum found was 42%. Reusing test cases saves work of the tedious kind and time. For using cloning technique approach practically much of the research remains that can be done by the researchers i. e. the future scope of exploring the automated test case generation by cloning technique is still open ended.
program in to changeable data structure are discussed. Future scope includes enhancing the efficiency for encoding Java and other programs for test data generation. J.J. Gutierrez, M.J. Escalona, M. Mejias, I.Ram & J. Torres[40] proposed an approach based on MDE (Model Driven Engineering) paradigm for the synthetic generation of system test cases from function requirements. The approach they used is integrated in the enterprise environment. The approach is integrated on Navigational Development Tool (NDT)-Suit, future research in this direction includes research in new tools possibilities which is interesting. Anthony J. H. Simons & Wenwen Zhao[41] presented JWake Tool suit that is feedback based code exploration to learn the algebraic specification of test class with hints from the programmer. They compared three different pruning rules for test set reduction in their study. Role of algebra in test prediction is presented. Future research in this direction includes studying the role of algebraic structures in object oriented test case generation. Wei Han, Na Zhang & Zuohua Ding[42] proposed a new method to generate test cases based on Object Oriented Net(OON) model. They described initially OON then move to Z notated schema, four schemes are discussed: Port, State, Node and Connection. And they also presented a case study of a user management system for three servers: Web server, logic server and a database server. In future, as OON can be used for designing the system and its refinement, it can help in testing from code implementation based on an extended OON model for effective and efficient test case generation. Aparajita Rao, Kavitha Elizabeth George, G. Logeshwari, S. Viveka Katherine & T. Mythili[43] proposed a model for the development of a tool for test case generation. They tested this tool for a partial set of inputs. Future scope includes more mutant addition in their tool and checking the strength of the test cases. Fanping Zeng, Zhide Chen, Qing Cao & Liangliang Mao [14] proposed a novel method of OO test case generation based on UML state diagrams and and LTS (Label Transition System) and presented a result of test cases in three steps: (i) Modeling the software system by UML state diagrams. (ii) Transforming UML state diagrams in to LTS. (iii) Testing the program by test data generation. State diagrams provide a solid basis for test cases generation in a form that can be easily operated. LTS is an internal model to precisely represent the state transition. From LTS test cases are generated by dividing the procedure in two steps: first mapping state diagram to LTS and second generating test cases from LTS. Future Scope includes application of the method in GUI and other kind of testing. Esmaeil Mirzaeian, Homayun Motameni, Samad Ghaderi Mojaveri & Ahmad Farahi[15] proposed a technique for generating the test case by using Colored Petri Nets (CPN). UML state charts are used and object tokens are parsed by using some designated rules observed for
985
dynamic operation, the generated test cases are divided into different partitions. Furthermore, on the basis of fitness dynamics corresponding to the paths, the number of test cases generation can be regulated to automatically produce test cases. In their experiments they show that this method can produce test cases more quickly and it has improved the testing efficiency. Nicolas Frechette, Linda Badri, andMourad Badri (2013)[53] proposed the structural test case generation technique. In their study they have presented: (i) a literature-survey of 159 published articles, and (ii) an empirical study on 16 open source applications. Their study focuses mainly on bounded iteration and search-guiding heuristics. These frequently and widely used techniques to deal with looping problems. They identified the challenges that compromise the effectiveness of these techniques of looping. They also provided the guidelines for addressing these challenges. These predictions can be used for future research in handling similar kind of looping problems for generating structural test cases. Praveen Ranjan Srivastava et al.(2013)[54] proposed the test optimized case generation using firefly algorithm. The future scope includes the improvement in the test suite generated and applying the method on practical test case generation in software industries. Stefan J. Galler & Bernhard K. Aichernig. (2013)[55] presented a survey on test data generation tools-an evaluation of white- and gray-box testing tools for C#, C++, Eiffel, and Java which is helpful in selecting the tool appropriate for our testing from the existing number of testing tools. Several researchers have been successfully proposed test case generation methods for various software under various circumstances as discussed above, main categories of test case generation techniques are:
The existing test cases can be obtained from the central repository of test cases maintained at Noida India[25]. This is still open area for lot of research work in future. Jirawat Chaiareerat, Peraphon Sophatsathit & Chidchanok Lursinsap[21] proposed dynamic test case generation approach for OO programming classes using evolutionary search to find test cases that will satisfy branch coverage criteria. They presented test case generation framework. In future the methods can be investigated to reduce the search space for the solution of problem. Narayana Maruvada[46] introduced the important aspects that are to be kept in mind by the test case designer while designing the test cases. He listed the important questions and their corresponding type of testing technique used. Muhammad Dhiauddin Mohamed Suffian & Fairul Rizal Fahrurazi[48] proposed the Designing the Performance Test Cases: An Experience from Testing the Cloud-Based System. The cloud bases systems are drawing attention of researchers due to their number of advantages over traditional systems. In their study the authors presented the logical architecture of the system under test and designing the performance test cases are discussed based on the performance criteria set during the planning and analysis stage of the system design. Liping Li, Xingsen Li, Tao He & Jie Xiong (2013)[50] proposed Extenics-based Test Case Generation for UML Activity Diagram. In their paper they considered the concept of combining software testing with Extenices and proposed an automatic approach to generating reduced test cases from UML activity diagrams partly using Extension Theory. They tried to formalize the activity diagram using ndimensional matter-elements. To increase the faults detection by minimized test cases, they transformed the activity diagram to the Euler circuit based on transformation theory and then test cases are generated automatically by Euler circuit algorithm. They only focused on the basic test coverage criteria like activity state coverage and transition coverage but in future work it is considered that how to generate test cases which satisfy more test coverage from UML diagrams. Noura Ouerdi, Mostafa Azizi, Jean-louis Lanet, Abdelmalek Azizi, & M’hammed Ziane. (2013)[51] proposed EMV Card: Generation of Test Cases based on SysML Models. The security of the system is dependent on the factor that how secure the test cases for the system are designed. In this paper they presented the generation of secure test cases for EVM card. Zhang Junli and Song Limin[52] suggested that with the automation in software testing the testing time and cost increases. In their paper they presented the testing division of OO program and the testing methods used in different periods. They also described the class service testing technology and the automatic testing technology. The structural testing refers to the path coverage criteria. They proposed that the path coverage testing samples can be artificially produced and also be automatically generated. Automatic generation of testing samples can be constructed with genetic algorithm. According to the triggering path in
Scenario-Based Model-Based Path-Oriented Goal-Oriented Genetic-Approaches Scenario-Based Test Case Generation: these are based on concurrent approach with concurrent coverage criteria [8][48]. Model-Based Test Case Generation: Model-Based technique identify respective test case for the software with respect to the UML diagrams such as Activity, state chart, and object diagrams etc. [5][6][10][22][29][36][40][43][47]. The model-based approach draws main concern of researchers. They consider use-case and sequence diagram in test case generation scheme. In this approach they convert SUT (System Under Test) in to System Test Graph(STG) which is an integration of use case and sequence diagram, use case diagram(UD) is the first converted to Use-case Diagram Graph (UDG), sequence diagram (SD) into sequence diagram graph (SDG) and then integrate UDG and SDG to form STG. The graph is traversed to generate test cases automatically based on coverage criteria and a fault
986
x Scenario-Based Test Case Generation: In the literature researchers mainly focused on concurrent process is only in activity diagram which can be extended by exploring more options available. So the research scope in this direction is still open. x Model-Based Test Case Generation focused on various state chart diagrams, sequence diagrams, object diagrams and use-case diagrams to generate test case but fails in obtaining optimal one. The optimal ones can be obtained by exploring the combination of techniques which are not explored. x Genetic–Based Test Case Generation helps to produce an optimal test case sequence, it further can be extended for model based techniques. In recent trend, Model-Based Test Case Generation techniques attract many researchers to research by using some data mining concept to produce optimal automated test cases to reduce cost and effort needed.
model. Information necessary for test case generation is prestored into this graph. This information is retrieved from extended use case diagram also called use case template class diagram and data dictionary expressed in the form of object constraint language (OCL) which are associated with the UML diagrams. Teemu Kanstrén[49] suggested that model-based testing is used for testing a lot of variation in the current version of the software and a lot of interactions among modules. Model based testing is reliable for to automated systems. Path-Oriented Test Case Generation: it is based on static as well as dynamic control flow of the software. Static path testing is done by symbolic execution and dynamic path testing in based on run time test of execution program [26][27][29]. Test adequacy criteria are nothing but an essential and important predicate, which shows the adequacy. In other words it is a stopping condition to stop the testing process, when all the defined criteria have been met. Goal-Oriented Test Case Generation: in these test cases cover predefined/selected goal such as a statement or branch coverage irrespective of the path coverage. Genetic-Approach based Test Case Generation: these test case generation techniques use genetic algorithm (evolutionary approach) for generating test cases for software systems such as automatic test case generation for UML object diagrams using genetic algorithm by M. Prasanna and K.R. Chandran & K. Thiruvenkadam[32] and N.K. Gupta & M.K. Rohil[39]. To sum up, there are many techniques available for generating test cases to satisfy test coverage criteria. A random test case generator may create many test data; but might fail to find test case to satisfy requirements. A path oriented approach identifies path for which test case has to be generated, however the path might fail to find an input that will traverse the path. It is quite complex to use an intelligent approach for generating test case quickly. Comparing with these techniques, Model based testing is a valuable one, since it creates flexibility, useful test automation from practically first day of development. Models are simple to modify, generate innumerable test sequences and allow the testers to get more testing accomplished in shorter time. Even though varied test case generation approaches are available, model based testing approach has attracted many researchers and still research is being carried out to optimize the generation of test cases with minimum human effort. The intelligent algorithms can be used for this purpose by finding the ways for their use in different situations (cases). Nowadays, the researchers are actively involved in test case generation with reduced test suite size with the help of various techniques available in the literature like swarm intelligence, BFO method etc.
IV.
CONCLUSION AND FUTURE WORK
The various techniques and /or methods are presented by various researchers for generating test cases to satisfy various coverage criteria like path coverage etc. This paper shows that there is huge scope to conduct research in the field of test case generation mainly for object oriented systems. The research gaps discussed can be refined / narrowed by selecting sub-topics for the research. For our future plan, there is to propose a novel technique to overcome the explored shortcomings in the existing systems and then to evaluate the performance of the proposed system. ACKNOWLEDGMENT It is our sincere thanks to the friends and senior faculty members of Computer Science and Engineering department at DCRUST, Murthal, India who helped us by giving their moral support. REFERENCES [1]
[2]
[3]
[4] [5]
III. RESEARCH GAPS There are various techniques proposed by various researchers available for generating test cases for OO systems to satisfy test coverage criteria. The research gaps are as under:
[6] [7]
987
Y. G. Kim, H. S. Hong, S.M. Cho, D.H. Bae and S.D. Cha,”Test Case Generation from UML State Diagrams”, IEEE proceedings-Software, Aug. 1999, Vol. 146, No. 4, pp. 187-192. MIAO Huaikou and LIU Ling,” A Test Class Framework for Generating Test Cases from Z-Specifications”, IEEE proceedingsSystems and Software, 2000, pp. 164-17. Vincenzo Martena, Alessandro Orso and Mauro Pezze,” Inter Class Testing of Object Oriented Software:, Technical Report GIT-CC-0228,May 2002, pp. 1-26. Paolo Tonella,” Evolutionary Testing of Classes”, ACM proceedings, ISSTA’04,July 11-14,2004,Boston, Massachusetts, USA, pp. 119-128. Mats P.E. Heimdahl and Devraj George,” Test suire Reduction for Model Based Tests: Effects on Test Quality and Implications for Testing”, IEEE proceedings of 19th International Conference on Automated Software Engineering(ASE’04),2004. Z. Liu and K. Araki,” Combining Algebric and Model-Based Test Case Generation”, ICTAC-2004, LNCS 3407, 2005, pp. 250-264. Philips Samuel, Rajib Mall and Sandeep Sahoo,” UML Sequence Diagram Based Testing Using Slicing”, IEEE Indicon2005conference, Chennai, India, 11-13 Dec. 2005, pp. 176-178.
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25] Umesh Kumar, Ashitabha Panda and Bipin Bedi,”Central Repository of Test Cases”, proceedings of ASCNT – 2010,, C-DAC, Noida, India, pp. 200-205. [26] Standard for Software Test Documentation. ANSI/IEEE STD 8291983 [27] Cem Kaner, J.D., “What Is a Good Test Case?”, Florida Institute of Technology Department of Computer Sciences, May-2003. [28] J.B. Goodenough and S.L. Gerhart,”Towards a Theory of Test Data Selection”, ACM New York digital library, proceedings of the international conference on Reliable Software, 1975, pp. 493-510, DOI = 10.1145/390016.808473. [29] A. Andrews, R. France, S. Ghosh and G. Craig,”Test Adequacy criteria for UML design models”, Software Testing, Verification and Reliability, Wiley International American Journal of Scientific Research, Vol. 13, pp. 95-127, 2003, DOI: 10.1002/stvr.270 and Subsequent release 2012, ISSN 1450-233X Issue 42(2012), pp. 7284. [30] G. Rothermal, M. Harrold, J. Ostrin and C. Hong,” An empirical Study of The Effects of Minimization on the Fault Detection Capability of Test Suites”, IEEE Journal’s publication of proceedings in the International Conference on Software Maintenance, Nov. 1998, pp. 34-43. [31] P. Frankl and S.N. Weiss,”An Experimental Comparision of Effectiveness of the all-uses and all-edges adequacy criteria”, ACM New York digital library published in Proceedings of the Symposium on Testing, Analysis and Verification, 1991, pp. 154-164, ISBN: 089791-449-X, DOI: 10.1145/120807.120821. [32] M. Prasanna, K.R. Chandran & K. Thiruvenkadam,”Automatic Test Case Generation for UML Collaboration Diagram”, IETE Journal of Research,57(1):Jan – Feb.2011, pp. 77-81. [33] J. Offutt and A. Abdurazik .,” Generating Test Data From StateBased Specifications”,in proceedings of 2nd International Conference and UML lecture notes in Computer Science, 1999, pp. 416-429. [34] J. Offutt and A. Abdurazik and P. Ammann et al.,” Generating Test Data From State-Based Specifications”, IEEE proceeding in Software Testing, Verification and Reliability, 13:2003, pp. 25-23. [35] Thierry Jeron, Jea-Marc Jezequel and Alain Le Guennec,” Validation and Test Generation for Object-Oriented Distributed Systems”, IEEE, the proceedings of PDSE’98, 1998, DOI: 10.1109/PDSE.1998.668150, pp. 20-29. [36] Michael Scheetz, Anneliese von Mayrhauser, Robert France, Eric Dahlman and Adele E. Howe,” Generating Test Cases From on OO Model with an AI Planning System”, IEEE Computer Society Washington DC, proceeding of the10th International Symposium on Software Reliability Engineering, 1999, pp. 250-260. [37] Zhe (Jessie) Li and Tom Maibaum,” An approach to Integration Testing of Object-Oriented Programs”, IEEE, proceedings of the 7th International Conference on Quality Sofware, 2007, pp. 268-273. [38] A.O. Arantes, N.L. Vijaykumar, V.A. de S. Junior and D. Guimaraes,” Test Case Generation for Critical Systems through a Collaborative Web-Based Tool”, IEEE, proceedings of CIMCA-2008, pp. 163-168, DOI=10.1109/CIMCA.2008.143. [39] Nirmal Kumar Gupta and Dr. Mukesh Kumar Rohil,” Using Genetic Algorithm for Unit Testing of Object Oriented Software”, IEEE, proceedings in 1st International Conference on Emerging Trends in Engineering and Technology, 2008, pp. 308-313, DOI: 10.1109/ICETET.2008.137. [40] J.J. Gutierrez, M.J. Escalona, M. Mejias, I. Ramos and J. Torres,” An Approach for Model-Driven Test Generation”, IEEE, proceedings in 3st International Conference on Research Challenges in Information Science, 2009, pp. 303-312, DOI=10.1109/ IEEE, proceedings in 1st International Conference on [41] Anthony J H Simons and Wenwen Zhao,” Dynamic Analysis of Algebric Structure to Optimize Test Generation and Test Case Selection”, IEEE, proceedings in Testing: Academic and Industrial Conference-Practice and Research Techniques,2009, pp. 33-42, DOI: 10.1109/TAICPART.2009.28.
Clemuntine Nebut, Fraack Fleurey, Yves Le Traon and Jean-Marc Jezequel,” Automatic Test Generation: A Use Case Driven Approach”, IEEE Transactions on Software Engineering, Vol. 32, No. 3, March 2006, pp. 140-155. Philip Samuel, Rajib Mall and Pratyush Kanth, “Automatic Test Case Generation from UML Communication Diagrams.” ELSEVIER proceedings in Information and Software Technology, Vol 49, 2007, pp. 158-171. Monalisa Sarma and Rajib Mall,” Automatic Test Case Generation from UML models”, IEEE 10th International Conference on Information Technology, 2007, pp. 196-201. Monalisa Sarma, Debasish Kundu and Rajib Mall,” Automatic Test Case Generation from UML sequence Diagrams”, IEEE Computer Society proceedings in 15th international conference on Advanced Computing and Communications, 2007, pp. 60-76 Jatarat Jaroenpiboonkit and Tratip Suwannasart,” Finding a Test Order using Object-Oriented Slicing Technique”, IEEE 14th AsiaPacific Software Engineering Conference, 2007, pp. 49-56. Hang Zhou, Zhiqiu Huang and Yi Zhu,” Polymorphism Sequence Diagrams Test Data Automatic Generation Based on OCL(Object Constraint Language)”, IEEE proceedings in the 9th International Conference for Young Computer Scientists, 2008, pp. 1235-1240. Fanping Zeng, Zhide Chen, Qing Cao and Liangliang Mao,” Research on Method of Object Orinted Test Cases Generation Based on UML and LTS(Label Transition System)”, IEEE proceedings in the 1st International Conference on Information Science and Engineering(ICISE2009), 2009, pp. 5055-5058. Esmaeil Mirzaeian, Samad Ghaderi Mojaveri, Homayun Motameni and Ahmad Farahi, “ An Optimized Approach to Generate Object Oriented Software Test Case by Colored Petri Net”. IEEE 2nd International Conference on Software Technology and Engineering(ICSTE), 2010, pp. V2-251-V2-255. Hojun Jaygarl, Kai Shin Lu and Carl K. Chang,” GenRed: A Tool for Generating and Reducing Object-Oriented Test Cases”, IEEE 34th Annual IEEE Computer Software and Applications Conference, 2010, pp. 127-135. Nagendra Pratap Singh, Marinal Kanti Debbarma, Rishi Mishra and Sanjay Sachan,” The Review: Lifecycle of Object-Oriented Software Testing”, IEEE transactions proceedings in 2011, pp. 52-56. Yury Pavlov and Gordon Fraser,”Semi-Automatic Search-Based Test Generation”, IEEE Computer Society 5th International Conference on Software Testing, Verification and Validation, 2012, pp. 777-784. Pietro Braione, Andrea Mattavelli and Ali Muhammad,” An industrial Case Study of Effectiveness of Test Generators”, IEEE proceedings in AST-2012, Zurich, Switzerland, 2012, pp. 50-56. Mathias Landhauber,” Automated Test Case Generation by Cloning”, IEEE proceedings in AST-2012, Zurich, Switzerland, 2012, pp. 8388. Jirawat Chaiareerat, Peraphon Sophatsathit and Chidchanok Lursinsap,” Test Case Generation for Clesses in Object-Oriented Programming using Grammatical Evolution”, Springer Science Business Media B.V. 2012, pp. 251-527. Baiuntha Narayan Biswal,” Test Case Generation and Optimization of Object-Oriented Software using UML Behavioural Models”, MTech(Research) Thesis (Roll No. 607CS003), CSED, NIT, Rourkela, Orissa-769008(India), July-2010. Ranjita Kumari Swain, Prafulla Kumar Behera and Durga Prasad Mahapatra,” Generation and Optimization of Test Cases for ObjectOriented Sofware using State Chart Diagram”, proceedings of International Journal CSIT-CSCP-2012, pp. 407-424. W. Eric Wong, Joseph R. Horgan, Aditya P. Mathur and Alberto Pasquini,” Test Set Size Minimization and Fault Detection Effectiveness: A Case Study in a Space Application”, in proceedings of IEEE Computer Society in 21st Annual International Computer Software and Applications Conference, Aug. 1997, pp. 522-528, DOI: 10.1109/CMPSAC.1997.625062.
988
[42] Wei Han, NA Zhang and Zuohua Ding,”Test Generation Based on Notated Net”, IEEE, proceedings of 4th International Conference on Computer Sciences and Convergence Information Technology, 2009, pp. 1168-1173, DOI: 10.1109/ICCIT.2009.107. [43] A. Rao, Kavitha Elizabeth George, G. Logeshwari, S. Viveka Katherine and T. Mythili,” A Model for the Development of a Mutation Testing Cum Test Case Generation Tool”, IEEE, proceedings of International Conference on Advances in Recent Technologies in Communication and Computing, 2009, pp. 812-814, DOI: 10.1109/ARTcom.2009.194. [44] A. Arcuri and X. Yao,” On Test Data Generation of Object-Oriented Software”, IEEE, proceedings of Testing: Academic and Industrial Conference-Practice and Research Techniques, 2007, pp. 72-76, DOI=10.1109/TAIC.PART.20072.11. [45] Gordon Fraser and Andrea Arcuri,” Evosuit: Automatic Test Suite Generation for Object Oriented Software”, proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering(ESEC/FSE’11), 2011, pp. 416419, DOI=10.1145/2025113.2025179. [46] Narayana Maruvada. (2013). Understanding the essentials of test case design. Test Case Design. testing experience- the magazine for professional testers. Dec. 24,2013, pp.28-30. Retrieved from: www.testingexperience.com/issues/testingexperience24_12_2013.pdf [47] Jeremy Ristau & Harish Narayan.(2013). Leveraging test case architectural design to promote reuse and scalability. Test Case Design. testing experience- the magazine for professional testers. Dec. 24,2013, pp.45-47. Retrieved from: www.testingexperience.com/issues/testingexperience24_12_2013.pdf [48] Muhammad Dhiauddin Mohamed Suffian & Fairul Rizal Fahrurazi.(2013). Designing the performance Test Cases: An Experience from Testing the Cloud-Based System . Test Case Design. testing experience- the magazine for professional testers. Dec. 24,2013, pp.50-53. Retrieved from: www.testingexperience.com/issues/testingexperience24_12_2013.pdf [49] Teemu Kanstrén. (2013). Test design with model-based testing. Test Case Design. testing experience- the magazine for professional testers. Dec. 24,2013, pp.56-59. Retrieved from: www.testingexperience.com/issues/testingexperience24_12_2013.pdf [50] Liping Li, Xingsen Li, Tao He & Jie Xiong. (2013). Extenics-based test case generation for UML activity diagram. Proceedings’ of Elsevier Information Technology and Quantitative Management (ITQM2013). Procedia Computer Science 17 ( 2013 ) 1186 – 1193, DOI: 10.1016/j.procs.2013.05.151 [51] Noura Ouerdi, Mostafa Azizi, Jean-louis Lanet, Abdelmalek Azizi, & M’hammed Ziane. (2013). EMV Card: Generation of test cases based on SysML models. In the proceedings of 2013 International Conference on Electronic Engineering and Computer Science ,Elsevier IERI Procedia 4 ( 2013 ) 133 – 138. DOI: 10.1016/j.ieri.2013.11.020 [52] Zhang Junli & Song Limin. (2013). Research on Object-oriented Software Testing Cases of Automatic Generation. Journal of Networks, Vol. 8,No. 11, November 2013. DOI:10.4304/jnw.8.11.2627-2634 [53] Xusheng Xiao, Sihan Li, Tao Xie & Nikolai Tillmann.(2013). Characteristic studies of loop problems for structural test generation via symbolic execution. Retrieved from: (http://research.microsoft.com/pubs/200740/ase13-loopstudy[1].pdf) on Jan. 03,2014 [54] Praveen Ranjan Srivastava et al. (2013). Optimal test sequence generation using firefly algorithm. Swarm and Evolutionary Computation Journal ,Vol 8,No 1,pp44-53 (Elsevier) [55] Stefan J. Galler & Bernhard K. Aichernig. (2013). Survey on test data generation tools-an evaluation of white- and gray-box testing tools for C#, C++, Eiffel, and Java.(Springer). April 11,2013, DOI: 10.1007/s10009-013-0272-3
989
