Automated Generation of Test Cases from Output Domain and Critical

Automated Generation of Test Cases from Output Domain and Critical Regions of Embedded Systems Using Genetic Algorithms Chandra Prakash Vudatha

Dr. Sastry KR Jammalamadaka

Dept. of Information Technology, KL University [email protected]

Dept. of Information Technology, KL University [email protected]

Sateesh Nalliboena

Bala Krishna Kamesh Duvvuri

Dept. of Information Technology, KL University [email protected]

Department of Freshmen Engineering, KL University [email protected]

Dr. Reddy L.S.S. LBR College of Engineering, Mylavaram, [email protected]

Abstract— A primary issue in black-box testing is how to generate adequate test cases from input domain of the system under test on the basis of user’s requirement specification. However, for some types of systems including embedded systems, developing test cases from output domain is more suitable than developing from input domain, especially, when the output domain is smaller. Exhaustive testing of the embedded systems in the critical regions is important as the embedded systems must be basically fail safe systems. The Critical regions of the input space of the embedded systems can be pre-identified and supplied as seeds. In this paper, the authors presents an Automated Test Case Generator (ATCG) that uses Genetic algorithms (GAs) to automate the generation of test cases from output domain and the criticality regions of an embedded System. The approach is applied to a pilot project ‘Temperature monitoring and controlling of Nuclear Reactor System” (TMCNRS) which is an embedded system developed using modified Cleanroom Software Engineering methodology. The ATCG generates test cases which are useful to conduct pseudo- exhaustive testing to detect single, double and several multimode faults in the system. The generator considers most of the combinations of outputs, and finds the corresponding inputs while optimizing the number of test cases generated. Keywords - Automated Test case generation, Output domain testing, Genetic algorithms, embedded systems, Cleanroom software engineering, combinatorial testing, pseudo-exhaustive testing.

I. INTRODUCTION Clean room software engineering methodology stresses on minimum testing as the verification and validation processes are carried out throughout the development of any system. However, the process of testing must be done as exhaustively as possible with minimal test cases that test any system across all the cross sections of the system.

978-1-4244-9581-8/11/$26.00 © 2011 IEEE

Testing is an important part in the development of any system as it represents the ultimate verification and validation of specification, design and code. Once source code has been generated, whether it represents hardware or software, the system must be thoroughly tested to uncover as many errors as possible before delivery to the customer. The goal of testing is to design a series of test cases that has the highest likelihood of finding most of the errors with a minimum amount of time and effort. The techniques used to test the embedded systems provide systematic guidance for designing tests that exercise the internal logic of ES components and test the input and output domains of the program to uncover errors in program function, behavior and performance. Black-box testing focuses on the functional requirements of the Embedded Systems without regard to the internal working of the system. The primary issue in black-box testing is the selection of adequate test cases on the basis of requirements specification to detect as many faults as possible at a minimum cost and time. Combinatorial testing is a black-box technique that could dramatically reduce the number of tests as it is a highly efficient technique to detect software faults. The method generally followed in combinatorial testing is to derive test cases from input domain of the system under test. But, when the input domain is larger and the output domain is much smaller, it is preferable to go for testing the output domain either exhaustively [1,2] or as much as possible. For some types of systems like safety critical embedded systems, developing test cases drawn from output domain will be more appropriate than from input domain as it ensures that all or as many possible output combinations are thoroughly tested.

Exhaustive testing [3,4] of output domain is out of question when many input variables exist and they act in several combinations. Pseudo-Exhaustive testing aims at considering only those combinations that will most likely result in failure conditions [3].

Power supply Key-C

Key-D

Key-E

Key-F

Key-8

Key-9

Key-A

Key-B

Key-4

Key-5

Key-6

Key-7

Key-0

Key-1

Key-2

Key-3

The remainder of this paper is organized as follows: Sections 2 and 3 provide the background of TMCNRS and GAs respectively. Section 4 describes test case generation using GAs. Section 5 provides the results and analysis. Conclusion and future work are presented in Section 6. II. PILOT PROJECT ON TEMPERATURE MONITORING AND CONTROLLING OF NUCLEAR SYSTEM (TMCNRS) TMCNRS is the system under test for which ATCG generates a test suit. The hardware integration diagram of TMCNRS is shown in Fig. 1. The temperature sensors are connected to signal conditioners and the output of signal conditioners are connected to A/D converter which communicates with Micro Controller using I2C Communication protocol. The output devices are connected to the Micro Controller through its output ports. The Relays connected to the pumps are connected to the I/O pins of the Micro Controller. The Micro Controller is also connected to the PC (HOST) using RS232C interface. The sensors are mounted on the water tube situated in the mechanical setup. Flow control is achieved through activation and deactivation of the relays that control the Start-stop mechanism of the pumps. The pumps are part of the mechanical setup whereas the relays are part of the embedded system. The relays are fed with input power and the output power of the relay is connected to the pumps by drawing the power lines. Buzzers are connected to Micro Controller. LCD display is connected to the Micro controller. A key board is connected to the Micro Controller through A/D converter and the Data from Key board is read in I2C communication.

. . .

. . Relay-4

Several authors have developed genetic algorithms for generating test cases [5,6,7] from input domain and those algorithms are specially meant for generating test cases for loaded systems. The authors of this paper propose a tool named Automated Test Case Generator based on genetic algorithms that generates a suite of test cases for almost all combinations of outputs for a pilot project entitled Temperature Monitoring and controlling of Nuclear Reactor system (TMCNRS).

Pump-1 Relay-1

89c51 Micro Controller

LCD Display Buzzer-1

Pump-4

. .

I2C Communication A/D Converter

Buzzer-3

RS232C Temp-1 Signal Conditioner

S1

. . . .

Temp-4 Signal Conditioner

RS232C

S4

Fig. 1 Hardware Integration Diagram of TMCNRS

III. GENETIC ALGORITHMS Genetic algorithm is a global optimization search technique that begins with a set of initial individuals as the initial population, which is usually randomly generated from the problem domain. GA performs selection, crossover and mutation operations to obtain the best solution for the problem..Each generated individual is evaluated with a fitness function so that the evolution of the individual may approach the optimal solution. This evolution process continues until the whole population contains the individuals with high fitness value i.e., the desired solution is obtained. Selection operator is used to select two parents from the population based on some criteria so that crossover can be applied. Crossover is a process in which a gene is randomly selected from first parent and is exchanged with a gene from the second parent. This results in evolution of two children. Mutation is a process in which a gene is randomly selected from a parent and is altered. This results in evolution of one child. The importance of the newly generated child is evaluated using fitness function. IV.

GENERATION OF TEST CASES FOR TMCNRS USING GENETIC ALGORITHMS

The Temperature Monitoring and Control Mechanism of the embedded system TMCNRS and the technique of employing GAs to generate test cases automatically for TMCNRS are discussed in detail in this section. A. Temperature monitoring and control mechanism of TMCNRS Functional requirements of the pilot project are shown in Table 1. The function of TMCNRS is to maintain a temperature gradient in the water tube located in the mechanical setup. Four sensors are mounted on the water tube.

The purpose of the sensors is to monitor the temperatures (1 to 99°C) at different points in the water tube. Whenever the temperature at a point is more than the corresponding reference temperature that is to be maintained at that point, the corresponding pump is activated (ON) to send cold water to that point. When the temperature drops to its reference temperature, the pump is deactivated (OFF). Similar control mechanism is applied to all the pumps in the system. When the absolute difference of temperatures (sensed by sensors) between any two successive points is more than the GRADIENT value (assumed to be 2 herein) the corresponding buzzer is activated (ON); otherwise the buzzer is deactivated (OFF). If there are ‘n’ sensors in TMCNRS, then there will be ‘n’ pumps and ‘n-1’ buzzers. B. Test case generation using genetic algorithms Genetic algorithms can provide a basic framework for solving complex problems like generation of an efficient test suite for a safety critical embedded system [8] like TMCNRS. To start with, a chromosome representation (Table 2) of the solution to the target problem must be defined. Each chromosome encloses an input vector of temperatures in which each temperature is considered as a gene. It also encloses an expected output vector viz. Pumps-Status-Code (PSC), a binary bit map in which the status of the each pump is represented as one bit. The “OFF” condition of the pump is represented as ‘0’ and the “ON” condition is represented as ‘1’. It also encloses another expected output vector viz. Buzzer-Status-Code (BSC), another binary bit map, in which the status of the each buzzer is represented as one bit. In order to compute the expected output for the selected input, the input-output relationship depicted in the functional requirements of TMCNRS (shown in Table 1) is to be considered. The reference temperatures considered in this work are 30, 32, 34 and 36. Since Temp1 (=31) > Ref. Temp1 (=30), Pump1 is expected to be “ON” (represented as ‘1’ by bit B0 in PSC). As Temp2, Temp3 and Temp4 are less than their respective ref. temperatures, the corresponding pumps P2, P3 and P4 are expected to be ‘OFF’ (represented as ‘0’ by bits B1, B2 and B3 in PSC). Since ABS (Temp1 - Temp2) > 2, Buzzer1 is expected to be “ON” (represented as ‘1’ by bit B0 in BSC). Since ABS ((Temp3 - Temp4) > 2, Buzzer 3 is “ON” (represented as ‘1’ by bit B2 in BSC). The relationships that exist between the input domain and output domain of TMCNRS based on control mechanisms discussed earlier can be tabulated as shown in the Tables 3 and 4. These relationships are used by ATCG for generating the test cases. In addition, Criticality Regions of each of the Temperatures can be defined in the region of reference Temperatures. The criticality regions can be used as Initial seeds to the generation of test cases. In the TMCNRS system the criticality regions considered will be one less and one more than the reference Temperature.

Table 5 shows the Initial seeds that will be supplied to the Genetic algorithm detailed below based on which all the test cases that satisfy the output domain and the criticality regions are generated.. The seed test cases are generated at the Initialization stage. TABLE I. S. No. Req. 1

REQUIREMENTS SPECIFICATION OF TMCNRS

Functional requirements Sense Temperature-1(Temp1) after 10 milliseconds and display the same on LCD and send the same to HOST. If the sensed Temp1 > Ref. Temp1 then Relay connected to the Pump1 must be activated (ON); else deactivated (OFF). Sense Temp2 after 10 milliseconds and display the same on LCD and send the same to HOST. If the sensed Temp2 > Ref. Temp2 then Relay connected to the Pump2 must be activated (ON); else deactivated (OFF). If ABS (Temp1 – Temp2) > 2 then Assert the Buzzer1 (ON); else de-assert the Buzzer1 (OFF) Similar control mechanism (as stated in Req. 1, Req. 2 and Req. 3) is applied to Pump3, Pump4, Buzzer2 and Buzzer3

Req. 2

Req. 3

Req. 4

TABLE 2

REPRESENTATION OF CHROMOSOME IN THE TEST SUITE GENERATED FROM THE OUTPUT DOMAIN OF TMCNRS Input vector Temp2

Temp1 (gene1) 31

(gene2) 27

Temp3 (gene3) 29

Temp4 (gene4) 26

Expected Output vectors PSC BSC (Binary) (Binary) 0001 101

TABLE 3 RELATIONSHIPS BETWEEN INPUT TEMPERATURES AND THE EXPECTED PUMP OUTPUTS

Temperature sensed by Sensor1 20

Ref. Temp1

40

30

30

Expected status of Pump1 0 (Pump OFF) 1 (Pump ON)

TABLE 4 RELATIONSHIP BETWEEN INPUT TEMPERATURES AND THE EXPECTED BUZZER OUTPUTS Temperature sensed by Sensor1 20

Temperature sensed by Sensor2 21

20

23

Expected status of buzzer1 0 (Buzzer OFF) 1 (Buzzer ON)

The output parameters of TMCNRS are related to 4 pumps and 3 buzzers. Each output parameter (the status of a pump or a buzzer) can take two values i.e., either ‘0’ or ‘1’. In order to conduct exhaustive testing of output domain (considering all possible combinations of the output parameters), the number of test cases required are 24x23 = 128. Similarly, for an upgraded configuration of TMCNRS with 8 sensors, 8 pumps and 7 buzzers, the number of test cases required=2 8x27=32,768

(a super large figure). In this case, the testing becomes impracticable. TABLE 5 SEED TEST CASES BASED ON CRITICALITY REGIONS Temperatu re sensed by Sensor1 29 30 31

Temperature sensed by Sensor2 31 32 33

Temperatur e sensed by Sensor3 33 34 35

Temperatur e sensed by Sensor3 35 36 37

The authors propose an optimization technique to solve this problem in which pseudo-exhaustive testing can be conducted. The proposed genetic algorithm generates a set of test cases (set P) in which the expected output vector contains all possible combinations of all pumps. A second genetic algorithm is proposed that generates a set of test cases (set B) in which the expected output vector contains all possible combinations of all buzzers. Then, the total number of test cases is 24+23=24 in case of the first configuration of TMCNRS with 4 sensors, 4 pumps and 3 buzzers. This number 24 is much smaller than 128. Again, in case of the second configuration of TMCNRS with 8 sensors, 8 pumps and 7 buzzers, the total is 28+27=384. This value is extremely smaller when compared with 32,768. The authors propose one more optimization technique which finds the union of the two sets (set P U set B) getting a resultant set whose size is smaller in most of the cases i.e., 256