ConData: A Tool for Automating Specification-Based Test Case ...

Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000

ConData: a Tool for Automating Specification-based Test Case Generation for Communication Systems Eliane Martins, Selma B.Sabião ∗ Institute of Computing State University of Campinas (UNICAMP) {eliane, selmasab}@dcc.unicamp.br

Abstract. This paper describes a tool called ConData used as test generation for communication protocols specified as extended finite state machines. The strategy for test generation combines different specification-based test methods: (i) transition testing for the control part of a protocol and (ii) syntax and equivalence partitioning for the data part. The tool uses a representation of the protocol in PSL (Protocol Specification Language), which is transformed into a format readable by a Prolog program. This implements the test strategies mentioned above. The text also presents some results obtained in the test generation for the protocol of the Tele-command Communication System of the SACI-1 satellite. Keywords: communications protocol testing – EFSMbased testing - logic programming

1. Introduction Conformance testing of communication protocols aims at verifying that the external behaviour of a protocol implementation complies with the protocol specification. This is the first step to assure compatibility with other implementations of the same protocol [1, 4].

∗

Ana Maria Ambrosio Ground System Division National Institute for Space Research (INPE)

[email protected]

Standardisation committees have defined conformance test standards (“ISO9646 -- Conformance Testing Methodology and Framework”) to guide testing of protocols based on OSI Reference Model. According to these standards the implementation under test (IUT) is considered as a “black box”, observed only through its upper and lower service access points (SAP). Therefore, conformance testing is a black-box testing strategy, which implies that a precise protocol specification is needed for the derivation of test cases and the analysis of the results. A protocol specification must cover the control part, relative to the temporal ordering of the interactions and the data part, relative to the coding of the interactions, range of parameter interactions and rules concerning the actual values of parameters. Many different techniques have been proposed for protocol specification, including finite state machines (FSM), Petri nets, grammars, high-level programming languages, among others. ISO and ITU (former CCITT) have proposed three formal description techniques for the description of OSI protocols and services, that are ESTELLE, LOTOS and SDL. These models are mainly concerned with the control part of a protocol specification. For the data part, another notation is needed, as for example, ASN.1, a standardised notation aimed at specifying interactions (PDUs and ASPs). Several methods have been proposed for generating tests from FSM-based specifications. The aim is to exercise each transition in the FSM at least once, covering the control part of the protocol specification. FSM is a simple and widely used model, but can represent only the

Researchers partially supported by grants from CNPq.

0-7695-0493-0/00 $10.00 (c) 2000 IEEE

1

Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000

control part of a protocol specification; therefore, extended FSM (EFSM) models are generally used to describe also the data part. An EFSM combines an FSM model with data representing state variables and interactions parameters with predicates involving these data. This paper presents a strategy for developing test inputs from EFSM models. The model is represented in a Protocol Specification Language (PSL), which also incorporates some data types from ASN.1 for the description of PDUs and ASPs. By combining different test methods (transition test, syntax-based and equivalence testing), it is possible to generate test cases covering both aspects of a protocol specification. A tool called ConData, partially implemented in Prolog, was built to support the proposed approach. In section 2, we outline some aspects of FSM-oriented models testing. Sections 3 and 4 are concerned with design and implementation issues of ConData. In section 5 we present results of test generation for the implementation of the transfer level of the ground-board protocol. This protocol is part of the Tele-command Communication System of the SACI-1, a scientific Brazilian satellite. Finally, section 6 concludes the text.

2. Testing based on FSM-oriented models This section presents FSM models and its extension, and techniques most commonly used to derive test cases from these models. The concepts presented here are based on references [1, 4, 19, 21] which can be consulted for detailed information.

2.1.

The finite state model (FSM)

A finite state machine can be defined as M = , where: S is a finite, non-empty set of states; s0, represents the initial state (s0 ∈ S); I is a finite set of inputs; O is a finite set of outputs and t is the transition function, t: DS → S × O, where DS is a specification domain such that DS ⊆ S × I. The transition function represents the possible transitions of the machine. The expression t(si, x) = (sn, y) represents the transition from state si to state sn when input symbol x is received, producing the output y. When DS ⊂ S × I, t is not defined for each pair (s, x), representing machines that are incompletely specified. For such machines, a completeness assumption should be adopted to specify the output of the FSM, in each state, to each unexpected input. For example, M ignores each unexpected input by producing a null output and remaining in the current state. When |t(s,x)| = 1 ∀x ∈ DS, the FSM is deterministic; otherwise it is non-deterministic.

An FSM can also be represented by a directed graph G = (V, E), where V, the set of vertices, represents the set of states, S; and a set of directed edges E={(vi, vn; x/y), vi, vn∈V} represents the FSM transitions, e.g., each edge ein = (vi, vn; x/y), represents a state transition from vi to vn with input x and output y. 2.1.1. Fault model. Given an FSM specification of a system (denoted as MS) and an implementation of this FSM (denoted as MI), the objective of conformance testing is to determine whether MI conforms to (i.e., behaves in accordance to) MS by testing MI as a blackbox. A black-box testing does not assume the availability of implementation details about MI such as number of states and transitions. Therefore the conformance of MI to MS can only be determined by the analysis of the observed outputs of MI. MI can differ from MS in many ways, but a black-box testing is able to detect only the following differences: • M I may have missing states or transitions; • the completeness assumption considered; • the tail state of a transition (transfer fault); • the output of a transition (output fault). Missing states as well as transfer faults can be detected by the use of characterising sequences (mentioned later). Missing transitions as well as incorrect implementation of the completeness assumption will exhibit themselves as instances of either output or transfer faults. 2.1.2. Test selection. Test selection is based on transition testing which is aimed at determining whether every transition of MS is correctly implemented by MI. The implementation of a given transition t jk = (sj, s k; x/y) is checked according to the procedure: 1. MI is brought to state sj; 2. input x is applied to MI and output produced is checked for whether it is y (to detect output faults); 3. the state reached by MI after application of x is checked for whether it is sk (to detect transfer faults). Testing is carried out by using test sequences; a test sequence consists of a sequence of inputs (and eventually the corresponding outputs) that takes an FSM from its initial state; it contains a test sub-sequence for each transition of the FSM and returns the FSM to the initial state. Therefore, for a transition testing purposes, it is assumed that MS is such that every state is reachable from the initial state, and that the initial state is reachable from any state. Step 3 requires the existence of a characterising or state identification sequence, which is the shortest sequence of inputs that allows to verify that a designated state is reached. Various test sequence generation methods were proposed for transition testing, and they vary according

0-7695-0493-0/00 $10.00 (c) 2000 IEEE

2

Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000

to the assumptions about the FSM. For the sake of conciseness, they will not be presented here; the aforementioned references are a good starting point to learn about them.

2.2.

Test selection for extended FSM (EFSM)

An FSM model is adequate only to represent the control part. To model the aspects relative to the data part it can occur a state explosion problem, since the number of states grows rapidly. For this reason extended FSM (EFSM) models are often used. An EFSM can be represented as M = , where S, s0, I, O have the same meaning as in 2.1. T is a non-empty set of transitions and V is the set of variables. Each transition t ∈ T is of the form t = (sj, sk, x, p, act, y), where sj and sk represents the starting and the tail state of t, respectively; x can be either an input interaction in I or empty, in which case, a spontaneous transition is defined. These transitions do not depend on input interactions but only on a predicate associated with the transition. A predicate, p, is expressed in terms of the variables in V, the parameters of the input interaction and some constants; act is an action and y is an output in O. The semantic is as follows: if the machine receives the input x at state sj, the predicate p is tested and, if it is satisfied, the action act is executed, producing output y (and possibly updating some of the variables) and the state sk is reached. 2.2.1. Test selection methods. The notation used for the description of predicates and actions is usually based on some high-level language. So the aspects of EFSM-based testing are very close to the aspects of software testing. The traditional methods for testing FSM models are no longer adequate. They can result in non-executable test sequences. This can happen doe to non-satisfied predicates or when actions include loops. Determining which inputs should be used in order to force the selection of a particular transition is undecidable. Another difficulty in EFSM based testing is relative to the selection of appropriate values for input interactions parameters from a large range of possible values (input domain). This is the parameter variation problem [23]. Due to the similarities to white-box testing, several researchers proposed to separate testing in control flow, using FSM testing methods to the control part of the specification, and data-flow for the data part [1]. Dataflow analysis involves the input and output interactions and the variables. The idea is to select test cases that are appropriate for the transitions based on data-flow test selection criteria developed for white-box testing. To clearly separate control and data flow aspects, a normal form specification was proposed in [18]. In such specification, all control aspects are represented in the

FSM model, that is, the actions can contain neither branch statements nor loops. The methods proposed in [20, 23 and 2] are examples of the use of such approach. The study in [20] uses data flow analysis of normal form specifications in ESTELLE, a standardised formal description technique based on EFSM. Test sequences are derived to cover all pairs of definitions and uses of context variables and interactions parameters. The work does not consider the control part, and the test sequences only cover partially the data aspects of a protocol specification. [23] presents an extension of this method in which test sequence selection has 2 phases: the first phase selects test sub-sequences based on data-flow selection criteria; the second phase selects values for each interaction parameter for each subsequence according to the input domain of the specification. The method do not cover the control part, as the former one, and the executability problem is left to the user. [2] presents a method combining control and data flow testing to generate test sequences. The method considers the executability of test sequences during path generation: to make non-executable paths executable, Cycle Analysis is performed to find the shortest cycle1 to be inserted in a path so making it executable. In [24] the limitations of data-flow analysis based test generation is discussed. As each variable is tested along a selected path, faults existing in alternate paths may remain undetected, e.g., wrong assignment to the variable in the alternate path. The method proposed in this study uses constraint-based test sequence generation which, according to their authors, allows better fault coverage than data flow based criteria. A tool called TESTGEN supports the proposed method. It accepts a protocol description in an Extended Transition System model (ETS, which has similarities with Estelle), for the control part, and ASN.1 to represent the data part. For each selected subtour (starting and ending in the initial state) in the ETS, a test case is generated that satisfies a set of user definable restriction mechanisms (constraints). A constraint is a predicate that must hold for each subtour and test case. These constraints specifies the minimum and maximum number of times in a subtour that: (i) a state or transition has to be reached; (ii) a variable is assigned a value; (iii) a timer action (start, stop, reset) is performed for each timer in the specification. In addition, there are parameter variation constraints defining a set of values for each parameter of each input ASP or PDU that can be sent to the IUT. Other methods view an EFSM as a compressed form of an FSM and are based on the unfolding of an EFSM into a pure FSM by expanding the values of parameters. To avoid state explosion problems, the domains have to be reduced. In this way, FSM-based methods can be used to 1

A cycle is composed by one or more transitions t1, ..., tk such that the ending state of tk is the same as the starting state of t1.

0-7695-0493-0/00 $10.00 (c) 2000 IEEE

3

Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000

cover the control and data flow aspects of the protocol. The study presented in [7] uses this approach. The authors classify predicates in three classes: controlled, settable and uncontrolled. Controlled predicates concern the variables whose values do not change, or whose values are changed by local actions following a specified event. Assuming that local variables (completely controlled by the model, opposed to global variables) are discrete and bounded, this class of predicate can be removed from the model by creating a new set of states. Settable predicates mainly concern parameter values of PDU’s, so they can be directly set by a lower tester (see 1) or, in other words, they can be treated as particular inputs. Uncontrolled predicates represent the effect of the environment on entity behaviour, so they are not controlled either by the entity itself or by the testers. They introduce nondeterminism in the specification and many repetitions may be necessary to obtain all possible outputs. The method uses regular expressions to define all the cycles from the initial state, and the user gives the number of repetitions for each cycle.

of the comparison, the last line of the table also shows the contribution of this study, which will be described in the sections that follow.

2.2.2. Final comments. From what is stated above, test selection from EFSM is complex because the data part of the protocol must be considered. In particular, the selection of parameter values for the input interactions is subject to the following constraints [23]: • Parameter variation: how to choose the fields whose values need to be varied and the selection of representative values for these fields from a relatively large domain? • Path executability: how to select values that satisfy the predicates associated with the transitions in a test sub-sequence, when there are such sub-sequences that corresponds to non-executable paths? A synthetic view of the solutions given by the studies presented here is provided in Table 2-1. For the purpose

3.1.

3. The proposed testing approach The objective of the approach is to allow the combination of different black-box testing methods to cover all aspects of the communication system. As stated in [15], the following aspects are to be considered: events, states, data, predicates and actions. For the moment, the following testing methods are implemented: (i) transition testing, (ii) syntax testing, to create the structures for the input interactions, relative to the allowable order and format of the parameters of these interactions, and (iii) equivalence class partitioning, to generate parameters values for the input interactions. These methods are further explained in section 4. In the following, the methodology for test case generation, followed by an overview of PSL.

Methodology overview

The methodology for test case generation [17] consists of the following steps: i) the communication protocol is specified in the form of an EFSM model which is represented in PSL (Protocol Specification Language); ii) the communication protocol specification is transformed into a test specification and, iii) the test specification is used to generate test cases guided by user defined constraints. Steps (ii) and (iii) are performed with the aid of the ConData tool and are described in 4.2 and 4.3.

Table 2-1.Overview of the test approaches presented. [2]

Model EFSM

Approach Normal form

[7]

EFSM

Unfolding

[18] [20] [23] [24]

EFSM (in Estelle) EFSM (in Estelle) EFSM (in Estelle) EFSM (in ETS, a subset of Estelle and ASN.1) EFSM (in PSL)

Normal form Normal form Normal form

This study

Normal form

Contribution Combines control and data flow testing. Uses heuristics to verify path executability Unfolding based on predicate characteristics, allowing to cover control and data aspects. Use of regular expressions to nondeterministic parts of the specification. Data-flow coverage (data part) and functional testing (control part) Data-flow coverage (data part) Data-flow coverage + parameter variation (data part) Use of user-definable constraints to cover control and data aspects (including parameter variation), reduce test sequence size and to treat executability in a user-controllable way. Combines black-box testing methods to cover control and data parts. Use of constraints to reduce test sequence size and control nondeterminism.

0-7695-0493-0/00 $10.00 (c) 2000 IEEE

4

Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000

For step (i) the user should create an specification which is required to be initially connected, i.e., every state should be reachable from the initial state. The EFSM model may be non-deterministic, partially or completely specified. The model should contain only normal form transitions, so that transition testing could cover all control aspects of the communication system being tested.

3.2.

The Protocol Specification Language (PSL)

The EFSM is specified using PSL [17], which provides a simple, yet formal notation to express both control and data part of the model. The control part of PSL extends the language presented in [25]. For the definition of the data part, a small subset of the ASN.1 notation was incorporated to the language. A protocol description in PSL is into 5 parts: (1) variables declaration; (2) state declaration; (3) input and output interactions declaration; (4) data definition; (5) transitions definition. A brief description of each part follows: • Variables declaration This part is used to declare names and types of the variables used in the specification. The types allowed are: integer, boolean, real, bitstring, octetstring and enumerated, as defined in ASN.1; they are briefly explained later. For example: VARIABLES: integer number; boolean expire_timer; •

States declaration This section is used to declare the names of the states. The initial state is preceded by the symbol ‘#”. For example: STATES: #idle; waitconection; •

Input and output interactions declaration Input and output interactions are declared separetely. For example: INPUTS: DataRequest; Disrequest; OUTPUTS: CR; SendRequest; Disrequest; Interaction names that can be used both as input and output must appear twice, as it is the case of the Disrequest interaction in the previous example. •

Data definition

This part describes the syntax of the interactions, as well as the names and types of their parameters. Parameter types are the same used for variables: integer and real, for numerical values; boolean, for values “true” or “false”; bitstring, to represent any sequence of bits of a specified size; octetstring, for any sequence of ASCII characters enclosed between “ “; and enumerated types for a finite set of values such that only one of these values may be chosen for a particular instance of the type. Parameters may also be represented by structured types, which were also borrowed from ASN.1: sequence, set, sequence of, set of and choice. The type sequence is used to group an ordered collection of types. Set designates an unordered collection of types. Sequence of is used to represent an ordered sequence of values of the same type. Set of defines an unordered set of values of the same type and choice allows the definition of a set of types, one of which may be chosen for a particular instance of the parameter being defined; it is similar to union types in C. Differently from ASN.1, the declaration of nonstructured types must include the length or the range of the allowable values. For the structured types sequence of and set of, the size must be specified. For example, the parameter definition of the output interaction DataRequest could be: DATA: DataRequest ::= sequence { SDU octetstring 5, NbOfSegment enumerated 2 | 4 | 8, Blkbound integer 3 .. 15 }; For more specific details about these types, a reference on ASN.1 should be consulted, such as [13]. •

Transitions definition This section describes the control part of the communication system. Each transition has a name, which is preceded by an “*” and consist of a current state name, an input name, a predicate, an action part, an output name and next state name. A current state name is preceded by the symbol “>”, and the next state name by “connected // input from the upper interface, and its parameters ?U.DataRequest(SDU, NbOfSegment, Blkbound) // action involving variables { number :=0; counter :=0 } // next state blocked // predicate involving variables (a boolean // and an integer one) and an input parameter [expire_timer and counter < = Blkbound] // output to the lower interface !L.Tokenrelease connected ?U.DATAreques(SDU,NbOfSegment,Blkbound) {number := 0}