Generating Test Sequences from Statecharts for ... - CiteSeerX

IEICE TRANS. INF. & SYST., VOL.E89–D, NO.4 APRIL 2006

1459

PAPER

Generating Test Sequences from Statecharts for Concurrent Program Testing Heui-Seok SEO†a) , Member, In Sang CHUNG†† , and Yong Rae KWON† , Nonmembers

SUMMARY This paper presents an approach to specification-based testing of concurrent programs with representative test sequences generated from Statecharts. Representative test sequences are a subset of all possible interleavings of concurrent events that define the behaviors of a concurrent program. Because a program’s correctness may be determined by checking whether a program implemented all behaviors in its specification or not, the program can be regarded as being correct if it can supply an alternative execution that has the same effects as the program’s behavior with each representative test sequence. Based on this observation, we employ each representative test sequence as a seed to generate an automaton that accepts its equivalent sequences to reveal the same behavior. In order to effectively test a concurrent program, the automaton such generated accepts all sequences equivalent to the representative test sequence and it is used to control test execution. This paper describes an automated process of generating automata from a Statecharts specification and shows how the proposed approach works on Statecharts specifications through some examples. key words: statecharts, concurrent program, specification-based testing, test sequence, equivalent class

1. Introduction In specification-based testing of concurrent programs, a program is usually checked against its specification by executing the program with test sequences derived from specifications. In general, a test sequence corresponds to an interleaving of concurrent events and are derived by analyzing the sequencing constraints between the events [1]–[3]. Most of the prior work to date on specification-based testing of concurrent programs requires that all possible interleavings of concurrent events in a specification should be implemented in its program [4]–[6]. This poses a problem for testing because many interleavings can be generated as test sequences causing a high cost in test execution. This paper presents an approach to generating test sequences which can reduce the execution cost significantly. Since internal events in different threads can occur in an arbitrary order, multiple runs of a concurrent program may yield different results due to interleavings of the events even with the same input. However, a concurrent program may still meet its specification even when only a subset of all possible interleavings is implemented in the program. For Manuscript received April 28, 2005. Manuscript revised September 27, 2005. † The authors are with the Department of Electrical Engineering and Computer Science, Korea Advanced Institute of Science and Technology (KAIST), Korea. †† The author is with School of Computer Engineering, Hansung University, Korea. a) E-mail: [email protected] DOI: 10.1093/ietisy/e89–d.4.1459

example, receiving a dial tone of a caller and receiving a ringing tone of a callee are concurrent events in a phone conversation. In this case, two interleaved sequences of the events are possible: a caller hears a dial tone before a callee hears a ringing tone and a vice versa. Obviously, they are equivalent; they reveal the same behavior. Thus the program should be regarded as being correct even if only one of the two sequences is implemented in the program. Based on this observation, we have previously defined two types of conformance relations between a program and its specification [7], [8]; a program conforms to its specification if the program can show all possible behaviors described in the specification. Let valid(S, x) be the set of all valid interleavings of internal events prescribed in the specification S for an input x. A program P is said to be a synchronization equivalence of its specification S for an input x if and only if P implemented all interleavings in valid(S, x). If P is a synchronization equivalence of S for all inputs, then P obviously conforms to to S . We also call a program P a synchronization reduction of its specification S for an input x if and only if P implemented representative sequences from valid(S , x) where a representative sequence is one of equivalent interleavings that reveal the same behavior. Through representative sequences, a program which is a synchronization reduction of its specification for all inputs can exhibit all possible behaviors without realizing all valid sequences. In this paper, we present an approach to generating test sequences from Statecharts based on the notion of a synchronization reduction. However, it must be noted that exploring all equivalent interleavings of concurrent events is still necessary for testing, because we have no priori knowledge about which interleavings the program implemented. When a test sequence derived from a specification is not implemented in a program, the program’s correctness cannot be determined until checking other equivalent interleavings are completed. One straightforward approach we can consider for testing is to generate equivalent classes such that each equivalent class contains all equivalent interleavings in a specification and check whether at least one interleaving for an equivalent class has been implemented in a program. For this purpose, we can employ a flattening method, where a target specification (e.g., Statecharts) is flattened so that we can construct a reachability graph which represents all possible interleavings contained in the specification [4], [6]. However, such an approach offers no hint that one sequence

c 2006 The Institute of Electronics, Information and Communication Engineers Copyright


1460

is equivalent to any other sequence. The rationale behind our approach is that all equivalent interleavings can be generated from an interleaving by using the dependency information between events [8], [9]. If we can find representative interleavings for all behaviors, we can expand each representative interleaving to a distinct equivalent class that contains all its equivalent interleavings separately. Our approach first identifies possible macro steps (PMSs) in Statecharts, which represents behaviors of events in concurrent threads. Then, the dependency information between events is acquired by comparing PMSs. In order to generate representative interleavings, we have developed a state-space exploration technique with PMSs based on partial-order methods. Once representative interleavings are obtained, we construct automata to represent equivalent classes such that each automaton accepts all sequences equivalent to a representative interleaving. Unlike the traditional deterministic execution, our automata-based execution allows a program to be executed according to sequences accepted by the automaton. Thus a single execution of automata-based execution can determine whether a program implemented at least one of equivalent sequences [8], [9]. The rest of the paper is organized as follows. In Sect. 2, we briefly describe Statecharts and partial-order methods which are the background of our approach. Section 3 describes the details about a stepwise process of our approach: identification of PMSs in Statecharts, determination of event dependency, derivation of representative interleavings and construction of automata. In Sect. 4, we show the effectiveness of the proposed approach with a gas station example. Section 5 presents a brief survey of related work. Finally, Sect. 6 concludes our work and presents the directions of future work. 2. Background This section presents a brief overview of Statecharts and partial-order methods. A Statecharts specification is a target specification of our approach and partial-order methods are used for discovering representative interleavings to show different behaviors. 2.1 Statecharts Statecharts are an extended finite state machine (EFSM) with diverse high-level language features such as hierarchy, concurrency and broadcast communication [10], [11]. In Statecharts, state transitions triggered by events describe dynamic behaviors and the extended features facilitate the description of behaviors. However, the extended features make it difficult to analyze specifications because a simple expression can latently contain complex behaviors. While some extended features such as hierarchy can be analyzed directly [12], concurrency is not efficiently analyzed due to its large interleavings. We focus on concurrency and interactions in Statecharts since we are concerned about event-

Fig. 1 The specification of a pager system to be described with Statecharts: Network and User are active threads in which events occur in an arbitrary order.

driven behaviors of concurrent programs. In Statecharts, concurrency and interactions are represented by AND decomposition and broadcast communication, respectively. Figure 1 shows the specification of a pager system that consists of four concurrent components (threads). Network describes the process of sending a page and User describes the process of changing the pager’s modes. Since they are active components in which events occur without any input, they will be implemented as threads in a program. Main Control and Mode Control are the components that react to internal events caused by other components. They will be implemented as shared objects. The components communicate with each other via broadcast communication using internal events. When the event page occurs in Network, it is broadcast and any executable transitions caused by page are simultaneously triggered. If a current state in Main Control is Waiting, the transition with the label (page/alarm) can be triggered and alarm is broadcast. Therefore, a pager system gives a new page with BEEP or VIBRATION depending on states in Mode Control. The semantics of Statecharts are defined as step semantics on configurations [11], [13]. A configuration is defined as a maximal set of states in which a system can be simultaneously. Because we focus on only concurrency, a configuration has exactly one state in each thread. For example, the initial configuration of the pager system is [Initial, Initial, Initial, Beep-Mode]. A (macro) step is a maximal transition between configurations by an event during one unit time, and so it includes all mutually non-conflicting transitions that can be triggered in a current configuration. In a concurrent program, events in concurrent threads can internally occur in an arbitrary order and different behaviors may be shown according to the orders of occur-

SEO et al.: TEST SEQUENCE GENERATION FOR CONCURRENT PROGRAMS

1461

Therefore, partial order methods explore a reduced state space of a specification by making use of the dependency information between (global) transitions [14], [15]. In a concurrent program, the notion of the transition dependency is described as follows. • Let T be the set of transitions in a concurrent program and D ⊆ T × T be a binary, reflexive, and symmetric relation. The relation D is a valid dependency relation for T iff for all t1 , t2 ∈ T , (t1 , t2 ) D (t1 and t2 are independent) implies that both following properties hold for all global states s ∈ S in the program: Fig. 2 The reachability graph derived from the specification of the pager system: E11, E12, E21, and E22 denote the occurrence of signify, page, change MV, and change MB, respectively.

rence. Therefore, a step is denoted by 3-tuple = (source configuration, occurrence of an event, target configuration). In the pager system, 4 events occur in Network and User. For understandability, we describe them as Eij that denotes the occurrence of the jth event in the ith thread: for i, 1 means Network and 2 means User. In the initial configuration, either E11 or E21 is possible and so there exist two different steps: ([Initial, Initial, Initial, Beep-Mode], E11, [Ready, Initial, Waiting, Beep-Mode]) and ([Initial, Initial, Initial, Beep-Mode], E21, [Initial, Vib.-Mode, Initial, Vib.Mode]). When the event page occurs in [Ready, Vib.-Mode, Waiting, Vib.-Mode], both the transition (page/alarm) and the transition (alarm/VIBRATION) are enabled by page and alarm. Therefore, its step is ([Ready, Vib.-Mode, Waiting, Vib.-Mode], E12, [End, Vib.-Mode, Initial, Vib.-Mode]). In this paper, we assume that an event to cause internal events is executed with the internal events. To a concurrent system specified with Statecharts, its behaviors are represented as interleavings in a reachability graph. A (complete) reachability graph of a Statecharts specification is derived by connecting all steps from an initial configuration. Figure 2 shows the reachability graph derived from the pager system specification. We note that configurations including the same states are distinguished from each other by a set of events that are executed for reaching the configurations. From the reachability graph, we can generate 6 sequences that represent the behaviors of the pager system. However, we cannot know that any sequences are equivalent. 2.2 Partial-Order Methods Partial-order methods have been developed as a means to avoid the state-explosion problem in verification of concurrent programs. One major source of state space explosion is the representation of concurrent executions in terms of interleavings of concurrent events. The basic idea of partialorder methods is that exploring all possible interleavings is not necessary for the purpose of verifying some properties because certain events can be executed in an arbitrary order without affecting the results or the state that is reached.

- if t1 is enabled in s and (s → s ) by t1 , then t2 is enabled in both s and s - if t1 and t2 are enabled in s, then there is a unique state s such that (s → s ) by executing not only the sequence t1 , t2 but also the sequence t2 , t1 There exist various techniques for reducing a state space to be explored in verification and they basically differ in how to select a subset of the enabled transitions. Examples include ample sets, sleep sets, and stubborn sets. The basic idea is to limit the expansion of successor states to those states that are reachable via dependent transitions, that is, to ignore independent transitions. Our approach uses the stubborn set technique [15] for generating representative sequences, and so, we will prove that all representative sequences are generated by the stubborn set technique which is sketched as follows. 1 Take one transition t that is enabled in a global state s, Let T s = {t}. 2 For all transitions t in T s : 2.1 If t is enabled in s, add all transitions t such that t and t are in conflict, that is, there exists any thread that is active for both t and t and the thread can choose between t and t . 2.2 If t is disabled in s, add all transitions t such that t is a preceding transition of t. 3 Repeat step 2 until no transition is added. Then return all transitions in T s that are enabled in s. 3. Overview of Stepwise Process Figure 3 shows our approach to generating test sequences from Statecharts. Firstly, it identifies the information about transitions triggered by events as possible macro steps (PMSs). Then, it generates a dependency table of PMSs and it drives a reduced reachability graph by applying partialorder methods adapted for PMSs. From the reduced reachability graph, representative sequences are generated. Finally, it separately generates equivalent classes in the form of automata from each representative sequence using the dependency table. Sections 3.1 through 3.3 describe these activities in more details.


1462 Table 1

Fig. 3 The process of our approach: equivalent classes are generated in the form of automata.

3.1 PMS Elicitation Since our approach is concerned with the observational behavior of a concurrent program, we need to represent (macro) steps in terms of occurrence of events in active threads. For this, we introduce the notion of an abstract configuration to allow a don’t care state and define a possible macro step (PMS) on them. A don’t care state of a component in Statecharts is a special state that can be replaced by any specific state in the component. An abstract configuration is a configuration that can have don’t care states to be denoted as ‘*’ in some components, while a concrete configuration must have specific states in all components. A PMS is defined as a step on abstract configurations. In a PMS, don’t care states indicate unaffected components directly as well as indirectly when an event occurs in a thread. As a matter of fact, abstract configurations and PMSs are based on the observation that events in threads often affect only a subset of concurrent components. Therefore, several specific steps are generated from a PMS according to states in unaffected threads. Definition 1 (Possible Macro Step (PMS)): A possible macro step consists of 3-tuple PMS = (scn , en , tcn ), where en is an event to occur in an active thread, scn is an abstract configuration including all possible source configurations of en and internal events caused by en , and tcn is a target abstract configuration resulted from executing the events in scn . In a Statecharts specification, PMSs are elicited from transitions. First, for the occurrence of an event in an active thread, its PMS is directly elicited by converting states into abstract configurations. For example, in the pager specification, E12 triggers the transition from Ready to End in Network. In this case, we can elicit its PMS ([Ready, *, *, *], E12, [End, *, *, *]) from the transition.

PMSs elicited from the pager specification.

PMS

Source

Event

Target

p1 p2 p3 p4 p5

[ I, *, I, * ] [ R, *, W, BM ] [ R, *, W, VM ] [ *, I, *, BM ] [ *, VM, *, VM ]

E11 E12 p2 E12 p3 E21 E22

[ R, *, W, * ] [ E, *, I, BM ] [ E, *, I, VM ] [ *, VM, *, VM ] [ *, BM, *, BM ]

Second, the PMS is concretized by transitions triggered by the internal events. If the transitions are in distinct components, we have one PMS such that each don’t care state of its abstract configurations is instantiated into a concrete state according to its corresponding transition. Otherwise, the triggered transitions separately refine abstract configurations of the PMS and several resultant PMSs will be generated. For example, E12 causes causes page and it triggers the transition in Main Control. From this, we can elicit ([Ready, *, Waiting, *], E12, [End, *, Initial, *]) by refining the PMS of E12. And then, the abstract configurations are separately refined by the transitions triggered by alarm. As a result, we can elicit two PMSs, ([Ready, *, Waiting, Beep-Mode], E12, [End, *, Initial, Beep-Mode]) and([Ready, *, Waiting, Vib.-Mode], E12, [End, *, Initial, Vib.-Mode]). Algorithm 1 describes the PMS elicitation process from transitions. In the algorithm, the step 2 describes the refinement process of abstract configurations. Table 1 lists the PMS elicited from the pager specification. We assume that each event involving PMSs is distinct, i.e., two events with the same name are treated as different ones if different PMSs are involved. Therefore, we add its PMS number to the external event as a subscript in the case of E12. Algorithm 1 (PMS Elicitation) 1 Generate a PMS p from a transition t in which an event occurs without any input event and P = {p}. 2 For each PMS p in P which causes internal events, do the following. 2.1 Find all transitions triggered by internal events caused by p. 2.2 Make all combinations C T = T 1 × .. × T n , where T i is a set of transitions triggered by internal events in a thread i. 2.3 Add all PMSs pi such that configurations of pi are refined from configurations of p by ci ∈ CT . 2.4 Remove p from P. 3 Return P. 3.2 PMS Dependency Analysis and Reduced Reachability Analysis The definition of the dependency relation between PMSs follows the definition of the transition dependency in Sect. 2.2. Even though the definition describes semantic properties of the dependency relation, it is not practical to determine the PMS dependency and it is desirable to be able


1463

to detect the PMS dependency as efficiently as possible. For this reason, we introduce easily checkable syntactic conditions that are sufficient for the PMSs to be independent. Definition 2 (Dependency between PMSs): Let p1 and p2 be PMSs and sc[i] and tc[i] be states of abstract configurations in a component Ci . p1 and p2 are independent if they satisfy either of the following conditions in all C i , • Con1: p1 and p2 do not affect an identical thread T i , that is, p1 .sc[i] = ‘*’ and/or p2 .sc[i] = ‘*’. • Con2: p1 and p2 have an identical state in T i , that is, p1 .sc[i] = p1 .tc[i] = p2 .sc[i] = p2 .tc[i]. To show that Definition 2 does not contradict the transition dependency, let p1 and p2 be independent of PMSs recognized by Definition 2. According to the transition dependency, they should satisfy two semantic properties: for all concrete configurations c ∈ S in the program, 1) if p1 is enabled in c and changes (c → c ), p2 is enabled in both c and c and 2) if p1 and p2 are enabled in c, there is a unique configuration c such that (c → c ) by both p1 , p2 and p2 , p1 . We observe that p1 and p2 satisfy the first property because p1 changes only states irrelevant to p2 (Con1) or preserves states (Con2). For the same reason, p1 and p2 also satisfy the second property. Table 2 shows the dependency relation between the PMSs in Table 1 which we obtained by using syntactic conditions in Definition 2. Moreover, the dependency relation of events conforms to the dependency relation of the PMSs because a PMS is defined as an affection of an event. Therefore, the dependency relation can be used not only for the reduced reachability analysis but also for the automata generation. Next we describe how to generate representative interleavings based on partial order methods. Although the technique of partial order methods had been adapted to test execution of a concurrent program [16], it is not a technique tailored to test sequence generation. Therefore, our approach begins with constructing a reduced reachability graph from which representative interleavings are derived. As a matter of fact, the reduced reachability graph is a result of a selective search which explores a subset of all enabled PMSs at each configuration encountered during the search. Thus, it is important to determine the PMSs that need to be explored at each configuration and we adopt the stubborn set technique proposed by Valmari which is rephrased as Algorithm 2 below. Table 2 The dependency relation between PMSs in Table 1, where “1” denotes that the PMSs are related, while “0” denotes that the PMSs are independent. p1 p2 p3 p4 p5

p1 1 1 1 0 0

p2 1 1 1 1 1

p3 1 1 1 1 1

p4 0 1 1 1 1

p5 0 1 1 1 1

Algorithm 2 (Computation of Stubborn Set) 1 Take one starting PMS p that is enabled in a given configuration c, Let PMS c = {p}. 2 For all PMSs p ∈ PMS c : 2.1 If p is enabled in c (c ⊂ p.sc), add all conflictingly dependent p on p such that (p ↔ p). 2.2 If p is disabled in c (c p.sc), add all sequentially dependent p on p such that (p → p). 3 Repeat step 2 until no more PMSs can be added, then return all PMSs in PMS c to be enabled in c. Definition 3 (Conflicting Dependency): Let pi and p j have a dependency relation. Then, pi and p j are conflictingly dependent (pi ↔ p j ) iff pi .sc and p j .sc have at least one concrete configuration to be included into both pi .sc and p j .sc. Definition 4 (Sequential Dependency): Let pi and p j have a dependency relation. Then, pi and p j are sequentially dependent (pi → p j ) iff pi .tc and p j .sc have at least one concrete configuration to be included into both pi .tc and p j .sc. The conflicting dependency, (pi ↔ p j ), means that there exists a concrete configuration in which both pi and p j are executable but only one PMS can be executed in the configuration. The sequential dependency, (pi → p j ), means that pi and p j can be sequentially executed in some concrete configurations. These dependencies correspond to “in conflict” and “preceding” in the original stubborn set technique, respectively. The process of building a reduced reachability graph is straightforward. We start with the initial configuration and compute a stubborn set of all enabled PMSs using Algorithm 2. The PMSs in the set and their target configurations are then added to the initial configuration. This process is repeated for each new target configuration, where the PMSs executed for reaching the configuration are not expanded. Therefore, each path in the reduced reachability graph does not contain an identical PMS twice. Figure 4 shows the reduced reachability graph derived from the pager specification. Although the stubborn set technique is a well-formed method in formal verification, we adapt it for generating representative interleavings with PMSs in test sequence generation. Therefore, we are going to prove that, for any interleaving generated from the reachability graph, its representative interleaving can be generated from the reduced reachability graph. Let the sequence ω = p1 , p2 , .., pi−1 , pi , .., p j−1 , p j , .., pn which is generated from the reachability graph but cannot be generated from the reduced reachability graph. It means that the reduced reachability graph includes the path for the subsequence p1 , p2 , .., pi−1 from an initial configuration c0 to the configuration ci−1 but it does not contain the edge for pi from ci−1 , that is, pi is not in the stubborn set of ci−1 . Therefore, the stubborn set contains p j that is independent of pi , .., p j−1 - if p j is dependent upon one of them, the stubborn set


1464

Fig. 4 The reduced reachability graph derived from PMSs in the pager specification.

Then, an immediate precedence relation PRω is defined as a binary, irreflexive and asymmetric relation on Eω (PRω ⊆ Eω × Eω ) such that, for all events ei , e j ∈ Eω , (ei , e j ) ∈ PRω if it satisfies two following conditions: 1) e j occurs after ei in ω and 2) ei and e j are immediately dependent events, i.e., each event between ei and e j in ω is independent of ei or e j . Therefore, all sequences equivalent to ω preserve the precedence orders in PRω . In this paper, (ei , e j ) ∈ PRω is denoted by (ei → e j ). A transitive closure of PR is denoted by PR+ω and (ei , e j ) ∈ PR+ω is denoted by (ei ⇒ e j ). Algorithm 3 describes a process of generating PRω from a sequence ω in which the number of events is l. In the algorithm, ω[i] denotes the ith event of ω. Note that the algorithm takes advantage of a dependency relation D on a set of external events that conforms to the dependency relation on a set of PMSs. Algorithm 3 (Generation of Precedence Relation)

should contain pi by Algorithm 2. Because independent PMSs affect the different threads, p j is also enabled in ci−1 and ω = p1 , p2 , .., pi−1 , p j , pi , .., p j−1 , .., pn is equivalent to ω. Furthermore, the reduced reachability graph includes the path for the subsequence p1 , p2 , .., pi−1 , p j from c0 to the configuration c j . By repeating the above process for ω , we can obtain the equivalent interleaving ω . ω is the representative interleaving of ω and it can be generated from the reduced reachability graph. Therefore, the following interleavings derived from the reduced reachability graph in Fig. 4 are all of representative interleavings of the pager system. It means that the pager system has 3 different types of behaviors according to internal sequences - E11, E12, E21, and E22 denote the internal occurrence of signify, page, change MV, and change MB, respectively. - ω0 = E11, E21, E22, E12 p2 - ω1 = E11, E21, E12 p3 , E22 - ω2 = E11, E12 p2 , E21, E22 3.3 Equivalent Class Construction From each representative interleaving, we generate an equivalent class in the form of an automaton. Basically, the automaton can be used to control the program’s execution to follow whichever path that includes the implemented sequence among equivalent sequences [9]. In this case, the automaton must accept all equivalent sequences since there is no prior knowledge about which sequences have been implemented in the program. Therefore, with a single execution, automata-based execution can determine whether the program reveals the behavior corresponding to the automaton or not. Our approach constructs such an automaton by two sub-activities: 1) generating an immediate precedence relation from a representative sequence and 2) generating an automaton from an immediate precedence relation. In order to define an immediate precedence relation, let Eω be the set of events in a representative sequence ω.

1. Let PRω = { } and let PR+ω = { }. 2. pre = from l-1 to 1 and post = from pre + 1 to l: 2.1. if (ω[pre], ω[post]) ∈ Dω and (ω[pre] ⇒ ω[post]) PR+ω , perform the steps A and B. A. add (ω[pre] → ω[post]) to PRω and (ω[pre] ⇒ ω[post]) to PR+ω . B. add (ω[pre] ⇒ ω[i]) to PR+ω such that (ω[post] ⇒ ω[i]) ∈ PR+ω . 3 Return PRω . Given a representative sequence ω, Algorithm 3 starts with the last event of ω and chooses a pair of events. If two events, ω[pre] and ω[post], are interdependent and (ω[pre] ⇒ ω[post]) does not exist yet in PR+ω , then the two events have an immediate precedence (ω[pre] → ω[post]). After the algorithm adds (ω[pre] → ω[post]) to PRω , it recomputes its transitive closure PR+ω . Note that PRω always returns an immediate precedence relation for ω because the algorithm compares all pairs of events and detects an immediate precedence. As an example, we apply Algorithm 3 to a representative interleaving ω = E11, E21, E22, E12 p2 whose dependency relation D is shown in Table 2. The algorithm first chooses the pair of ω[3] = E22 and ω[4] = E12 p2 since the number of events in ω is 4 (l = 4). Because they are interdependent in D and (E22 ⇒ E12 p2 ) PR+ω , (E22 → E12 p2 ) is added to PRω . So, PRω = {(E22 → E12 p2 )} and PR+ω = {(E22 ⇒ E12 p2 )}. Next, the pair of ω[2] = E21 and ω[3] = E22 are chosen. They are also interdependent and (E21 ⇒ E22) PR+ω , (E21 → E22) is added to PRω . So, PRω = {(E21 → E22), (E22 → E12 p2 )} and PR+ω = {(E21 ⇒ E22), (E22 ⇒ E12p2 ), (E21 ⇒ E12p2 )}. In the case of ω[2] = E21 and ω[4] = E12 p2 , although they are interdependent, their order is not an immediate precedence since PR+ω already contains (E21 ⇒ E12 p2 ). As a result, PRω with the following precedences can be generated from ω. Figure 5 (A) presents the PRω in the form of a graph called the precedence graph.


1465

- From E22: (E22 → E12p2 ) - From E21: (E21 → E22) - From E11: (E11 → E12p2 ) Equivalent sequences to a representative sequence ω are the sequences which preserve PRω . We generate an automaton which accepts all such equivalent sequences. Algorithm 4 describes a process of generating an automaton from PRω . In the algorithm, a node n reachable from an initial node n0 is represented through a set of executed events, where n0 is an empty set. An automaton AT ω is represented as a set of transitions triggered by events, where a transition t = a source node, an event, a target node . Eω is a set of events in ω. Algorithm 4 (Generation of Automaton) 1. Let denote an automaton by AT ω = { } and a set of nodes by Nω = {n0 }. 2. For each node n in Nω : 2.1. find all executable events e ∈ (Eω − n) such that (e → e) PRω for all e ∈ (Eω - (n ∪ {e})), A. n = n ∪ {e}. add n to Nω if n Nω . B. add a transition n, e, n to AT ω . 3 Repeat step 2 for all nodes, and then return AT ω . The key step here is to find an executable event. In the line 2.1 of Algorithm 4, two events e, e ∈ (Eω − n) means that they are unexecuted until n is reached from n0 and (e → e) PRω means that e has no preceding event among unexecuted events. Therefore, e is executable in n without violating the precedence orders in PRω . As an example, we apply Algorithm 4 to PRω given in Fig. 5 (A). In the initial node n0 , all events are unexecuted events, but E11 and E21 are executable events which have no preceding event. From E11, a node n1 = {E11} is added to Nω and a transition n0 , E11, n1 is added to AT ω . From E21, n2 = {E21} and n0 , E21, n2 are added to Nω and

Fig. 5 For the sequence E11, E21, E22, E12 p2 , (A) its immediate precedence relation and (B) its automaton.

AT ω , respectively. In n1 , E21 is an executable event since E22 and E12 p2 have preceding events. So, n3 = {E11, E21} is added to Nω and n1 , E21, n3 is added to AT ω . In n2 , both E11 and E22 are executable events since E12p2 has the preceding events. From E11 in n2 , the target node becomes {E11, E21} but it is an element n3 of Nω . So, n2 , E11, n3 are added to AT ω . In this fashion, an automaton AT ω is generated as shown in Fig. 5 (B) which accepts three sequences equivalent to the sequence ω = E11, E21, E22, E12 p2 . - E11, E21, E22, E12 p2 - E21, E11, E22, E12 p2 - E21, E22, E11, E12 p2 We are going to prove that an automaton generated by our approach accepts only the sequences equivalent to a test sequence. By Algorithm 4, the automaton preserves an immediate precedence relation of a test sequence and so the sequences that are accepted by an automaton have the same orders of dependent events. By the definition of the equivalence relation on sequences, those sequences are equivalent to each other. Now, we prove by contradiction that an automaton excludes no equivalent sequence. Assume that an automaton accepts a sequence ω = e1 , e2 , .., ei , ei+1 , .., el but does not accept its equivalent sequence ω = e1 , e2 , .., ei , e j , .., el , where ei+1 e j . Then, the automaton does not include the transition triggered by e j in a node ni = {e1 , e2 , .., ei }. However, Algorithm 2 does not add a transition only when there is (e → e j ) where e is also unexecuted in ni (that is, e ni ). While ω to be accepted by the automaton satisfies such a (e → e j ), ω violates the same (e → e j ) since e j is executed before e . It means that (e → e j ) ∈ PRω but (e → e j ) PRω . ω is not equivalent to ω due to PRω PRω . This contradicts the assumption; therefore, the automaton accepts all sequences equivalent to a test sequence. Figure 6 shows the automata generated from three rep-

Fig. 6

The automata generated from representative interleavings.


1466

resentative interleavings in the pager specification. In this way, our approach can separate all possible interleavings derived from the complete reachability graph into 3 equivalent classes. If a program implements at least one interleaving for each equivalent class, the program is synchronization reduction of the pager specification. 4. Experiments In this section, we discuss on an experiment which has been designed to show the effectiveness of our approach to generating equivalent classes from Statecharts and to compare the efficiency in sequence management of our approach and the flattening method to two examples: one is the pager system previously discussed so far and the other is a gas station system. These examples are typical concurrent programs that consist of concurrent threads to communicate with each other. In order to conduct the experiments, we have developed a tool with which we can generate automata from a specification. The automaton is then used as a test controller which controls test execution in order to check whether a target program implements at least one of those sequences accepted by the automaton. Refer to [9] for details about how to use an automaton to construct a test controller and how to control test execution. 4.1 The Pager Example In test sequence generation by applying the flattening method and our approach, Table 3 summarizes the size of the constructed reachability graphs and the size of the constructed automata. The last column gives the number of interleavings derived from the reachability graphs and the number of interleavings accepted by automata. Our approach generates three automata from the pager specification, which correspond to three different types of the pager’s behaviors. With each automaton, the automatabased testing guides a program to follow an execution path that is consistent with a sequence accepted by the automaton. Because three automata covers all of 6 sequences and each automaton only accepts equivalent sequences, we can cover three different types of behaviors with three executions of automata-based testing and we can check whether a program is a synchronization reduction of the specification. In conventional concurrent program testing, the flattening method generates all of 6 possible sequences and the deterministic testing guides a program to follow an execution Table 3 The comparison among graphs derived from the pager specification: ReachabilityC and ReachabilityR are the complete reachability graph and the reduced reachability graph, respectively. Approaches Flattening Stepwise

Graphs ReachabilityC ReachabilityR Automaton0 Automaton1 Automaton2

nodes 12 8 7 7 7

edges 17 9 9 9 9

sequences 10 3 3 4 3

path that is consistent with each sequence. Because the conventional method offers no hint that one sequence is equivalent to any other sequence, we need 6 executions of deterministic testing to cover three different types of behaviors in the worst cast. Since controlling execution paths is generally a time-consuming work, the reduced number of test executions also reduces the overall costs of concurrent program testing. Moreover, the speed of automata-based testing is slightly faster than that of deterministic testing. For example, deterministic testing with ω0 allows the execution of E11 but blocks the execution of E21 which can occur before E11 in the program. However, automata-based testing with Automaton0 allows the executions of either E11 or E21 in the program. By tailoring partial-order methods, our approach may have an additional advantage that its subgraphs, a reduced reachability graph and automata, can be constructed stepwise. Moreover, we find that each automaton can be individually constructed. Therefore, each phase in our approach manages one of subgraphs of a reachability graph and each subgraph contains manageable sequences, which means that our approach can efficiently manage larger programs than the flattening method can handle. 4.2 The Gas Station Example A gas station problem is frequently employed as a typical example of concurrent programming. A gas station system consists of customers, pumps and operators, and its behavior is described by the following process. A customer selects a pump and prepays the operator. If the selected pump is not being used, the operator can activate it, otherwise a customer waits for his/her turn. After it is activated, a customer starts pumping, finishes pumping and asks the operator for change. The operator returns changes to the customer. The gas station example considered in this paper consists of three customers, two pumps and one operator. Figure 7 shows the Statecharts specification of the gas station example that consists of those six threads. In the specification, we assume that both Customer 1 and Customer 3 use Pump 1 and Customer 2 uses Pump 2. In this case, only customers are active threads in which events can occur in an arbitrary order and the occurrences of events should be controlled in concurrent program testing. For understandability, Eij denotes occurrence of the jth event in the ith customer: for j, 1 means “prepay”, 2 means “start pumping”, 3 means “finish pumping” and 4 means “ask for change”. Figure 8 shows a snapshot given by the tool for the gas station system. The tool first elicits 18 PMSs of Eij from the gas station specification. Because the gas station consists of six components, (i.e., three customers, one operator, and two pumps), a configuration c is defined by c = [Customer 1, Customer 2, Customer 3, Operator, Pump 1, Pump 2]. In the next step, the dependency table is generated by comparing PMSs. Then 30 representative interleavings are generated during the selective search of the state space of the gas station system. Finally, an automa-


1467

Fig. 7

The Statecharts specification of the gas station example. Table 4 The comparison among graphs derived from the gas station in the case that it begins with prepaying of Customer 1: ReachabilityC and ReachabilityR are the complete reachability graph and the reduced reachability graph, respectively. Approaches Flattening Stepwise

Fig. 8

Graphs ReachabilityC ReachabilityR Automaton0 Automaton3 Automaton7 Automaton12 Automaton14 Automaton16 Automaton18 Automaton22 Automaton26 Automaton28

nodes 41 29 13 21 21 21 21 29 13 21 21 29

edges 68 34 12 28 28 28 28 44 12 28 28 44

sequences 330 10 1 19 15 15 19 141 1 19 15 85

Test Case Generator: products to the gas station example.

ton is generated from each representative sequence. For instance, the automaton for the sequence, Seq16:E11, E12, E13, E21, E22, E23, E14, E31, E32, E33, E24, E34 accepts 141 sequences which are equivalent to it. In our approach, 30 representative sequences were generated and 30 automata were generated from them. This means that the gas station system have 30 different types of behaviors. Because 30 automata covers all of 990 sequences to be generated from a (complete) reachability graph and each automaton only accepts equivalent sequences, we can cover 30 different types of behaviors with 30 executions of

automata-based testing, instead of 990 executions of deterministic testing. As a result, we can assure that our approach is suitable for testing a program based on synchronization reduction and reduces the number of test executions from the number of all possible sequences to the number of all equivalent classes. In concurrent program testing, test execution is generally the time-consuming work, and so, the reduced number of test executions reduces the overall costs of testing. By tailoring partial-order methods, our approach may have an additional advantage that each phase in our approach manages one of subgraphs of a reachability graph. Table 4 lists the sizes of the graphs derived from the gas


1468

station specification in the case that the gas station begins with prepaying of Customer 1. Since each subgraph contains manageable sequences and our approach constructs those subgraphs step by step, we can assure that our approach manages larger programs than the flattening method can handle. 5. Related Work In this section, we describe related work on specificationbased testing of concurrent programs. We can include the flattening method as a typical example. It generates all possible interleavings of concurrent events in a specification and constructs a reachability graph [4], [5], [17], [18] or a finite state machine (FSM) involving no concurrency [6], [19], [20]. Test sequences are selected from the reachability graph or the FSM. The FSM describes equivalent behaviors of the specification and diverse coverage criteria in state-based testing [19], [21], [22] can be applied to them for selecting test sequences. However, the flattening method offers no information about equivalence of sequences. Moreover, it limits to explore all interleavings of concurrent events and enumerated all possible sequences at a time. Constraints-based testing [1], [2], [23] classified relationships between dependent events into three types of sequencing constraints: namely always valid, possibly valid and never valid. They represent, respectively, the cases where the front event should, may and should not precede the rear event. Then, test sequences were selected by the so called CSPE-1 coverage criterion for sequencing constraints: the CSPE-1 coverage criterion requires that each always valid or never valid constraint is covered at least once and the true and false situations of each possibly valid constraint are covered at least once. However, constraintsbased testing had some limitations due to generality of sequencing constraints. One limitation is that some sequences may be excluded. In the case of always valid, a concurrent program must satisfy the sequencing constraint in all states by the definition. However, the CSPE-1 coverage criterion requires covering it at least once and so the CSPE-1 coverage criterion is satisfied by confirming it not in all states but in a specific state. Therefore, constraints-based testing may exclude the sequences including the sequencing constraints in the other states. The other limitation is difficulty in acquiring the sequencing constraints from a Statecharts specification. A concurrent program must satisfy sequencing constraints in all states by the definition. Therefore, for acquiring sequencing constraints, it should be checked that a concurrent program satisfies each sequencing constraint in all states. Incremental integration testing [3] is another approach to concurrent program testing which alleviates the stateexplosion problem by reducing equivalent behavior. Its target specification is ALTS (annotated LTS), which describes behavior of each concurrent thread, and it incrementally generated a composite ALTS by combining ALTSs. Test se-

quences were selected from the composite ATLS, and then it was reduced for next composition. Incremental testing used three reduction procedures: Collapse, τ-Elimination and Prune. These reduction procedures were based on τ event which was a pair of a sender event and its receiver event and so they reduced equivalent sequences of internal events. Because communications in ALTS are described with one-to-one relations of internal events, τ event is useful for analyzing ALTS. However, communications in Statecharts are described with a broadcast communication mechanism. 6. Conclusion In this paper, we have presented the approach to generating test sequences from Statecharts for concurrent program testing. In a concurrent program, events in active threads can occur without any input but in an arbitrary order. A program may yield different results according to the orders of events’ occurrence, and so, the orders should be controlled in concurrent program testing. Our approach identifies the information about the occurrence of events in a Statecharts specification and the information about the dependency relation among the events. Then, it generates representative interleavings, which represent the behavior of a concurrent program, by adopting partial order methods. Finally, it separately generates automata from each representative interleaving, which accepts all equivalent sequences of a representative interleaving. Our approach offers the following advantages. First, the generated automata correspond to different types of behaviors of a program and they are effective in determining whether a program is a synchronization reduction of its specification or not. Second, by the automata, the number of test executions can be reduced from the number of all possible sequences to the number of all automata but we can still cover all possible behaviors of a program at least once. Third, even when all possible sequences are generated, our approach can manage lager programs than the flattening method can handle because our approach uses subgraphs of a complete reachability graph separately. In this paper, we have demonstrated the effectiveness of our approach through experiments. Although the proposed approach has been successful at separate generation of automata, the automata may often limit the state-explosion problem. If a concurrent program includes a lot of concurrent events and few dependent events, the generated automata are similar size to the complete reachability graph. To resolve such a limitation, we plan to incrementally generate a part of an automaton during test execution that is only relevant to a realized execution path. Another future work is to extend our approach to various types of specification languages. Acknowledgments The research was financially supported by Hansung Univer-


1469

sity in the year of 2005. References [1] R.H. Carver and K.C. Tai, “Use of sequencing constraints for specification-based testing of concurrent programs,” IEEE Trans. Softw. Eng., vol.24, no.6, pp.471–490, 1998. [2] B. Karacali and K.C. Tai, “Automated test sequence generation using sequencing constraints for concurrent programs,” Proc. Int’l Symp. on Software Engineering for Parallel and Distributed Systems (PDSE99), pp.97–108, 1999. [3] P.V. Koppol, R.H. Carver, and K.C. Tai, “Incremental integration testing of concurrent programs,” IEEE Trans. Softw. Eng, vol.28, no.6, pp.607–623, 2002. [4] G. Luo, G.v. Bochmann, and A. Petrenko, “Test selection based on communicating nondeterministic finite-state machines using a generalized Wp-method,” IEEE Trans. Softw. Eng., vol.20, no.2, pp.149–162, 1994. [5] A. Bader, S.M. Sajeev, and S. Ramakrishnan, “Testing concurrency and communication in distributed objects,” Proc. 5th Int’l Conf. on High Performance Computing (HIPC98), pp.422–428, 1998. [6] J. Hartmann, C. Imoberdorf, and M. Meisinger, “UML-based integration testing,” Proc. Int’l Symp. on Software Testing and Analysis (ISTTA’00), pp.60–70, 2000. [7] I.S. Chung, B.M. Kim, and H.S. Kim, “A new approach to deterministic execution testing for concurrent programs,” Proc. Int’l Workshop on Distributed System Validation and Verification, pp.E59– E66, 2000. [8] I.S. Chung and B.M. Kim, “A new approach to deterministic execution testing for concurrent programs,” IEICE Trans. Inf. & Syst., vol.E84-D, no.12, pp.1756–1766, Dec. 2001. [9] H.-S. Seo, I.S. Chung, B.M. Kim, and Y.R. Kwon, “The design and implementation of automata-based testing environment for Java multi-thread programs,” Proc. Asia-Pacific Software Engineering Conf. (APSEC2001), pp.221–228, 2001. [10] D. Harel, “Statecharts: A visual formalism for complex system,” Science of Computer Programming, vol.8, pp.231–274, 1987. [11] D. Harel and A. Naamad, “The STATEMATE semantics of statecharts,” ACM Trans. Softw. Eng. Methodol., vol.5, no.4, pp.293– 333, 1996. [12] K. Bogdanov and M. Holcombe, “Statecharts testing method for aircraft control systems,” Software Testing, Verification and Reliability, vol.11, no.1, pp.39–54, 2001. [13] A. Pnueli and M. Shalev, “What is in a step: On the semantics of statecharts,” Int’l Conf. on Theoretical Aspects of Computer Software (LNCS-526), pp.244–264, 1991. [14] P. Godefroid, Partial-Order Methods for the Verification of Concurrent Systems: An Approach to the State-Explosion Problem, Springer-Verlag, 1996. [15] A. Valmari, “Stubborn sets for reduced state space generation,” Advances in Petri Nets 1990 (LNCS-483), pp.491–515, 1991. [16] K.C. Tai and B. Karacali, “On Godefroid’s state-less search technique for testing concurrent programs,” Proc. 5th Int’l Symp. on Autonomous Decentralized Systems, pp.77–84, 2001. [17] G.v. Bochmann and A. Petrenko, “Protocol testing: Review of methods and relevance for software testing,” Proc. Int’l Symposium on Software Testing and Analysis, pp.109–124, 1994. [18] B. Sarikaya, G.v. Bochman, and E. Cerny, “A test methodology for protocol testing,” IEEE Trans. Softw. Eng., vol.SE-13, no.5, pp.518– 531, 1987. [19] H.S. Hong, Y.G. Kim, S.D. Cha, D.H. Bae, and H. Ural, “A test sequence selection method for statecharts,” Software Testing, Verification and Reliability, vol.10, no.4, pp.203–227, 2000. [20] B.Y. Tsai, S. Stobart, N. Parrington, and I. Mitchell, “An automatic test case generator derived from state-based testing,” Proc. AsiaPacific Software Engineering Conf. (APSEC98), pp.270–277, 1998. [21] T.S. Chow, “Testing software design modeled by finite-state ma-

chine,” IEEE Trans. Softw. Eng., vol.SE-4, no.3, pp.178–187, 1978. [22] D. Lee and M. Yannakakis, “Principles and methods of testing finite state machine — A survey,” Proc. IEEE, vol.84, no.8, pp.1089– 1123, 1996. [23] F.J. Daniels and K.C. Tai, “Measuring the effectiveness of method test sequences derived from sequencing constraints,” Proc. Conf. on Technology of Object-Oriented Language and Systems (TOOLS), pp.74–83, 1999.

Heui-Seok Seo received the B.S. and M.S. degrees in computer science from the Korea Advanced Institute of Science and Technology (KAIST) in 1998 and 2000 respectively. He is presently a Ph.D. student of software engineering in the Department of Electronic Engineering and Computer Science at the KAIST. His research interests include software testing for concurrent programs.

In Sang Chung received the B.S. degree in computer engineering from Seoul National University in 1987, and the M.S. and Ph.D. degrees in computer science from the Korea Advanced Institute of Science and Technology (KAIST) in 1989 and 1993, respectively. After getting the Ph.D. degree he spent half year in ETRI as a post-doc researcher, where he worked on the development of a tool for debugging parallel and distributed programs. From 1994 to 1999, he was a faculty member of Hallym University. In 1999, he joined a faculty member of the School of Computer Engineering of Hansung University. His research interests include software testing for concurrent/distributed programs, object-oriented program testing based on UML models, and application of formal methods such as model checking to test case generation.

Yong Rae Kwon received the B.S. and M.S. degrees in physics from Seoul National University in 1969 and 1971 respectively, and a Ph.D. in physics from the University of Pittsburgh in 1978. He taught as an instructor at the Korea Military Academy from 1971 to 1974. He was on the technical staff of the Computer Science Corporation from 1978 through 1983 working on the ground support software systems for NASA’s satellite projects. In 1983, he joined the Faculty of the Computer Science of the Korea Advanced Institute of Science and Technology (KAIST). He is working as a Professor of Software Engineering in the Department of Electronic Engineering and Computer Science at the KAIST. His research interests include verification of real-time parallel software, object-oriented technology for real-time systems, and quality assurance for high-dependable software.