A Technique to Generate Feasible Tests for Communications Systems

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 11, NO. 5, OCTOBER 2003

1

A Technique to Generate Feasible Tests for Communications Systems with Multiple Timers ¨ Mariusz A. Fecko, M. Umit Uyar, Ali Y. Duale, Paul D. Amer

Abstract— We present a new model for testing real-time protocols with multiple timers, which captures complex timing dependencies by using simple linear expressions involving timerrelated variables. This new modeling technique, combined with the algorithms to eliminate inconsistencies, allows generation of feasible test sequences without compromising their fault coverage. The model is specifically designed for testing to avoid performing full reachability analysis, and to control the growth of the number of test scenarios. Based on extended finite state machines, it is applicable to languages such as SDL, VHDL, and Estelle. The technique models a realistic testing framework in which each I/O exchange takes a certain time to realize and timers can be arbitrarily started or stopped. A software tool implementing this technique is used to generate test cases for the US Army wireless standard MIL-STD 188-220. Index Terms— conformance testing; test case generation; timing constraints; testing timers

I. I NTRODUCTION

E

FFICIENT algorithms are essential for test generation for network protocols with timer requirements. This paper contributes towards achieving this goal by presenting a new method for testing protocols with multiple timers. The methodology includes a novel model (introduced in Ref. [16]) that captures complex timing dependencies by using simple linear expressions with timer-related variables. However, the conflicting conditions based on these variables are likely to make test sequences unrealizable. This problem is tackled by the INconsistencies DEtection and ELimination algorithms (INDEEL) [12], [38] to remove inconsistencies in EFSM [26] models, which are augmented here to efficiently handle timer-related variables. The new model presented in this paper offers several advantages over the existing models for testing timed systems (for example, see [15], [20], [22], [33]). It is specifically designed for testing purposes, allows intuitive modeling of a timed system, algorithmically finds proper timeout settings, and is readily applicable to languages such as SDL [21], [35], VHDL [38], and Estelle [17]. The presented methodology, under widely accepted assumptions [19], [25], detects all single 1clock and n-clock timing faults [15], and certain faults due to incorrect settings for timer lengths. M.A. Fecko is with Applied Research, Telcordia Technologies, Inc., Piscataway, NJ. M.U. Uyar is with EE Dept., CCNY, City University of New York, NY. A.Y. Duale is with Syst. Architecture Compliance, IBM, Poughkeepsie, NY. P.D. Amer is with CIS Dept., University of Delaware, Newark, DE. M.A. Fecko initiated the timed EFSM model [16] at U Delaware. A.Y. Duale co-designed the INDEEL [12] at CUNY. They were then funded by the ATIRP Consortium sponsored by U.S. Army Research Lab (ARL) under the FedLab Program, Cooperative Agreement DAAL01-96-2-0002.

This paper is organized as follows. Section II presents the conflicting timers problem and the related work. The novel model and its properties are introduced in Sections III and IV, respectively. Section V illustrates an application of the methodology to an example FSM. The INDEEL algorithms and their adaptation for testing systems with timers, the faultcoverage analysis, and application to real-life protocols are described in Section VI. II. C ONFLICTING T IMERS P ROBLEM The conflicting timers problem is a special case of the feasibility problem of test sequences, which is an open research problem for the general case [7], [19], [25]. An efficient solution to this special case is possible due to two simplifying features of the conflicting timers problem: (1) timer-related variables are linear, and (2) the values of timekeeping variables implicitly increase with time. The goal of the presented technique is to cover every feasible state transition defined in the specification at least once. In Section VI-D we prove that the above criterion is sufficient to detect 1- and n-clock timing faults as well as incorrect settings for timer lengths. During testing, when a node vp is visited, an efficient test sequence should either (1) traverse as many self-loops (i.e., transitions that start and end in the same state) as possible before a timeout, or (2) leave vp immediately through a nontimeout transition. Once the maximum allowable number of self-loops are traversed, a test sequence may leave vp . This approach does not perform full reachability analysis; however, considering only the above two cases is sufficient to include at least one feasible path for each transition, provided such a path is not prohibited by the original specification (Section IV). In more complex cases, in addition to the timing constraint, traversal of a self-loop requires that its time condition be satisfied, i.e., certain timers be active (or, similarly, other timers be inactive). The model in Section III enables generation of a low-cost test sequence that follows the above guidelines, satisfies time conditions of all composite edges, and is not disrupted by timeouts (i.e., contains only feasible transitions). A. Benefits of new model The related work on testing timed systems focuses on Timed Automata (TA) [3], [33] and testing architectures [29]. Other formalisms used for testing real-time systems are timed extensions of temporal logic [28] and of a formal description technique LOTOS such as RT-LOTOS [9] and ET-LOTOS

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 11, NO. 5, OCTOBER 2003

[4]. Reported results on TA-based testing have several shortcomings. Some produce a prohibitively large number of test cases [33], or sample the time space with random granularity [15], which may not be relevant to a real transition’s duration defined by the specification. Other methods may introduce nondeterminism and infeasible tests [22], while some may take too long to choose a feasible test sequence from the generated set [20]. The new model presented in this paper offers several advantages over the modeling based on TA’s region graph [3] when applied to test-case generation: • It is tailor-designed for testing, without requiring full reachability analysis (Section IV-B). • It allows to define a timer length as a constant or variable rather than a fixed value as in TA to model flexible timeout settings (Section IV-D). • It allows intuitive modeling of an implementation and testing procedure: each I/O exchange is assigned certain time to realize (TA use instantaneous transitions); timers remain in either on or off state (they are always on in TA) (Section IVD). • It uses the paradigm of EFSM [25], which makes it easily applicable to the languages such as SDL [35], VHDL [38], and Estelle [17]. For example, it is straightforward to model the time extensions for SDL [21] (Section III-D). • It allows testing timed systems with well studied EFSM- and FSM-based test generation methodologies [12], [25], [26]. While designed to efficiently limit the growth of the state space of a conflict-free graph, the augmented INDEEL may or may not produce smaller graphs than the minimization techniques for TA [2]. However, the above benefits justify the introduction of the new methodology. III. N OVEL T ESTING M ODEL A protocol can be modeled as a deterministic, completely specified FSM (Mealy) represented by a directed graph G(V, E). An F SM = h S, A, O, δ, λ i, where S, A, O, δ, and λ are a finite set of states, a finite set of inputs, a finite set of outputs, a state transition function δ : S × A → S, and an output function λ : S × A → O, respectively. In the presented model, F SM is extended with a set of timers K = {tm1 , . . . , tm|K| }. The state transition function becomes δK : S × A × K → S, and an output function becomes λ : S × A × K → O. As part of this model, we introduce a set of constants and a set of variables V = {T1 , f1 , . . . , T|K| , f|K| }, as defined below. For each tmj , we introduce the following parameters: • Tj ∈ {0, 1}—boolean variable indicating whether a timer is running (Tj = 1) or not (Tj = 0); + • Dj ∈ R —timeout value (i.e., timer length); + • fj ∈ R ∪ {0, −∞}—time-keeping variable denoting the current time of tmj , which is running when 0 ≤ fj < Dj , and not running (expired or stopped) when fj ≥ Dj or fj = −∞. fj is set to 0 or −∞ when tmj is started or stopped/expired, respectively. Let us define EX(T1 , . . . , T|K| ) as the set of all boolean expressions on T1 , . . . , T|K| . A transition ei ∈ E is associated with:

2

ci ∈ R+ —the time needed to traverse ei ; • time condition hφi i—ei can trigger only if its associated time formula φi ∈ EX is satisfied; • action list {ϕi,1 , ϕi,2 , . . .}—each action ϕi,k is an ordered pair (x ∈ V, β(x) ∈ EX(V, R, {+, −, ∗, /})). For example, if no time formula is associated with ei , its time condition is defined as h1i; if ei ’s time condition involves φi = T1 ∧¬T3 , the transition can trigger only if tm1 is running and tm3 is not running. Function β(x) belongs to the set of all linear expressions on V, the real numbers R, and arithmetic operands. It is used to update x’s value, e.g., {T1 = 1; f2 = f2 + 5} means: start tm1 and increment the value of the timekeeping variable for tm2 by 5. •

A. Limiting number of test scenarios To limit the number of test scenarios, G with variables V is first converted into G0 (V 0 ≡ V, E 0 ≡ E), while imposing graph traversal constraints through (1) merging self-loop edges, and (2) forcing an order to traverse certain transitions. First, a set of variables V and parameters defined in G is enhanced in G0 as follows: V0

= V ∪ {L1 , . . . , L|V | , ts1,1 , . . . , t|V |,M|V | }

Second, for each vp ∈ V , define the following: + s • cp ∈ R —time needed to traverse a self-loop of vp s • Np,l —a set of merged non-timeout self-loops of vp sharing the same time condition hφp,l i, where 1 ≤ l ≤ Mp s • Mp —the number of sets of Np,l for node vp s s s • tp,l —the number of untested self-loops in Np,l . tp,l is s initialized to |Np,l | • Lp ∈ {0, 1, 2}—the ‘exit’ condition for state vp . The majority of self-loops are inopportune transitions with comparable traversal times, and can therefore be approximated with one value of cp . In addition, any self-loop that starts/stops a timer is not merged with others. If it were, the shared time condition might change when a set of merged self-loops were executing, possibly making the remaining transitions in the set infeasible. The ‘exit’ condition is interpreted as follows: if Lp = 0, no transition outgoing from vp and no timeout transition in vp may be traversed; if Lp = 1, a test sequence may traverse any vp ’s outgoing non-timeout transition; if Lp = 2, any outgoing transitions (including timeouts) may be traversed. Graph G0 is then converted by the INDEEL into a conflictfree graph G00 to derive a feasible test sequence. B. Types of transitions We distinguish four types of transitions in graph G0 : Type 1: timeout transition eji = (vp , vq ), defined for each timer tmj (eji may be a self-loop, i.e., p = q) Type 2: non-timeout transition ei = (vp , vq ), which is either a self-loop that starts/stops a timer or a non-self-loop Type 3: merged self-loop transition ep,l = (vp , vp ), defined s for each node vp and each set Np,l Type 4: merged self-loop transition ejp,l = (vp , vp ), defined s for each node vp , each set Np,l that contains more than one self-loop, and each timer tmj

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 11, NO. 5, OCTOBER 2003

While visiting vp , a test sequence will be guided by the following rule: If there is enough time to test all self-loops of s Np,l before any timer expires, ep,l (Type 3) will be traversed; otherwise, ejp,l (Type 4) will be traversed with tmj expiring s before all self-loops of Np,l can be tested. C. Conditions A number of timing constraints must be appended to the time conditions for all transitions, as defined below. For each timeout transition eji = (vp , vq ) (Type 1), the following condition holds for each tmk6=j : ‘exit’ condition for timeouts in vp true AND tmj running AND (tmk not running OR tmj expires before tmk ), which formally is: h(Lp == 2) ∧ Tj ∧ (¬Tk ∨ (Dj − fj < Dk − fk ))i h(Lp == 2) ∧ Tj ∧ (Dj − fj < Dk − fk )i

≡ (1)

For each non-timeout ei = (vp , vq ) (Type 2), the following condition holds for each tmk : ‘exit’ condition for vp true AND (tmk not running OR there is time left to tmk ’s timeout). Formally, this condition is: h(Lp > 0) ∧ (¬Tk ∨ (fk < Dk ))i ≡ h(Lp > 0) ∧ (fk < Dk )ii

(2)

3

increment tmk ’s current time by the sum of ei ’s traversal time and the amount of time left until tmj ’s timeout: fk = fk + ci + max(0, Dj − fj ) • set tmj ’s time-keeping variable: fj = −∞ j • if ei is a non-self-loop, it should set the ‘exit’ condition for its end state vq to 1: Lq = 1 Since ‘max’ is not a linear action, eji should be split into eji,1 and eji,2 as follows: •

h(Lp == 2) ∧ Tj ∧ (fj ≥ Dj ) ∧ (Dj − fj < Dk − fk )i {Tj = 0; fk = fk + ci ; fj = −∞} (5) h(Lp == 2) ∧ Tj ∧ (fj < Dj ) ∧ (Dj − fj < Dk − fk )i {Tj = 0; fk = fk + ci + Dj − fj ; fj = −∞} (6) In Fig. 1, tmj is started at time fj = 0. After fj reaches a value of fj0 , suppose that the two feasible transitions are e1 and e2 . Consider the case where e1 triggers, and fj is advanced to a value of fj1 = fj0 + c1 < Dj . In this case, tmj ’s timeout corresponds to traversing eji,2 , which advances all timers by ci + Dj − fj1 . In the case where e2 triggers, fj is advanced to a value of fj2 = fj0 + c2 > Dj , with tmj ’s timeout modeled by eji,1 . All timers will be advanced due to eji,1 only by ci , because tmj expired while e2 was being traversed.

For each merged self-loop transition ep,l (Type 3), the following condition holds for each tmk : there are untested s self-loops in Np,l AND (tmk not running OR all untested s self-loops of Np,l can be tested before tmk expires). For each s can be tested by traversing ep,l . This ep,l , all self-loops Np,l condition can be formalized as: h(tsp,l > 0) ∧ (¬Tk ∨ (tsp,l ∗ csp < Dk − fk ))i h(tsp,l > 0) ∧ (tsp,l ∗ csp < Dk − fk )i

c2

fj0

(3)

For each merged self-loop transition ejp,l (Type 4), the following condition holds for each tmk6=j : there are untested s self-loops in Np,l AND (tmj running AND there is enough time left before tmj expires to test at least one but not all s untested self-loops in Np,l ) AND (tmk not running OR tmj expires before tmk ). In other words, only some of the selfs loops of Np,l can be tested by traversing ejp,l . Formally: h(tsp,l > 0) ∧ (Tj ∧ (csp < Dj − fj < tsp,l ∗ csp )) ∧ (4) s (¬Tk ∨ (Dj − fj < Dk − fk ))i ≡ h(tp,l > 0) ∧ Tj ∧ (csp < Dj − fj < tsp,l ∗ csp ) ∧ (Dj − fj < Dk − fk )i Note that for a system with only one timer tmj , an inequality (Dj − fj < Dk − fk ) is dropped from the edge conditions (1) and (4). Also note that any nondeterminism due to multiple timeouts can be detected, e.g., if tmj and tmk are to expire simultaneously, then (Dj − fj = Dk − fk ) and their conditions cannot be satisfied. D. Actions A number of actions must be appended to the action lists for all transitions. For each timeout transition eji = (vp , vq ) (Type 1), for each k 6= j: • set variable Tj to 0 indicating timer expiry: Tj = 0

fj1 Dj

≡ Fig. 1.

ci

c1

0

fj

fj2 ci

Time dependencies in timeout transition eji .

For each non-timeout ei = (vp , vq ) (Type 2): • set the ‘exit’ condition for state vq to true: Lq = 1 • ∀k, increment tmk ’s current time by ei ’s traversal time: fk = fk + ci For each merged self-loop transition ep,l (Type 3): • set the ‘exit’ condition for state vp to false: Lp = 0 • ∀k, increment tmk ’s time by the time needed to traverse s all untested self-loops in Np,l : fk = fk + tsp,l ∗ csp s s • reset the number of untested self-loops in Np,l : tp,l = 0 If there are no untested self-loops of vp whose time condition is satisfied, Lp should be set to 2 (from either 0 or 1), enabling timeouts and all outgoing transitions in vp . In this case, Lp will be set to 2 by a so-called observer self-loop transition sp , with a condition: ∀l

h(Lp < 2) ∧ (tsp,l == 0 ∨ (tsp,l > 0 ∧ ¬φp,l ))i

(7)

and an action {Lp = 2}. Condition (7) is satisfied when all feasible self-loops of vp are tested. For each merged self-loop transition ejp,l (Type 4): • set the ‘exit’ condition for vp to true: Lp = 2 • ∀k, increment tmk ’s time by the time needed to traverse s all of the untested self-loops in Np,l that can be tested s before tmj expires: fk = fk + cp ∗ b(Dj − fj )/csp c

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 11, NO. 5, OCTOBER 2003

•

decrement the number of untested self-loops: tsp,l = tsp,l − b(Dj − fj )/csp c

1) Scalability and linearity: To prevent 2Mp parallel edges due to (7), sp is replaced with the set of vertices and edges as depicted in Fig. 2. The appended conditions and actions are derived from (7): sˇp : h1i {Lp = 2}, s0p,l : hLp < 2 ∧ tsp,l == 0i {}, s00p,l : hLp < 2 ∧ (tsp,l > 0 ∧ ¬φp,l )i {}

(8)

Condition (7) is satisfied when a feasible path exists from vp to wp,Mp . Since the edges of s0p,k and s00p,k are mutually exclusive, only one such a path is possible. The outgoing edge of wp,Mp , i.e., sˇp , sets the ‘exit’ condition to true. sp

s'p,2

Wp,1

p

Wp,2

s''p,1

s'p,1 p

Fig. 2.

s''p,2

s''p,Mp šp

s'p,Mp

Wp,Mp

Making an observer transition sp scalable.

Type 4 actions are non-linear, since the number of selfloop traversals before a timeout is computed in ejp,l ’s actions by rounding down a fractional value to an integer z = b(Dj − fj )/csp c. Since the INDEEL are applicable only to linear actions, a straightforward graph modification removes this nonlinearity by avoiding the computation of z. Instead, z extra edges are added, and a test sequence is forced to traverse the z-th edge [16]. IV. M ODEL R EFINEMENT In this section we show that our method achieves the goal to cover every state transition at least once (Section II) through constraints and rules to control graph traversal in the model. It is shown that while the explosive growth of the number of tests from G is significantly reduced, all feasible transitions in G remain so in G0 . A. Rules for graph traversal The ‘exit’ condition Lp , together with the merging of selfloops in Type 3 and Type 4 edges, force the breadth-first search traversal algorithm for graph G0 to automatically follow a number of rules, i.e., through satisfying conditions and executing actions. These rules control the number of generated test scenarios while enabling each functionality of the original system (each such a functionality is represented by a transition in G) to be tested. They guarantee that all feasible edges in vp that could be traversed in G for the given values of variables V = {T1 , f1 , . . . , T|K| , f|K| } will be also traversable in G0 during a visit to state vp .

4

Rule 1: After arriving at vp (which sets Lp to 1), traverse immediately outgoing non-timeout edges (Type 2), or, if possible, a Type 3 or Type 4 edge. Rule 2: After taking Type 3 edge (which sets Lp to 0, preventing a test sequence from leaving a state and timeouts from occurring), traverse further Type 3 edges (if possible). Rule 3: If no Type 3 edge can be taken, traverse a Type 4 edge, if possible (which sets Lp to 2, thus enabling timeouts and outgoing non-timeout edges). Rule 4: If no Type 3 or 4 edges can be taken, let the observer edge enable timeouts and outgoing non-timeout edges. Rule 5: If timeouts for vp are enabled, traverse immediately outgoing non-timeout edges (Type 2) or a timeout edge for the earliest timer to expire (Type 1). Rules 1–5 guide the flow of graph traversal at state vp . After entering vp , only Type 1 transitions are disabled through the ‘exit’ condition Lp . The flow can take transition of Type 2 and, depending on its end state, leave or remain in vp . The flow can also traverse a Type 4 transition or the maximum possible number of feasible Type 3 transitions while remaining in state vp . Afterwards, either a Type 4 transition or an observer edge sp (if no feasible Type 4 transition exists) is traversed, thus enabling Type 1 and 2 transitions. The flow then takes one of these transitions of which at least one should finally leave vp . B. Sets of feasible transitions Consider the following state space defined for V and variables V = {T1 , f1 , . . . , T|K| , f|K| }. U = {(vp , T1 , f1 , . . . , T|K| , f|K| ) : vp ∈ Vp , Tk , fk ∈ V}

(9)

The number of states in U is infinite since the domain of each variable fk is a dense subset of R+ . We consider each transition ei atomic, with the traversal time of ci . Hence, when INDEEL traverses G0 , only those values of each fk that can be set by executing the actions defined in G0 will be considered in U . The relevant state spaces of G and G0 are therefore defined as the subsets of U reachable in G and G0 (denoted 0 ), respectively. as UG and UG Suppose, for the timer with the length of D, f 1 < D when state vp is reached. Before the timer expires in a timeout transition ei = (vp , vq ), the Implementation Under Test (IUT) [26] will move in state space U through a sequence of states (vp , f 1 ), . . . , (vp , D), whose number is infinite. These states, however, are not relevant with respect to the timed behavior of the system, since transitions between them do not affect the IUT’s behavior. Only the boundary states (vp , f 1 ) and (vp , D) are thus considered. Let A be the input set for graphs G and G0 . Let the sets of 0 transitions between states in UG and UG be denoted as RG and 0 RG , respectively. (Graph G and variables in U form an EFSM; to convert this EFSM into FSM represented by (UG , RG ), one can use the well know procedures of [7], [25].) Each gk ∈ RG is derived from the G’s transition ek = (vik , vk2 ) and assigned the following parameters: • ek ∈ E—original transition labeled with (1) ak ∈ A—a tester’s input to an IUT (null for a timeout ek ); (2) ck — ek ’s traversal time; and (3) vk1 , vk2 ∈ V —ek ’s start and end states in G.

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 11, NO. 5, OCTOBER 2003

u1k , u2k ∈ UG —gk ’s start and end states in UG , where u1k = (vk1 , . . .) and u2k = (vk2 , . . .). 1 Definition 1: We say that state u1 = (v1 , T11 , f11 , . . ., T|K| ), 1 2 2 2 where f|K| ) ∈ U precedes state u2 = (v2 , T1 , f1 , . . ., T|K| , 2 f|K| ) ∈ U iff v1 = v2 , (∀k) Tk1 = Tk2 , and (∀k) fk1 ≤ fk2 . Definition 2: A trace of G in state space UG is defined as a feasible sequence of tuples tG = (d1 , g1 ), . . . , (dm , gm ), where, for each gk , input ak is delayed by dk . For a nontimeout gk that starts a timer, dk ∈ R+ ; for a timeout gi = gij , dk = max(0, Dj − fj ), for the others, dk = 0. To achieve satisfactory test coverage, any feasible transition defined for vp in G should remain feasible in G0 (possibly in a different test scenario). Let us now formulate the first two of the sufficient conditions for the sets of feasible transitions in G and G0 to be identical: Condition 1: Given up ∈ UG , all feasible edges in up that could be traversed in G will also be traversed in G0 during a visit to up . Condition 2: For each up reachable in G, there exists u0p reachable in G0 that precedes up , i.e., up ∈ UG ⇒ (∃u0p ∈ 0 ) u0p ≤ up . UG It was shown in Section IV-A that Condition 1 is satisfied in the model while reducing the number of test scenarios by Rules 1–5. To prove that Condition 2 is also satisfied in the model, consider tG , an arbitrary trace of G of length m, where u11 = (v0 , 0, −∞, . . . , 0, −∞) is the system’s initial state and 2 2 2 , T12 , f12 , . . . , T|K| , f|K| ). Trace tG may or may not u2m = (vm 0 be possible in G due to Rules 1–5. However, one can show that, given tG , there exists a corresponding trace t0G of G0 such 1 02 2 that u01 1 = u1 and um ≤ um . Details of the proof are omitted due to the page limitation. •

C. Delaying start of timers 0 The INDEEL create a set Up0 ⊂ UG of copies of state vp , each with different values of variables V = {T1 , f1 , . . . , T|K| , f|K| }. Some of the G’s original transitions defined for vp may be feasible only in a particular subset of Up0 . In Sections IV-A and IV-B it was shown that (1) all feasible transitions defined for up in UG are still traversable when up 0 is reached in UG , and (2) each state up that is reachable in 0 UG has a reachable preceding state in UG . To traverse any feasible transition of the original system, our model must also satisfy the condition: Condition 3: For each vp ’s edge, a particular subset of Up0 in which this edge is feasible will be created. To satisfy this condition, transitions that start a timer must be delayed by certain amount of time before being traversed. The action of delaying such transitions allows us to explore various orderings of timers’ expirations by causing certain timers to expire before others. Besides the rules introduced in Section IV-A, the following rule will be applied to graph G0 traversal. Rule 6: If ei starts a timer, and at least one timer is on when ei is to be traversed, delay ei such that it is traversed before the earliest timeout. Example 1 (Delaying transitions): In the FSM in Fig. 3 (the initial state v0 ), all four transitions e1 through e4 take

5

1 sec to traverse and have the time condition h1i. Timer tm1 (started by e1 ) has the length of D1 = 4 and the timeout transition e3 , and tm2 (started by e2 ) has the length of D2 = 2 and the timeout transition e4 . Transitions e3 and e4 also explicitly stop tm2 and tm1 , respectively. c1 = c2 = c3 = c4 = 1 D1 = 4, D2 = 2

vt eon

v0

e3 - tm1 timeout

eoff e1

v1

e2

v2

e4 - tm2 timeout Fig. 3.

Delaying e1 and e2 affects reachability.

Let us illustrate that e3 may not be traversed if no delaying is used. When the IUT is in v0 , since all timers are inactive, there is no need to delay e1 . Suppose a tester does not delay e2 either. In this case, when v2 is visited, tm1 and tm2 have 3 sec and 2 sec left until expiration, respectively. Timer tm2 will expire first in the timeout transition e4 , which will also stop tm1 . The system goes back to state v0 with all timers stopped and e3 never traversed. On the other hand, when e2 is delayed by more than 1 sec, tm1 and tm2 have less than 2 sec and exactly 2 sec left until expiration, respectively. In this case, tm1 will expire first in the timeout transition e3 , which will also stop tm2 . We can see that by choosing the value of the delay in e2 as 0, a tester can traverse e4 . At another visit to v1 , the value of greater than 1 will make e3 feasible. Formally, Rule 6 can be justified as follows. Consider the following state space defined for states in V and variables V = {T1 , f1 , . . . , T|K| , f|K| }. W

= {(vp , (τj1 , yj1 ), . . . , (τjn , yjn )) : vp ∈ V, Tjk ∈ V}

where τj1 , . . . , τjn are indices of running timers in the order of expiration. Each yjk6=1 is defined as the time between tmjk−1 ’s and tmjk ’s timeouts; yj1 is set to 0. Let WG0 be the subset of W reachable in G0 , with the set of transitions between states 0 0 in WG0 denoted as XG . Each xi ∈ XG is derived [7] from the original G0 ’s transition ei = (vi1 , vi2 ) and labeled with the following parameters: • ei —original transition in E 1 2 0 0 • wi , wi ∈ WG —xi ’s start and end states in WG , respec1 1 1 2 tively, where wi = (vi , . . .) and wi = (vi , . . .) 0 Suppose that transition xi ∈ XG is to be traversed. It is clear that the time-related components (i.e., variables of Ti and yi ) of wi1 and wi2 are identical unless xi is derived from a timeout transition or a transition that starts/stops a timer. All other transitions alter neither the order nor the time between timer expirations and, therefore, it is unnecessary for a tester to delay their inputs. If xi is a timeout transition for tmj , the amount of time a tester can delay xi is independent of the tester’s action and equal to Dj − fj . If xi is not a timeout transition, one of the

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 11, NO. 5, OCTOBER 2003

timers—say tma —is to expire first. Let dm i be the amount of time by which xi is delayed in this case. It is clear that if xi is to be traversed instead of tma ’s timeout, dm i must be less than Da −fa (Fig. 4 (b)). In the case where none of the timers are running before traversing xi (Fig. 4 (a)), dm i will be set to 0 because time passage does not affect system behavior if all timers are inactive. (a)

Dj

ci

(b)

Da - fa dim ci

(c)

Dj

Da - fa dim ci

Dj

Fig. 4. Delaying transition xi : (a) all timers inactive, no delay; (b) tma to expire first, delay less than Da − fa ; (c) tma to expire first, delay greater than Da − fa cannot be applied due to tma ’s timeout.

If xi stops a timer, delaying xi by any 0 < dm i < Da − fa does not result in the end state wi2 different from the end state for a zero delay. Therefore, dm i is set to 0. If xi starts a timer, delaying xi by any 0 ≤ dm i < Da − fa is likely to result in multiple end states wi2 depending on the value of 2 0 dm i . These copies of wi need to be considered in G , since 0 certain transitions in G may be feasible only for a specific copy. To satisfy this condition, each xi will be replaced by a set of transitions Xi . Transition x0i ∈ Xi handles the case with dm i = 0, where all timers are inactive before traversing xi . Transition x0i has the following appended condition for each tmk : tmk not running, i.e., h¬Tk i. The case where dm i is upper bounded by a running timer tma with the shortest time to expire is handled by transitions xai ∈ Xi , defined for each a : 1 ≤ a < |K|. The transitions xai have the following appended condition that holds for each tmk6=a : tma running AND tma is to expire before tmk . Formally, this condition is: hTa ∧ (Da − fa < Dk − fk )i Thus, xi is replaced with |K| + 1 transitions in Xi , out of which only one is feasible, i.e., x0i if no timer is running, or xai for a particular tma that is to expire first. Each xai also has the following appended action: • for each k, increment tmk ’s current time by the introm duced delay: fk = fk + dm i , where 0 ≤ di < Da − fa m The delay of di is involved in actions with lower (0) and upper (Da −fa ) bounds. In the INDEEL, the two inequalities of m dm i ≥ 0 and di < Da −fa must be included in the conditions m involving di . The actual instantiation of dm i , i.e., assigning a particular value from between dm ’s bounds, takes place after i generating a test sequence. In addition, if xi stops or starts tmj , the actions {Tj = 0; fj = −∞} or {Tj = 1; fj = 0} must be appended to xi ’s action list, respectively. Example 1 (cont’d): For the FSM of Fig. 3, consider state 0 space WG0 and its transition set XG . The IUT traverses x1 to

6

enter w1 . In w1 , if no delay is applied before traversing x2 , the IUT will move to state w2 (0) = (v2 , (2, 0), (1, 1)) regardless of a possible delay applied for x1 . If, however, delay dm 2 is applied before x2 is triggered, the IUT may be in multiple states distinguished by the value of dm 2 . For the delay 1 < d12 < 3, x3 is feasible and x4 is not feasible in the states defined by w2 (d12 ) = (v2 , (1, 0), (2, 2)). For 0 ≤ d22 < 1, in any state w2 (d22 ) = (v2 , (2, 0), (1, 1 − d22 )), x4 is feasible and x3 is not feasible. 0 A test sequence can be algorithmically derived in XG as the following parameterized trace: t0G (d12 , d22 ) = (10) (0, x1 ), (d22 , x2 ), (2, x4 ), (0, x1 ), (d12 , x2 ), (3 − d12 , x3 ) where x2 = x12 ∈ X2 , 0 ≤ d12 , d22 < 3, and the accumulated conditions are as follows: x3 : x4 :

h(D1 − f1 < D2 − f2 ) ∧ . . .i ≡ h(d12 > 1) ∧ . . .i h(D2 − f2 < D1 − f1 ) ∧ . . .i ≡ h(d22 < 1) ∧ . . .i

The corresponding trace (test sequence) in E with the instantiated values of d12 = 2 and d22 = 0 is as follows: t0G (2, 0) = (11) (0, e1 ), (0, e2 ), (2, e4 ), (0, e1 ), (2, e2 ), (1, e3 ) Eq. (11) indicates that, when e2 is traversed the first and the second time, the length of its associated delaying timer is set to 2 and 0, respectively. D. Flexible timer settings Let us now illustrate the advantages of our approach with respect to flexible modeling of timeout settings and transition execution time, as listed in Section II-A. If the timer lengths are fixed in advance (as they are, for example, in the TA model) certain portions of the system may become unreachable. It is difficult to predict and manually assign correct timer lengths. Our model offers the capability to define timer lengths as variables, and have the INDEEL assign their values, as shown below. Example 1 (cont’d): Suppose the tentative timeout settings for the FSM of Fig. 3 are D1 = 4 and D2 = 5. For transitions x3 and x4 , the accumulated conditions are: x3 : h(d12 < 3) ∧ . . .i,

x4 : h(d22 < −2) ∧ . . .i (12)

The only possible sequence of delay/edge pairs through the FSM of Fig. 3 is trace t0G (d12 , d22 ) = (0, x1 ), (d12 , x2 ), (3 − d12 , x3 ) containing x3 with feasible condition in (12). During test generation, path (0, x1 ), (d22 , x2 ), (5, x4 ) that contains x4 is also considered, but pruned because its condition in (12) is always false for the timeout settings given above. The initial timeout settings make x4 always infeasible by not allowing the SUT to reach a state where x4 is testable. In consequence, those portions of the graph that can be reached only through x4 become unreachable. In our model, to find the timeout settings that make x4 feasible, D2 is a variable rather than a constant. In this case,

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 11, NO. 5, OCTOBER 2003

7

the following parameterized trace will be obtained: e3

t0G (d12 , d22 , D2 ) = (13) 2 1 1 (0, x1 ), (d2 , x2 ), (D2 , x4 )(0, x1 ), (d2 , x2 ), (3 − d2 , x3 )

v1

x12

e2

where x2 = ∈ X2 , with the following conditions accumulated along the trace: e1

x3 : h(d12 > 3 − D2 ) ∧ . . .i, x4 : h(d22 < 3 − D2 ) ∧ . . .i (14) The linear programming [8] finds a solution for (14) to instantiate (13) as a test sequence in E, e.g.,

V. M ODELING E XAMPLE FSM In the FSM in Fig. 5 (the initial state v0 ), e3 takes 3 sec and each of the remaining seven transitions take 1 sec to traverse. Timer tm1 is started by e2 (with D1 = 5.5 and the timeout transition e8 ), and tm2 is started by e4 and stopped by e2 (D2 = 3.7 and the timeout transition e7 ). Transition e1 is associated with time condition h¬T1 ∧ T2 i, e5 and e6 are associated with time condition hT1 ∧ T2 i, and, the remaining transitions have the time condition h1i. State vt is introduced as the system initialization state, where a test sequence originates and terminates. A test sequence starts in state vt with edge eon : h1i {T1 = 0; T2 = 0; f1 = −∞; f2 = −∞; t0,1 = 1; t1,1 = 1; t2,1 = 2; L0 = 1},

e4 e5

v0

e8 - tm1 timeout

v2

e6

c1 = c0s = 1, c3 = c1s = 3 e7 - tm2 timeout c5 = c6 = c2s = 1 c2 = c4 = c7 = c8 = 1 D1 = 5.5, D2 = 3.7 N0,1s = {e1}, N1,1s = {e3}, N2,1s = {e5, e6}

t0G (2.5, 0, 1) = (0, e1 ), (0, e2 ), (1, e4 ), (0, e1 ), (2.5, e2 ), (0.5, e3 ) for d12 = 2.5, d22 = 0, and D2 = 1. The methodology not only finds D2 = 1 as tm2 ’s length, but also computes the appropriate lengths 2.5 and 0 for e2 ’s delaying timer. In TA there are no procedures to algorithmically obtain the proper timeout settings to make e4 feasible. But in our model, when D2 is interpreted as a variable rather than a fixed value, a proper timer length is obtained as D2 = 1 by solving linear programming system as shown above. Another important aspect of our method is the ability to assign nonzero execution time to each transition. This modeling property is intuitive, since it reflects the case with real systems (e.g., packet transmission, password authentication, and database queries consume time to realize). On the other hand, TA uses instantaneous transitions and let the time passage occur in locations. For graph G(V, E), the TA approach will require adding |E| extra timers, |E| extra states, and |E| additional edges to mimick G with nonzero transition execution times. For large graphs with thousands of edges, but with only a few timers, TA will dramatically increase the complexity of the input model. In the TA model, the only operation that a user can make on clocks is the reset to 0; otherwise, the clocks proceed continuously with time [3]. Suppose a transition ei triggers when tm1 , . . . , tmn are not running. In our model, this condition is naturally represented by h¬T1 ∧ . . . ∧ ¬Tn i. The TA has no memory about a timer being expired, since the timer resumes execution after the reset. To define the above condition in TA, one needs to define extra timers x1 , . . . , xn , started when tm1 , . . . , tmn expire, respectively. These additional timers are not necessary in our model.

1 time unit = 1 sec

Fig. 5.

FSM with conflicting timers tm1 and tm2 .

which initializes all timers and the variables of tp,l . A test sequence terminates when traversing edge eoff : h¬T1 ∧ ¬T2 i {}, bringing the IUT from v0 back to vt . The time condition of eoff ensures that all timers are inactive when the test sequence is terminated. Note that, unlike the regular states v0 through v|V | , vt is not split by the INDEEL—the final inconsistencyfree graph contains only one copy of vt . An example test sequence for the FSM of Fig. 5 begins with (eon , e1 , e2 , . . .). However, it does not satisfy the time condition for e1 : h¬T1 ∧ T2 i, since after traversing eon (initial power-up), neither timer is running. Similarly, any test sequence containing (. . . , e4 , e7 , e5 , . . .) is invalid because e5 ’s time condition requires that both tm1 and tm2 be running, which does not hold after tm2 expires in e7 . Let us consider transitions of Type 1 (e7 , e8 ) and Type 2 (e2 , e4 ), with the following appended conditions and actions (those for e8 are analogous to e7 ): e27,1 : hL2 == 2 ∧ f2 ≥ 3.7 ∧ (3.7 − f2 < 5.5 − f1 ) ∧ T2 i {L2 = 1; T2 = 0; f1 = f1 + 1; f2 = −∞} 2 e7,2 : hL2 == 2 ∧ f2 < 3.7 ∧ (3.7 − f2 < 5.5 − f1 ) ∧ T2 i {L2 = 1; T2 = 0; f1 = f1 − f2 + 4.7; f2 = −∞} e2 : hL0 > 0 ∧ f1 < 5.5 ∧ f2 < 3.7i {f1 = f1 + 1; f2 = f2 + 1; L1 = 1; T1 = 1; f1 = 0; T2 = 0; f2 = −∞} e4 : hL1 > 0 ∧ f1 < 5.5 ∧ f2 < 3.7i {L2 = 1; f1 = f1 + 1; f2 = f2 + 1; T2 = 1; f2 = 0} s Vertex v2 has two merged self-loops in N2,1 = {e5 , e6 }. As a result, transitions of both Type 3 (e2,1 ) and Type 4 (e12,1 , e22,1 ) are defined in v2 . Therefore, appended conditions and actions are obtained as follows:

e2,1 :

hts2,1 > 0 ∧ (T1 ∧ T2 ) ∧ (ts2,1 < 5.5 − f1 ) ∧ (ts2,1 < 3.7 − f2 )i {L2 = 0; f1 = f1 + ts2,1 ; f2 = f2 + ts2,1 ; ts2,1 = 0}

e12,1 :

h(T1 ∧ (1 < 5.5 − f1 < ts2,1 )) ∧ ts2,1 > 0 ∧ (5.5 − f1 < 3.7 − f2 ) ∧ (T1 ∧ T2 )i

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 11, NO. 5, OCTOBER 2003

{L2 = 2; f1 = f1 + 1; f2 = f2 + 1; ts2,1 = ts2,1 − 1} e22,1

:

ts2,1 ))

ts2,1

h(T2 ∧ (1 < 3.7 − f2 < ∧ >0 ∧ (3.7 − f2 < 5.5 − f1 ) ∧ (T1 ∧ T2 )i {L2 = 2; f1 = f1 + 1; f2 = f2 + 1; ts2,1 = ts2,1 − 1}

Since only one self-loop is defined in vertices v0 and v1 , both vertices will have merged self-loop transitions of Type 3 only. For v0 and v1 , merged self-loop transitions e0,1 and e1,1 s s are defined for the sets of N0,1 = {e1 } and N1,1 = {e3 }, respectively, with the appended conditions and actions derived as for e2,1 . TABLE I VALID

TEST SEQUENCE FOR THE

Step 1 2 3 4 5 6 7 8 9 10 11 12 13

Edge eon e2 e3 e4 e5 e8 e1 e2 e4 e6 e7 e8 eoff

Cost 0 1 3 1 1 1 1 1 1 1 1 1 0

T1 0 1 1 1 1 0 0 1 1 1 1 0 0

T2 0 0 0 1 1 1 1 0 1 1 0 0 0

FSM

OF

f1 −∞ 0 3 4 5 −∞ −∞ 0 1 2 5.7 −∞ −∞

F IG . 5 f2 −∞ −∞ −∞ 0 1 2.5 3.5 −∞ 0 1 −∞ −∞ −∞

Consider the test sequence for the FSM in Fig. 5 (Table I). While the test sequence is being executed, the values of timerrelated variables of the model change with the progress of time. After system initialization by transition eon , transition e2 starts tm1 . After arriving at state v1 , there are 5.5 sec left until tm1 ’s timeout; so, transition e1,1 can be tested, which takes 3 sec. After leaving v1 , tm1 has 2.5 sec left until timeout. In transition e4 , timer tm2 is started and the time-keeping variable for tm1 reaches f1 = 4. After the test sequence arrives at state v2 , tm1 and tm2 have 1.5 sec and 3.7 sec left until timeout, respectively—tm1 will therefore expire first. There is not enough time to traverse e2,1 (i.e., to test both e5 and e6 ); therefore, e12,1 is traversed (e5 is tested). This step leaves 0.5 sec and 2.7 sec until timeouts for tm1 and tm2 , respectively. After tm1 expires, the time-keeping variable for tm2 is advanced to f2 = 2.5, which gives enough time (1.2 sec) to traverse e0,1 . Traversing e0,1 is equivalent to testing e1 with the time condition of h¬T1 ∧ T2 i. Since at this point tm1 has expired and tm2 is running, e1 ’s time condition is satisfied and the transition is tested. Afterwards, e2 are e4 are traversed consecutively; already tested e3 is skipped. The test sequence arrives again at v2 , with 4.5 sec and 3.7 sec left until timeouts for tm1 and tm2 , respectively. Now tm2 is to expire first, leaving sufficient time to traverse e2,1 (test e6 ). Then, tm2 expires and the timekeeping variable for tm1 is advanced to f1 = 5.7, exceeding tm1 ’s length by 0.2. Therefore, e8 is traversed immediately, since tm1 expired while e7 was being traversed. Now the IUT is back in its initial state v0 with both timers inactive and all transitions tested, so the test sequence returns to the initialization state vt through eoff .

8

The test sequence shown in Table I satisfies all timing constraints imposed by the two timers tm1 and tm2 . In addition, the time conditions for all transitions in the FSM are satisfied at any time during the test sequence traversal. Section VI presents an algorithmic technique to obtain lowcost test sequences satisfying the above criteria. VI. T EST C ASE G ENERATION Graph G0 , which models the original system along with its timed behavior, represents an EFSM consisting of an FSM and a set of variables V 0 . Having built G0 , we can now apply any EFSM-based test-generation method [7], [25], [30]. In this paper, we augment the original INDEEL [12], [38] algorithms so that they could efficiently handle graph G0 with timerrelated variables. The following sections present the overview of the algorithms and their complexity. A. Inconsistency elimination Feasible test sequences can be generated from the EFSM models if inconsistencies among the actions and conditions are eliminated. In this paper, the INDEEL algorithms are augmented to handle the multiple-timer model of Section III such that only the variables for timers that are active (i.e., running) can cause inconsistencies. The algorithms first detect and eliminate action inconsistencies. Next, they handle condition inconsistencies by employing linear programming techniques. For example, in Fig. 5, the actions of e7 set T2 to 0. Since the time condition of e5 requires that hT2 i, e7 ’s action causes inconsistency with e5 ’s condition. Similarly, a test sequence including e1 and e5 has condition inconsistency—e1 requires that h¬T1 i and e5 that hT1 i. Both test sequences are infeasible. Formally, given the set of variables X and an edge ek , let us define the following: condition: action:

a01 x1 + . . . + a0m xm [op] d xi = a11 x1 + . . . + a1m xm + d

(15) (16)

where m, op, and d are the number of variables, an operator, and a constant, respectively. Definition 3: Edges ek and er have a condition inconsistency if there is no solution for the equations formed by the conditions accumulated in sub-path e1 · e2 · . . . · ek and er , where head(er ) is reachable from or equal to tail(ek ). Definition 4: Edges ek and er have an action inconsistency if there is no solution for the equations formed by the actions of ek and the condition of er , where head(er ) is reachable from or equal to tail(ek ). In the INDEEL, Eqs. (15) and (16) form the rows of matrices representing an edge’s conditions and actions to analyze their interdependence. (Note that, in the augmented INDEEL, only the variables for running timers are included in the equations.) In addition, the actions and conditions accumulated along the paths in the graph are represented by sets of Action Update Matrix (AUM) pairs and Accumulated Condition Matrix (ACM) triplets [12], respectively. Action conflicts are resolved in INDEEL as follows:

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 11, NO. 5, OCTOBER 2003

•

• •

•

Determine the symbolic values [10] of the variables at each node reached in the MBF (modified breadth-first [12]) graph traversal. Pass the effect of actions onto the condition variables. For each node vi , based on the symbolic values of the timer-related variables, determine the feasibility of each outgoing edge whose conditions use differently modified variables at vi . For each action inconsistency, split the graph such that any two edges with inconsistent actions and conditions are placed in two different sub-graphs.

The • •

•

INDEEL

for condition conflicts are outlined as:

Collect the accumulated conditions of the edges leading to vi reached with the depth-first graph traversal. Based on the symbolic values of the timer-related variables, find an edge reachable from vi with conditions inconsistent with those accumulated up to vi . For two edges with inconsistent conditions, split the graph and place these edges in separate sub-graphs.

The above algorithms differ from the original INDEEL. Here a relationship between Tj and time-keeping variable fj (for tmj ) is exploited to prevent unnecessary growth of G00 . The value of fj is relevant only if Tj is 1 (i.e., fj is active). The edge conditions (Section III-C) in the model are derived so that inactive fj could not create any inconsistencies. Each component involving inactive fj would evaluate to true all the time, and thus can be pruned from a condition. The rule for identifying an inconsistency in INDEEL is modified as follows: “when variable x is used in actions of edges incoming to vp , and either (1) x is not used in conditions of edges reachable from vp or (2) x is used in these conditions, but it is inactive in AU M (vp ), vertex vp and the following graph are not split.” B. Iterative inconsistency removal After G0 is derived, the INDEEL are applied iteratively: Step A (Graph extension): Extend the original graph G0 with vertex vt , edges of eon and eoff , and a number of observer edges as described in Section III (see Fig. 6 for an example). Mark and queue vertex v0 as v0.0 . Step B (Inconsistency removal): Unqueue vertex v0.k , i.e., a copy of v0 . Apply the INDEEL starting from v0.k until v0.k is reached again through a set of edges denoted by E0.k (the set of incoming edges of v0.k ). Step C (Initial-state splitting): Split vertex v0.k into a set of vertices V0.k ∪ {v0.k }; V0.k ’s cardinality is equal to the number of distinct AUMs associated with edges in E0.k (note: inc v0.k may belong to V0.k ). The set of V0.k is divided into V0.k (vertices associated with AUMs corresponding to all timers act (the remaining vertices in V0.k ). The set of inactive) and V0.k inc act edges E0.k is divided accordingly into E0.k and E0.k . Edge eon , whose traversal is mandatory in the test sequence, is incoming only to vertex v0 ; an edge eoff is outgoing from each inc vertex in V0.k . All copies of eoff are optional to traverse—they will be included in the test sequence only when necessary. Step D (Redundant-paths pruning): Remove from G0 inc edges in E0.k using a heuristic that prunes an edge unless it

9

enables reaching some untested edges in a cost-effective way, as described below in more detail. Step E (Initial-state queueing): Queue unmarked vertices act inc in V0.k (all) and in V0.k (only those with at least one undeleted inc edge in E0.k ). Mark queued vertices. If the queue is empty, terminate; else, go to Step B. The following two-phase heuristic is applied in Step D. During the graph traversal, we can associate a boolean array ˜0.k with vertex v0.k . A value of true in i-th position indicates E that ei has been traversed before unqueueing v0.k . When new paths are being created, the algorithm builds a similar array ˜i for each AU Mi . E inc First, delete any ei ∈ E0.k if neither of the following conditions is true: • Step D.1: A new edge can be traversed by keeping ei inc in the graph, i.e., the paths from v0.k to vertices in V0.k associated with AU Mi should contain an edge that has not been traversed before unqueuing v0.k . (To determine ˜i is compared whether or not the above condition is true, E ˜ against E0.k in O(|E|) time.) • Step D.2: Presence of ei enables traversing some of the s (i) s (0.k) untested self-loops, i.e., (∃p≤|V | ∃l≤Mp )tp,l < tp,l . inc inc such that: if ∃ ej ∈ E0.k Second, delete any ei ∈ E0.k • Step D.3: AU Mj includes AU Mi . Since all timers are inc , a sufficient condition for AU Mj to inactive in V0.k s (j) s (i) include AU Mi is: (∀p≤|V | ∀l≤Mp )tp,l ≤ tp,l , i.e., AU Mj allows testing more self-loops than AU Mi . • Step D.4: All edges in the paths from v0.k to vertices in inc V0.k associated with AU Mi have their copies in the paths inc from v0.k to vertices in V0.k associated with AU Mj . (A single comparison of two paths in the above condition can be done in O(|E|) time.) C. Complexity analysis The INDEEL for actions consist of an MBF traversal and constructing the new AUM pairs for each combination. The complexity for the MBF traversal is O(E 2 ). For each node vi , the number of AUM pairs is P|V |−1 vj →vi |E | × |AUM(vj , J)| (where |E vj →vi | is the 1 number of edges from vj to vi ) such that ∃ ek = (vj , vi ). The complexity of the INDEEL for conditions is bounded by the number of AUM pairs of each node and executing the linear programming for each edge. For m variables and S constraints, linear programming takes min(m2 , S 2 ) steps [8]. In the INDEEL, each graph split affects only a portion of (rather than the entire) EFSM graph. Therefore, the growth of the EFSM graph is bounded by the size of the subgraphs involved in a specific inconsistency. For general software, the inconsistency-removal problem is equivalent to the halting problem [25]; therefore, it is undecidable for the general case, yielding an exponential growth of the final conflict-free graph size. The minimization step of TA is also equivalent to the halting problem for timers with infinite domains; it is PSPACE-Complete for timers with finite domains [2], [7], [25]. However, the INDEEL take advantage of localized inconsistencies by splitting only the portions of the graph. Furthermore, in the augmented algorithms, G0 will only

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 11, NO. 5, OCTOBER 2003

be split because of the inconsistencies among variables for timers that are active in specific subgraphs. In other words, any condition or action using timer variables for inactive timers will not cause an inconsistency, and thus will not contribute to the growth of the graph. This methodology has been applied to two real-life protocols, resulting in conflict-free graphs that did not grow exponentially (see Section VI-E). Let α be the complexity of the INDEEL. Let us now assess the complexity of one iteration of the iterative inconsistencyremoval procedure presented in Section VI-B. • Step A: the complexity is proportional to the number of P observer edges that need to be added: O( p≤|V | Mp ); • Step B: the complexity is equal to α; • Step C: the complexity is upper bounded by the number of distinct AUMs associated with edges coming to a copy of the initial vertex v0.k , i.e., |V0.k |; • Step D: D.1 can be executed inPO(|E|); D.2 in P inc 2 O( p≤|V | Mp ); D.3 in O(|V0.k | ∗ p≤|V | Mp ); and inc 2 D.4 in O(|V0.k | ∗ |E|); • Step E: the complexity is at most |V0.k |. P Overall, one iteration runs in O( p≤|V | Mp + α + |V0.k | + inc 2 inc |E|P ∗ |V0.k | ) time. With the cardinality of V0.k at most s O( p≤|V |, l≤|Mp |Np,l |), the running time of one loop iteration is as follows: X X s O(α + |V0.k | + (|E| + Mp ) ∗ ( |Np,l |))2 p≤|V |

p≤|V |, l≤|Mp

Example 2 (Test generation): Let us now apply the above algorithm to the FSM of Fig. 5. First, the FSM is augmented with the auxiliary edges of eon and eoff , and a number of observer edges as shown in Fig. 6. The conditions and actions of the observer edges are defined based on (8) as follows: s0i,1 s000,1 s002,1 sˇ0

: : : :

hLi < 2 ∧ tsi,1 == 0i {}, where i = 1, 2, 3 s hL0 < 2 ∧ (t0,1 > 0 ∧ (T1 ∨ ¬T2 ))i {} hL2 < 2 ∧ (ts2,1 > 0 ∧ (¬T1 ∨ ¬T2 ))i {} h1i {L0 = 2}, sˇ1 : h1i {L1 = 2}, sˇ2 : h1i {L2 = 2} e1,1

s'1,1

vt

w1,1

eoff eon

w0,1

s'0,1 s''0,1 š0

š1

e2

v0

e0,1 ed e18,2

w2,1 s'

2,1

v1

v1’

e4

v0’ e2,1 e12,1 e22,1 e27,1 e27,2

e18,1

s''2,1

v2’

v2

š2 Fig. 6.

ed

ed

Augmented graph for the FSM of Fig. 5.

Typically, a test sequence is divided into a number of subtours—subsequences of a full test sequence that start and stop in v0 . Each subtour may or may not be preceded by a system power-down/power-up; therefore, when an IUT starts

10

executing, not only should it be brought to state v0 , in addition, all timers must be inactive. To ensure this behavior, each v0 ’s copy corresponding to an AUM with all timers inactive (i.e., inc any vertex in V0.k ) may be considered the start state of a subtour.  1: eon , e2 , e3 , e4 , e5 , e8 , e1 , e2 , e4 , e6 , e7 , e8 , eoff  2: eon , e2 , e4 , e5 , e6 , e7 , e8 (17)  3: e2 , e3 , e4 , e8 , e1 , e2 , e4 , e7 , e8 , eoff e3

v1

v1

e2 v0

e4

eoff

e8 e1 v0 v1

e4

v0

eon

v0 v1

v0

v0

v1 v2 e7

e7 v2

e8

e4

e3

e5+e6 e8

v2

v0

e2

v2

e2

v2

Fig. 7.

eoff e1

e2

v2

e4

vt

eoff

e8

e4 v2 e5 v2

v2

v1

e8

e6 v2 e7 v2

The final graph for the FSM of Fig. 5.

An application of the algorithm described in this section to the graph G0 of Fig. 6 produces the final conflict-free graph G00 shown in Fig. 7. All edges defined in G0 are included in G00 without the explicit delaying of tm1 and tm2 ; therefore, the technique presented in Section IV-C need not be applied in this case. A minimum-cost test sequence, given by (17), can be derived as a solution to the Rural Chinese Postman Problem [1] on this final graph. The test sequence of (17) consists of three subtours containing the edges defined in the original graph (Fig. 5) and the auxiliary edges of eon and eoff ; the observer edges are dropped. Note that the test sequence of Table I, which was derived manually, corresponds to Subtour 1. D. Fault detection In our analysis, several well known assumptions [5], [15], [19], [25] on the specification and the IUT are valid: (1) the specification is strongly-connected, reduced, and deterministic; (2) the IUT has the same input alphabet as the specification; and (3) the faults do not increase the number of states in the IUT. The detection of transfer/output faults [31], [40] depends on the state verification method [1], [6], [27], [32], [36], and is not part of the timing-fault analysis. If a timing fault results in a transfer/output fault, we assume that it is detected with high probability. This paper utilizes the classification of timing faults introduced in [15], [23]. We prove that under the above assumptions, all single 1-clock and n-clock timing faults [15] are detected when applicable. We also prove that certain faults due to incorrect settings for timer lengths are covered. Fault coverage for multiple simultaneous timing faults is an open problem regardless of the testing model.

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 11, NO. 5, OCTOBER 2003

For transition ei : (ai /oi ), after applying ai during testing, output oi is expected no later than θ. This behavior is controlled by a special-purpose timer that is part of a test harness (not the IUT), and whose length θ is 0 < θ < ci +  for a nontimeout ei , and max(0, Dj −fj ) < θ < ci +max(0, Dj −fj )+ for a timeout eij . 1-clock, interval fault:: •

• •

Timing requirement: For transitions ei = (vp , vq ; ai /oi ) and hk , transition ei can trigger only after applying ai within time boundaries [b1 , b2 ] measured from the execution of hk . Timing fault I: ai is applied at bai 6∈ [b1 , b2 ]; oi is observed and vq verified in less than bai + θ. Timing fault II: ai is applied at bai ∈ [b1 , b2 ]; oi is not observed or vq not verified in less than bai + θ.

The 1-clock, interval timing requirement can be modeled as shown in Fig. 8. First, tm1 (with D1 = b1 ) and tm2 (with D2 = b2 ) are started in hk : h1i{T1 = 1; f1 = 0; T2 = 1; f2 = 0}. As a result of the timing requirement, ei triggers after tm1 and before tm2 expire, and in its actions stops tm2 with output oi , i.e., ei : h¬T1 ∧ T2 i{T2 = 0}.

vk

1

hk

ck

b1

vk

2

{T1=1;T2=1; f1=0; f2=0}

b2

Fig. 8.

q oi observed; fault I

Timing requirement: For transitions ei = (vp , vq ; ai /oi ) and h1 , . . . , hn , transition ei can trigger after applying input ai only when, for any k < n, hk was executed before hk+1 . • Timing fault III: ai is applied, oi is observed and vq verified in less than bai + θ time when, for at least one k : 2 ≤ k ≤ n, hk is executed before hk−1 . For the n-clock timing requirement, timers tm1 , . . . , tmn with the infinite lengths are introduced (Fig. 9). Transition h1 starts tm1 , i.e., h1 : h1i{T1 = 1; f1 = 0}. Each hk (2 ≤ k ≤ n) is split into h1k and h2k : the former triggers before tmk−1 expires, and starts tmk , i.e., hk : hTk−1 i{Tk = 1; fk = 0}; the latter triggers when tmk−1 is not running, and in its actions stops tmk , i.e., hk : h¬Tk−1 i{Tk = 0}. Finally, ei triggers only when tmn is running. •

vk

1

v 11 h1

v 12

q ai

11

ai

no output; fault II

Modeling 1-clock interval timing fault.

Consider state and transition spaces (UG , RG ) (defined in Section IV-B) with each ui ∈ UG represented by (vi , T1 , f1 , T2 , f2 ). Transition ei is represented in RG by gi = (up , uq ), where up = (vp , 0, −∞, 1, f2 ∈ [b1 , b2 ]) and uq = (vq , 0, −∞, 0, −∞). The time condition of ei and the definition of up indicate that gi is included (and can trigger) in a conflict-free graph only at a point in time f2 within the boundaries of [b1 , b2 ]. Transition ei will not be included in a test sequence as originating from any up with f2 6∈ [b1 , b2 ]. Timing fault I is detected in three steps: (1) verifying state vp , (2) observing oi , and (3) verifying state vq . These steps correspond to the execution of infeasible transition gi , which is present in neither a conflict-free graph nor a valid test sequence. Timing fault II is detected in two steps: (1) verifying state vp , and (2) observing output ok 6= oi or verifying state vk 6= vq . These steps are not expected as a result of executing transition gi , which is included in a conflict-free graph and a valid test sequence. Therefore, all single 1-clock interval faults are detected. The above analysis can be easily extended for two interval faults such as time-constraint restriction and time-constraint widening faults, which occur when the IUT changes either the upper or lower bound of a time constraint [14]. n-clock fault::

hk

vk

vk1 hk1 hk2

vk2

vk

1

vk-12

hk1 hk2

v n1

vk-11 hk-11

1

hk-1

2

hn1

v n2

vk2

vp hn2

ei

vq

Fig. 9. Modeling n-clock timing fault. Transitions executed when the fault occurs appear in bold.

The INDEEL allow only feasible test sequences to be generated. Consider two such sequences: 1 2 1 2 2 • (h1 , . . ., hk , hk−1 , hk+1 , . . ., hn , hn+1 6= ei ), with the outputs of (oh1 , . . ., ohk , ohk−1 , ohk+1 , . . ., ohn , ohn+1 ), where ohn+1 6= oi or vq 6= tail(hn+1 ). Since hk precedes hk−1 , h1k ’s condition of hTk−1 i cannot be satisfied because only h1k−1 can start tmk−1 . Transition ei : hTn i is infeasible, since h2n does not start tmn ; 1 1 1 1 • (h1 , . . ., hk−1 , hk , . . ., hn , ei ), with the outputs of (oh1 , . . ., ohk−1 , ohk , . . ., ohn , oi ). The outputs observed when the timing fault III occurs are as follows: (oh1 , . . ., ohk , ohk−1 , ohk+1 , . . ., ohn , oi ). The above multi-clock fault is detected by the first valid test sequence, either by observing oi instead of ohn+1 , when ohn+1 6= oi , or by verifying state vq 6= tail(hn+1 ) when ohn+1 = oi . Therefore, all single n-clock faults are detected. Incorrect timer-setting fault:: • Timing requirement: For timer tmj of length Dj , timeout transition eji = (vp , vq , −/oji ) will trigger exactly in Dj time units after tmj is started in transition hk . j • Timing fault IV: After hk triggers, oi is observed and vq verified in less than Dj time. j • Timing fault V: After hk triggers, oi is observed and vq verified in more than Dj + ci time. There are several ways in which this type of timing faults can be detected. The first way is to take advantage of the

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 11, NO. 5, OCTOBER 2003

special-purpose timer with the length of θ. Suppose that transition eji triggers after Dj0 < Dj (timing fault IV). If Dj 0 and the fault is not detected through the special-purpose timer. (It may, however, be detected by observing incorrect outputs.) If Dj > fj , then θ > Dj − fj , and the fault is detected by observing oji in less than θ time. Suppose that transition eji triggers after Dj00 > Dj (timing fault V). If Dj θ by observing oji in more than θ time. When the fault cannot be detected by using the specialpurpose timer, in many cases it may be detected by observing expected outputs from other transitions affected by the fault. Suppose that, for the timing fault IV, the specification allows the following test sequence: (. . ., hk , . . ., hn , . . ., eji , . . .). Consider two cases of timing fault IV: (1) tmj expires in eji before the implementation is able to execute hn , and (2) the implementation executes the above test sequence in order. In the first case, eji triggers instead of hn . This error is detected by observing oji 6= ohn or verifying vq 6= tail(hn ). In the second case, whether or not the fault is detected depends on the state of the running timers at the time of error occurrence. If there are no running timers after tmj expires prematurely, timing fault IV is not detected other than by the specialpurpose timer. Da

Da

2.1

2.2 ck

D’j

ci Db

ck

D’j

ci Db

Fig. 10. Incorrect timer-setting fault IV: not detected in the case 2.1; detected in the case 2.2.

Figure 10 shows a different case where there are running timers when the fault occurs. Timer tma with output oa is started by hk (which also starts tmj ), and timer tmb with output ob 6= oa is started by eji . The outputs in a test sequence for this case are as follows: (. . . , ok , . . . , on , . . . , oji , . . . , oa , ob , . . .). When timing fault IV occurs and Dj0 > Da −ci −Db (Case 2.1), the order of outputs in the test sequence is preserved by the implementation, and the discrepancy between Dj0 and Dj is not large enough to be detected in this way. If Dj0 < Da − ci − Db (Case 2.2), the difference Dj − Dj0 is large enough to cause ob appear before oa , which is detected by the above test sequence. The analysis for the timing fault V is analogous. Therefore, many single incorrect timer setting faults are detected. Example 3 (Fault detection): Let us consider a system where 3 timeouts are required to occur in a specific order: timeouts for tm1 followed by tm2 and tm3 . A violation of this requirement results in a 3-clock timing fault. Our method can detect this 3-clock fault, which can occur due to several faulty timers as follows: (1) tm3 expires then tm1 then tm2 , (2) tm2 expires then tm1 then tm3 , or (3) tm2 expires then tm3 then tm1 , etc. In this example, the timer lengths are correct, but they are started incorrectly (too early or too late). Otherwise, the errors correspond to incorrect timer-setting

12

faults. As proven in this section, any single n-clock faults are detected by our method. We do not guarantee that all multiple n-clock or multiple incorrect timer-setting faults (or their combination) can be detected. Let us now consider the above error case tm3 expires followed by tm1 and then tm2 . Suppose also that there are incorrect timer settings: tm1 and tm2 are set to much shorter lengths than specified. In this case, the timers will all expire incorrectly (i.e., too early), but in the correct order. Such multiple faults cannot be detected in our model unless special external timers are present in the test harness to monitor timers’ expiration times. E. Application to Real-life Protocols An INDEEL software package has been implemented at CCNY to perform inconsistency removal [39]. This tool, after augmentation for timed-test generation, enables the application of the presented modeling and test-generation methodology to a range of communications protocols. The methodology was used to generate tests for MILSTD 188-220, a protocol suite for mobile combat network radios [37]. The formal specification for the Network layer has 7,150 lines of code, defining 34 states and 370 transitions in 7 EFSMs [17]. The sizes of the resulting FSMs derived from these EFSMs range from 48 to 303 states, and from 119 to 925 transitions. The corresponding test sequences range from 145 to 2,803 test steps. The total number of tests delivered to US Army Communications-Electronics Command (CECOM) is approximately 10,000 test steps [17]. The protocol’s Datalink Layer defines several timers that can run concurrently and affect behavior of the protocol. For example, if either BUSY or ACK timer is running, a buffered frame cannot be transmitted. If ACK timer expires while BUSY timer is not running, a buffered frame is retransmitted. If, however, ACK timer expires while BUSY timer is running, no output is generated. To enable efficient test generation for the timed behavior of 188-220, the techniques presented here were applied to several components of the protocol. Ref. [18] presents a detailed derivation of such tests for the two BUSY timers. The INDEEL were also successfully applied to the VHDL specification of the Local Proxy component of the Adaptive Computing Architecture (ACA) [11], a prototype of a militaryoriented network architecture [13]. The proxy was modeled as an EFSM with 18 states and 51 transitions. There were 6 variables causing inconsistencies. After applying the algorithms, the final conflict-free EFSM consisted of 48 states and 65 transitions. Based on these results, design recommendations were submitted to the proxy designers to enhance the specification’s testability, resulting in a more rigorously tested product. VII. C ONCLUSION This paper presents a study of generating test sequences when multiple timers are running simultaneously. The INDEEL [12], [38] are applied to a new model for testing protocols with multiple timers. The model is specifically designed for testing purposes to avoid performing full reachability analysis

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 11, NO. 5, OCTOBER 2003

and to significantly limit the explosive growth of the number of test scenarios. It is based exclusively on the paradigm of EFSM, and thus can be applied to the languages such as SDL, VHDL, and Estelle. The technique models a realistic testing framework in which each I/O exchange takes a certain time to realize and timers can be started or stopped in arbitrary transitions. It also gives the ability to test timed systems with well studied EFSM- and FSM-based test generation methodologies. 1 R EFERENCES [1] A.V. Aho, A.T. Dahbura, D. Lee, and M.U. Uyar. An optimization technique for protocol conformance test generation based on UIO sequences and rural Chinese postman tours. IEEE Trans. Commun., 39(11):1604–1615, 1991. [2] R. Alur, C. Courcoubetis, N. Halbwachs, D.L. Dill, and H. Wong-Toi. Minimization of timed transition systems. In Proc. CONCUR: Int’l Conf. Concur. Theory, vol. 360 of [Springer] LNCS, pp. 340–354, Stony Brook, NY, 1992. [3] R. Alur and D.L. Dill. A theory of timed automata. [Elsevier] Theoret. Comput. Sci., 126:183–235, 1994. [4] A.F. Ates and B. Sarikaya. Test sequence generation and timed testing. [Elsevier] Comput. Networks ISDN Syst., 29:107–131, 1996. [5] G.v. Bochmann, A. Das, R. Dssouli, M. Dubuc, A. Ghedamsi, and G. Luo. Fault models in testing. In Proc. IFIP IWPTS: Int’l Wksp Protocol Test Syst., pp. 17–30, 1992. [6] W.Y.L. Chan and S.T. Vuong. The UIOv—method for protocol test sequence generation. In Proc. IFIP IWPTS: Int’l Wksp Protocol Test Syst., Berlin, Germany, 1989. [7] K.-T. Cheng and A.S. Krishnakumar. Automatic generation of functional vectors using the extended finite state machine model. ACM Trans. Design Automat. Electr. Syst., 1(1):57–79, 1996. [8] V. Chv´atal. Linear Programming. A Series of Books in the Mathematical Sciences. W.H. Freeman & Co., San Francisco, CA, 1983. [9] J.P. Courtiat, C. Saibel, C. Lohr, and B. Outtaj. Experience with RTLOTOS, a temporal extension of the LOTOS FDT. In Lai [24], pp. 1104–1123. [10] D. Coward and D. Ince. Symbolic Execution of Software. Chapman & Hall, 1995. [11] S. Crawley, J. Indulska, and B. McClure. ODP-based adaptive management of network resources in heterogeneous defense networks. In Proc. IFIP/IEEE DSOM: Int’l Wksp Distrib. Syst. Operat. Manage., pp. 125–138, Newark, DE, 1998. [12] A.Y. Duale and M.U. Uyar. Generation of feasible test sequences for EFSM models. In TestCom’00 [34], pp. 91–109. [13] A.Y. Duale, M.U. Uyar, B. McClure, and S. Chamberlain. Conformance testing: Towards refining VHDL specifications. In Proc. IEEE MILCOM: Military Commun. Conf., pp. 140–144, Atlantic City, NJ, 1999. [14] A. En-Nouaary, R. Dssouli, and F. Khendek. Timed Wp-method: Testing real-time systems. IEEE Trans. Software Eng., 28(11):1023–1038, 2002. [15] A. En-Nouaary, R. Dssouli, F. Khendek, and A. Elqortobi. Timed test cases generation based on state characterisation technique. In Proc. IEEE RTSS: Real-Time Syst. Symp., pp. 220–229, Madrid, Spain, 1998. [16] M.A. Fecko, P.D. Amer, M.U. Uyar, and A.Y. Duale. Test generation in the presence of conflicting timers. In TestCom’00 [34], pp. 301–320. [17] M.A. Fecko, M.U. Uyar, P.D. Amer, A.S. Sethi, T.J. Dzik, R. Menell, and M. McMahon. A success story of formal description techniques: Estelle specification and test generation for MIL-STD 188-220. In Lai [24], pp. 1196–1213. [18] M.A. Fecko, M.U. Uyar, A.Y. Duale, and P.D. Amer. Efficient test generation for Army network protocols with conflicting timers. In Proc. IEEE MILCOM: Military Commun. Conf., pp. 133–138, Los Angeles, CA, 2000. [19] S. Fujiwara, G.v. Bochmann, F. Khendek, M. Amalou, and A. Ghedamsi. Test selection based on finite state models. IEEE Trans. Software Eng., 17(6):591–603, 1991. [20] T. Higashino, A. Nakata, K. Taniguchi, and A.R. Cavalli. Generating test cases for a timed I/O automaton model. In Proc. IFIP IWTCS: Int’l Wksp Test. Communicat. Syst., pp. 197–214, Budapest, Hungary, 1999. 1 The views and conclusions contained in this document are those of the authors and do not represent the official policies, either expressed or implied, of the Army Research Lab or the U.S. Government.

13

[21] D. Hogrefe, B. Koch, and H. Neukirchen. Some implications of MSC, SDL and TTCN time extensions for computer-aided test generation. In Proc. SDL-Forum Symp., vol. 2078 of [Springer] LNCS, Copenhagen, Denmark, 2001. [22] A. Khoumsi, M. Akalay, R. Dssouli, A. En-Nouaary, and L. Granger. An approach for testing real time protocol entities. In TestCom’00 [34], pp. 281–299. [23] A. Khoumsi, A. En-Nouaary, R. Dssouli, and M. Akalay. A new method for testing real-time systems. In Proc. IEEE RTCSA: Int’l Conf. RealTime Comput. Syst. Appl., pp. 441–450, Cheju Island, S. Korea, 2000. [24] R. Lai, ed. FDTs in Practice, vol. 23(12) S.I. of [Elsevier] Comput. Commun. 2000. [25] D. Lee and M. Yannakakis. Principles and methods of testing finite state machines—a survey. Proc. IEEE, 84(8):1090–1123, 1996. [26] R.J. Linn and M.U. Uyar. Conformance Testing Methodologies and Architectures for OSI Protocols. IEEE Comput. Soc. Press, Los Alamitos, CA, 1994. [27] G. Luo, G.v. Bochmann, and A.F. Petrenko. Test selection based on communicating nondeterministic finite state machines using a generalized Wp-method. IEEE Trans. Software Eng., 20(2):149–162, 1994. [28] D. Mandrioli, S. Morasca, and A. Morzenti. Generating test cases for real-time systems from logic specifications. ACM Trans. Comput. Syst., 13:365–398, 1995. [29] E. Petitjean and H. Fouchal. A realistic architecture for timed testing. In Proc. IEEE ICECCS: Int’l Conf. Eng. Complex Comput. Syst., pp. 109–118, Las Vegas, NV, 1999. [30] A. Petrenko, S. Boroday, and R. Groz. Confirming configurations in EFSM. In Proc. IFIP Joint Int’l Conf. FORTE/PSTV, Beijing, China, 1999. [31] A.F. Petrenko and G.v. Bochmann. On fault coverage of tests for finite state specifications. [Elsevier] Comput. Networks ISDN Syst., 29(1):81– 106, 1996. [32] A. Rezaki and H. Ural. Construction of checking sequences based on characterization sets. [Elsevier] Comput. Commun., 18(12):911–920, 1995. [33] J. Springintveld, F. Vaandrager, and P.R. D’Argenio. Testing timed automata. [Elsevier] Theoret. Comput. Sci., 254(1-2):225–257, 2001. [34] H. Ural, R.L. Probert, and G.v. Bochmann, eds. Proc. IFIP TestCom: Int’l Conf. Test. Communicat. Syst., Ottawa, ON, 2000. [35] H. Ural, K. Saleh, and A. Williams. Test generation based on control and data dependencies within system specifications in SDL. [Elsevier] Comput. Commun., 23(7):609–627, 2000. [36] H. Ural and K. Zhu. Optimal length test sequence generation using distinguishing sequences. IEEE/ACM Trans. Network., 1(3):358–371, 1993. [37] US Defense Dept. Military Standard—Interoperability Standard for Digital Message Device Subsystems (MIL-STD 188-220B), 1998. [38] M.U. Uyar and A.Y. Duale. Resolving inconsistencies in EFSM-modeled specifications. In Proc. IEEE MILCOM: Military Commun. Conf., pp. 135–139, Atlantic City, NJ, 1999. [39] M.U. Uyar and A.Y. Duale. INDEEL software package: Automatic test generation for EFSM models of communication protocols. In Proc. IIIS SCI: World Multi-Conf. System. Cybern. Inf., vol. XV, pp. 355–361, Orlando, FL, 2002. [40] J. Zhu and S.T. Chanson. Toward evaluating fault coverage of protocol test sequences. In Proc. IFIP PSTV: Protocol Specif. Test. Verif., Vancouver, BC, 1994.

Mariusz A. Fecko (M ’00) received MS degrees in EE and CS from Stanislaw Staszic University (AGH), Poland, and MS and PhD in CIS from the University of Delaware. Continuing at Delaware as a PLACE Post-Doc Fellow, he jointly developed a formal specPHOTO ification and conformance-testing methodologies for HERE radio-network protocols of the US Army CECOM. In 2000, Dr. Fecko joined Telcordia Technologies, New Jersey, as a research scientist in Applied Research Area. He works on process and test automation for telecom software systems and services, and serves as Principal Investigator in the ARL-funded research on survivable wireless networking. He received Telcordia CEO Team Award (2001) and Executive Director appreciation award (2002).

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 11, NO. 5, OCTOBER 2003

¨ M. Umit Uyar (SM ’91) has a BS degree from ¨ Istanbul Teknik Universitesi, and MS and PhD degrees from Cornell University, Ithaca, NY, all in EE. Dr. Uyar is currently with the EE Dept., the City PLACE College of the City University of New York. He was PHOTO a Distinguished Member of Technical Staff at AT&T HERE Bell Labs until 1993. In Bell Labs, he received a Vice Presidential Quality Award for co-designing software tools, three AT&T Bell Labs Vice Presidential Research Appreciation Awards, and a Best Paper Award in AT&T Electronic Testing Conference. He was granted “Doc¸ent” title by the National University Council of Turkey in 1992. He co-chaired the 12th Int. Symp. on Protocol Specification, Testing and Verification (1992), and the 6th Int. Conf. on Formal Description Techniques (1993). He co-edited “Conformance testing methodologies and architectures for OSI protocols,” from IEEE Comp. Soc. Press, and holds two U.S. patents.

PLACE PHOTO HERE

Ali Y. Duale received BE, ME, and PhD degrees from the City University of New York, all in EE, in 1995, 1997, and 2000, respectively. Dr. Duale joined the IBM S/390 and zSeries architecture verification group in 2000 as an Advisory Software Engineer. His recent work includes the development of techniques to generate terminating random test streams, the design of algorithms to test memory coherence for multiprocessor systems, and the verification of IEEE floating point implementation.

Paul D. Amer (ACM ’76/ A’ 93) received the BS degree summa cum laude in Math from SUNY Albany in 1974, and the MS and PhD degrees in CIS in 1976 and 1979 from the Ohio State University. Since PLACE 1979, he has been at the University of Delaware PHOTO where currently he is professor of computer science. HERE From 1978 to 1987, he was concurrently employed part-time as a Research Scientist for the National Bureau of Standards. In 1985, Prof. Amer spent a sabbatical in Paris teaching at ENST, and researching Estelle within ESPRIT Project SEDOS. In 1992, he spent his second sabbatical in Toulouse at LAAS-CNRS investigating a partial order transport protocol to support multimedia applications (RFC1693). In 1999, Prof. Amer returned to Toulouse sharing time between LAAS and ENSICA to investigate data compression in multimedia, and innovative transport layer services and protocols.

14