Generating test inputs for embedded control systems - CiteSeerX

Generating Test Inputs for Embedded Control Systems Existing simulation models, with carefully designed elements of a genetic algorithm, automatically create test inputs, eliminating the task of manual creation.

©MASTERSERIES

A

By Qianchuan Zhao, Bruce H. Krogh, and Paul Hubbard August 2003

n essential aspect of a high-quality design process is the development of test patterns, or sets of test inputs, that can be applied to the final product or intermediate instantiations to identify faults and confirm correct system behavior. In the design of embedded control systems, in particular, the growing use of tools for computer-aided design and simulation increases the prospects for performing extensive testing of the control logic before it is realized in software and implemented on the target processor. The rapid growth of the complexity of embedded control systems and the demand for short design cycles have increased the interest in effective methods for automatic test generation. This article presents a new method for leveraging existing simulation models for embedded control system designs to generate test inputs automatically, thereby eliminating the time-consuming task of creating them manually. Of the two principal ways testing is used to validate system designs, the most common is postproduction to confirm the functionality of the final implementation and check for fabrication flaws. The results of the tests are compared with those

0272-1708/03/$17.00©2003IEEE IEEE Control Systems Magazine

Authorized licensed use limited to: Tsinghua University Library. Downloaded on May 19, 2009 at 23:09 from IEEE Xplore. Restrictions apply.

49

of a so-called golden model to identify defects. To ensure that all possible defects are exposed, it is important that these tests drive the system through the complete range of operation where defects may impact behavior. This feature of a set of test inputs is referred to as coverage. The second common use of testing occurs earlier in the development process when new instantiations of the system model are generated, e.g., by automatic code generation or through incremental changes to the design. During development, testing is used in an exploratory manner to identify overlooked implications of new designs or modifications. Rather than a golden model, an implementation is

evaluation of embedded control systems, computer models are available in many cases. These models are not amenable to formal analysis, however, making it impossible to use them directly in recently proposed methods for test input generation using counterexamples generated by model checkers [10]-[12]. In contrast to integrated circuits for which very structured models using standard languages are developed during the design process, it is extremely difficult, if not impossible, to extract formal models (such as automata) from existing simulation models of embedded control systems. We propose a method for test input generation that addresses both of these challenges. We formulate test generation as an optimization problem that can be solved using a genetic algorithm (GA) with effective heuristics that meet the challenge of computational complexity for hybrid systems. Moreover, our approach uses existing simulation models directly, with very little modification, to evaluate the objective functions (also known as fitness functions) for given input signals in the GA implementation. There is no need to construct new formal models of the embedded controller and the plant to apply our technique. The use of GAs has been suggested in the context of discrete systems for the generation of test vectors and test input sequences [13], [14], but not in the context of test input generation for mixed-signal systems. Testing of control systems with GAs has also been studied in [15], but for a different purpose. In [15], the objective is to evaluate the performance of controllers in the presence of faults whereas our purpose is to find system inputs that drive the system (without faults) through specified behavioral patterns. Our “golden model” is a given executable specification. Our contributions are 1) the formulation of a coverage problem for hybrid systems with discrete and continuous inputs and outputs, 2) a pragmatic solution to the problem that uses a GA-based algorithm to produce a test input with the specified coverage, and 3) implementation and demonstration of the method for industrial-sized MATLAB Simulink/Stateflow simulation models. The following section presents a formulation of the general test input generation problem addressed in this article and describes the construction of a fitness function to evaluate the extent to which a given input sequence meets the coverage criteria. We then present the GA approach to searching for test inputs. We have implemented the procedure in MATLAB to generate test inputs based on simulation models of embedded control systems in MATLAB SimulinkStateflow [16]. We illustrate the method for the SimulinkStateflow demonstration model of an automatic transmission and also present computational results for large indus-

We formulate test generation as an optimization problem that can be solved using a genetic algorithm with effective heuristics that meet the challenge of computational complexity for hybrid systems. usually compared with an executable specification, which is a preliminary, possibly more abstract, design model. Again, coverage is important to provide high assurance that the system behavior satisfies the design specifications. Automatic test generation for embedded control systems is challenging for two major reasons. First, the problem is complex due to the hybrid nature of the controller and plant composition, which contains both logic (discrete states) and components with continuous dynamics (continuous states). Automatic test vector generation is well established for purely discrete-state systems, particularly for digital integrated circuits [1]. For example, procedures exist to generate test vectors that provide complete coverage to detect standard faults such as stuck-on conditions in digital circuits [2]. Analogous methods for mixed-signal integrated circuits and systems have remained elusive and continue to be an open and active field of research [1], [3]-[6]. Indeed, undecidability results for hybrid systems indicate that we should not expect to discover completely algorithmic procedures for test-vector generation for hybrid systems [7]. Hence, global search methods that use domain-specific heuristics are used to produce test inputs for mixed-signal circuits [1], [8], [9]. Similar methods are needed to generate test vectors for embedded control systems. The second challenge to developing automatic test input generation tools for embedded control systems is practicality. To develop automatic methods, the design must first be captured in a computer model that includes a representation of the plant as well as an embedded control algorithm. With the growing use of simulation models for design and

50

IEEE Control Systems Magazine


August 2003

trial systems. The concluding section summarizes the contributions of this work and discusses directions for further research.

Problem Formulation

Figure 1. Checking coverage for a given input. Figure 1 illustrates the elements of the general problem of test input generation to satisfy a given cov- done by adding new transition labels as necessary to sufficiently identify the path through the internal discrete state erage criterion. We discuss each aspect of the figure in turn. space and including these labels in the output alphabet. In many cases, this instrumentation can be accomplished auInput Space We consider the input space U = {(u ,v )}, where u is a vector tomatically, for instance with the Performance Tools [18] provided in MATLAB/Simulink, which provide an instruof real-valued, bounded, and piece-wise constant inputs, i.e., mented simulation of Simulink models and a data object with information such as decision coverage (for Switch u: ℜ → ℜn : u( t ) = U k , t k ≤ t < t k + 1, blocks and Stateflow states) or condition coverage (for where U k ∈ [u 1l ,u 1u]× L× [u pl ,u pu], and u •l ,u •u are lower and up- Logic blocks, and Stateflow transitions). per bounds of input variables, respectively. v is a discrete-valued input that remains constant over the same Measure of Coverage f is a bounded map Y → I that provides an integer-valued time intervals as u, i.e., coverage measure for each output ( y , z ). The desired coverage is defined by a set of regions R = { Rk , k = 1,K , N }, where v: ℜ → Σ I , v( t ) = σ k ∈ Σ I , t k ≤ t < t k + 1 , each region Rk is the union of a finite set of intervals in where Σ I is a finite set of values for v. Equivalently, the infor- ℜ m × Σ O , the range of ( y , z ). In particular, f is defined as mation in v can be captured in the set of pairs

{

}

v~ = ( t j , σ j ) j = 0 ,1,2,K , r ; t j + 1 > t j , σ j ∈ Σ I .

f ( y , z ) = − ∑ d(( y , z ), Rk ), k

The symbols in Σ I can each be finite-dimensional vectors if the system has multiple discrete-valued inputs.

Model Execution (P) A single execution (or simulation) of the model P with the input (u ,v ) produces the output ( y , z ), which we write as ( y , z ) = P(u ,v ).

Output Space We consider an output space Y = {( y , z )}, where y is a vector of real-valued outputs y: ℜ → ℜ m and z : ℜ → Σ O is a discrete-valued output, where Σ O is an output alphabet. As with the input v, we can define

{

z~ = ( t j , σ j ) j = 0 ,1,2,K , p;

}

t0 = 0,t j + 1 > t j ,σ j ∈ ΣO ,

where z can change value at arbitrary time instants. As the model P is an executable simulation, it is assumed that any integration routine in P has a finite nonzero minimum integration timestep and contains no algebraic loops, and so p is finite and the output ( y , z ) is defined for all inputs (u ,v ) ∈ U. The pairs (u ,v ) and ( y , z ) represent an extension to the notion of primary inputs and primary outputs defined in [13] and [17] to include discrete-valued inputs and outputs. It may be necessary to instrument the simulation model to bring the necessary switching values to an output value if these are needed in the coverage criterion. This could be

August 2003

(1)

where d(( y , z ), Rk ) = 0 if y( t i ), z ( t i ) ∈ Rk for some t i , and d(( y , z ), Rk ) > 0 otherwise. This provides a broad range of possible coverage criteria. For example, the requirement that a scalar output y must exceed a threshold c during a given simulation run is specified by setting R1 = [c , ∞ ). Testing that an output ranges over values in each of the intervals [0,1],[2,3],...,[9,10] can be specified by defining R1 = [0 ,1], R2 = [2,3],K , R10 = [9 ,10]. With appropriate definitions of the regions Rk , this measure of coverage can also be used to ensure that all branches of switching logic are exercised (e.g., if-then cases in a Simulink/Stateflow chart or guard conditions in a hybrid automaton [19]). For instance, for a scalar output y and a binary output z, letting R1 = ℜ × {1}, R2 = ℜ × {0} will check that the output z takes on both values during the course of the simulation.

Coverage Specification The last block in Figure 1 defines a coverage criterion S: U → {0 ,1} ( true, false ), based on the measure of coverage (i.e., S (u ,v ) = 1 ⇔ f ( P(u ,v )) = 0 ). An input (u ,v ) is said to satisfy the specification S if S (u ,v ) = 1. The test input generation problem can now be stated as follows.



51

Given a model P and a coverage criterion S, find an input (u ,v ) within the input space U such that S (u ,v ) = 1.

A GA Approach to Test Input Generation The coverage problem can be considered an optimization problem; that is, find (u ,v ) ∈ U so as to maximize the objective function f. The objective function is constructed such that the maximum value of zero is reached if and only if all

We now provide a short summary of GAs as needed for the solution of the problem. The reader is directed to any standard GA textbook (e.g., [20]) for a complete description; a summary of applications in control systems engineering can be found in [21]. GAs work by encoding the arguments of a problem into a chromosome (sometimes called an individual). A chromosome represents a point in the solution space (typically a finite-dimensional Boolean space or real number space). The basic idea of GAs is to maintain a chromosome set (also know as population) that evolves iteratively over generations through a process of competition and controlled variation. Each chromosome in the population has an associated fitness that determines the chromosomes that are used to produce the next generation (known as selection). The fitness is also the objective function when solving optimization problems. The goal is to find a chromosome with the best fitness. Chromosomes in the new generation are obtained from the chromosomes in the previous generation by mutation operators, which produce a new chromosome from a single chromosome, and by crossover operators, which produce a new chromosome by combining elements of two or more chromosomes from the previous generation. GA approaches are often successful because of the relative ease with which the practitioner is able to bring domain knowledge from the application area (such as relevance of parameters, sample times, or structural properties of the model) into the encoding, selection, and crossover mechanisms. The relative lack of formal results or categorizations of GAs has not limited their success in a wide range of problems (again, see [21] for control systems applications). GAs were originally developed for the setting of binary coding of combinatorics problems and have been extended to real-valued variables [20], [22] called real-coded GAs (RCGA). The chromosomes in RCGA are vectors of real numbers.

To develop automatic methods, the design must first be captured in a computer model that includes a representation of the plant as well as an embedded control algorithm. the regions Rk are reached. Given the complexity of simulation models for embedded control systems, traditional gradient-based optimization algorithms cannot be used to solve this optimization problem. Some type of global search technique is needed. Global optimization algorithms have received much attention in recent years. Several derivative-free algorithms have been developed, including deterministic direct methods and stochastic methods such as simulated annealing and GAs. The main advantage of the derivative-free algorithms is that no simple analytical form of the objective functions is needed. We propose to apply a GA-based global search engine to solve the test input coverage problem. Figure 2 illustrates the GA approach to test input generation. The top half of the figure represents the calculation of the fitness function by explicit simulation of the model followed by a measure of coverage. The bottom half shows the GA process varying the inputs to the simulation.

Chromosomes The performance of GAs depends greatly on the encoding method. In our implementation, the continuous inputs u are encoded as real-coded chromosomes and the discrete-valued inputs v are encoded with binary coding. The model inputs are captured without loss of information; that is, the input vector can be recreated completely from the Figure 2. Direct simulation of the model is used to measure fitness in a GA approach to test input chromosome (modulo generation. rounding errors). In the ex-

52



August 2003

amples provided in the next section, the number of samples for each input is held constant, so the size of a single chromosome remains constant. Although the input variables can be arranged in any order when they are encoded as chromosomes, a careful selection of the sequence will make the design of crossover operators more efficient. The initial population of chromosomes is seeded with random inputs.

Fitness Function The fitness of a chromosome (an input) is calculated by explicit simulation of the model with the chromosome translated to the input followed by a measurement of the coverage [see (1)]. The evaluation of the fitness function for a group of chromosomes provides potentially useful knowledge about the problem.The selection mechanism makes use of such information to guide the search procedure. Suppose x 1 ,K , x N is a population of chromosomes. A common selection approach assigns a probability of selection to x i , Ps ( x I ), as

where λ is selected uniformly on a bounded interval. Simple crossover takes advantage of the time-invariant nature of the hybrid systems in our applications. If one input sequence performs well in the beginning and another input sequence performs well at the end of the simulation, then combining the first piece of the former and the last piece of the latter may give a better input sequence if the states of the system match at the connecting time instants. Even when the states do not match, simple crossover may produce an input sequence that can be mutated to cover the mismatch in the state values. This crossover operator may

We propose to apply a genetic algorithm-based global search engine to solve the test input coverage problem

Ps ( x i ) =

f (x i ) N

∑ f (x j )

.

j=1

Other selection mechanisms are reviewed in [23]. The probabilistic selection above is employed in the results shown in the following section.

Mutation and Crossover Operators Crossover and mutation operators should be designed to take advantage of the structure of the problem. Commonly used crossover operators include simple crossover and arithmetic crossover (see [22] for details and other crossovers). Given two real-coded chromosomes x 1 = ( x 11 ,K , x n1 ) and x 2 = ( x 12 ,K , x n2 ), the simple crossover produces two new chromosomes x 1′ and x 2′:

be expected to take advantage of the linear component of the dynamics to produce better offspring out of good chromosomes. A new idea in our work is another crossover operator that makes use of the property of superposition. Formally, for given chromosomes x 1 and x 2 , the new crossover operator produces two chromosomes,

( (( x x ′ = min(max(( x x i1′ = min max 2 i

where the position i ∈ {1,K , n − 1} is chosen randomly. Note that if a binary encoding were used (i.e., each x ij is a binary bit rather than a real value), then the choice of position i would be restricted to boundaries between the encoding of the real values (see [13] for more discussion about alphabet size). The arithmetic crossover generates two new chromosomes, x 1′′ and x 2′′, as

August 2003

i

1

− x2

i

l i

u i

and (2)

x ′ = ( x 1 ,K , x i − 1 , si , x i + 1 ,K , x n ).

x 2′ = ( x 12 ,K , x i2 , x i1+ 1 ,K , x n1 ),

x 2′′ = λx 2 + (1 − λ )x 1 ,

) ) ) , x ), x ),

+ x 2 ) , x il , x iu

where i = 1,K , p * K , and x il and x iu are the lower and upper bound of the variable x i , respectively. The mutation operator used in real-coded GA plays the role of a local search engine that takes advantage of local correlations in the problem space. Mutation operators for RCGA include uniform mutation and boundary mutation. For a given chromosome x, uniform mutation randomly selects one variable x i and sets it equal to random number si uniformly distributed on the interval [x il , x iu ]. The result of mutation is given by

x 1′ = ( x 11 ,K , x i1 , x i2+ 1 ,K , x n2 )

x 1′′ = λx 1 + (1 − λ )x 2

1

The boundary mutation sets si to either the upper bound x iu or the lower bound x il .

Applications The MATLAB Simulink demonstration set includes a model of a drive train for an automobile, including a Stateflow chart for the logic for an automatic transmission [24]. To illustrate the GA approach to test input generation, we first consider the problem of generating acceleration profiles for testing the logic of the automatic transmission. The test inputs might be used to check the design model or could be



53

Figure 3. The automatic transmission example provided in the MATLAB Simulink demonstration package reduced to a subsystem with an input (throttle schedule) and outputs (speed and engine RPM). applied to the vehicle itself. In either case, it is desirable to have a test input with high coverage of the logical switching in the model. Figure 3 shows the Simulink model diagram for the automatic transmission subsystem. An input (throttle schedule) is pulled from the workspace and the outputs (speed and engine rpm) are sent to the workspace, as illustrated in Figure 4. The brake schedule has been fixed at the original values. The shift logic for the automatic transmission is shown in Figure 5. The test generation problem is to find an input throttle schedule such that a) the vehicle speed exceeds 120 km/h b) the engine speed reaches 4500 rpm c) all states are reached in the switching logic. To test criterion c), we require that the during function for each state in the Stateflow chart executes at least once during the simulation (see [16] for more information about the during function). This objective is translated into a fitness function of the form described in the previous section. The objective is to

find an input throttle schedule such that f ( P(u ,v )) = 0, where 9

f ( y , z ) = − ∑ d(( y , z ), Rk ), k=1

with d(( y , z ), Rk ) = 0 if y( t i ), z ( t i ) ∈ Rk for some t i and d(( y , z ), Rk ) = 1otherwise and the regions Rk , k = 1,2,K,9 are R1 = [120 , ∞ ) × ℜ × Σ O R2 = ℜ × [4500 , ∞ ) × Σ O Rj = ℜ × ℜ × {σ j ∈ Σ O },

j = 3 ,K,9

where Σ O is an alphabet that contains one symbol for each state in the Stateflow chart. In the Mathworks Performance Tools [18], the coverage tool provides a method for performing an instrumented simulation that records execution of the individual Simulink blocks and records the execution of the during function for each Stateflow state. We assume the execution of a state’s during function to be equivalent to the appearance of the state’s symbol in Σ O at the output. The sampling time for the throttle schedule is selected as 5 s, and the test is run for a total time of 30 s. Each chromosome is hence a six-element sequence, U = U 1 ,K ,U 6 , and the continuous-time input throttle value is given as u throttle ( t ) = U k , t k ≤ t < t k + 1 ,

Figure 4. The automatic transmission example as a subsystem with input and outputs where U k ∈[0 , 100] is the bound on possible throttle input values. connected to the workspace.

54



August 2003

Figure 5. The shift logic in the stateflow block for the automatic transmission. Typical crossovers, mutations, and selection mechanisms are used from the GAOT package [25], as well as the linear combination operator and restricted simple crossover described in the previous section. Each crossover (mutation) has equal opportunity to be applied when a crossover (mutation) operation is requested. A population of 20 chromosomes was used, and, in most cases, a chromosome with fitness 0 was produced in five generations or less. A typical input throttle schedule and output RPM and speed are shown in Figure 6. The chromosome in this case is

shown in the last column. Note that the number of executions of the model does not necessarily equal the population size multiplied by the number of generations because the fitness

U = 819 . , 314 . , 979 . , 85.2, 94.2, 82 .2. For this execution, all seven states in the Stateflow chart became active at some point during the execution. Several industrial examples from Ford Motor Company and General Electric Transportation Systems have been run, in addition to the example described above. The results are shown in Tables 1 and 2. For the small examples in Table 1, the GA produced inputs with the right cover- Figure 6. Inputs (throttle) and outputs (speed and engine RPM) for the generated test age in reasonable computation times input. August 2003



55

Example

Number of Blocks (SF States)

Number of Inputs

Sample Time (Sim Time)

GA Populations (Generations)

Model Executions (Compute Time)

Test drive (illustrative)

78 (7)

1

5.0 (30.0)

20 (5)

63 (45 s)

GE RTC embedded system

24 (4)

4

1.0 (10.0)

20 (2)

37 (30 s)

Ford drive train

232 (11)

6

1.0 (50.0)

40 (10)

372 (552 s)

does not have to be reevaluated for chromosomes that are carried into the next generation without being altered. We have applied our method to large-scale production models from industry. The results in Table 2 are for the GE Locomotive Traction Pilot model consisting of 1,478 Simulink blocks, 50 Stateflow states, and 15 input signals. The sample time and duration of simulation time were 0.01 s and 0.1 s, respectively. Results for three sizes of population of GA are shown in Table 2. For each population size, we ran 20 independent experiments. The maximum number of model evaluations for each experiment was set to 3,000. If the GA failed to find a test vector with a fitness function equal to zero (indicating it satisfies the desired coverage criteria) after 3,000 simulations, we terminate the experiment with the assumption that either a solution does not exist (which was not the case for the system) or the chromosome population has become “stuck” in a local maximum. Table 2 shows that the success rate of the GA method is still very high (90%), provided the population size is carefully chosen. With a fixed limit on the number of simulations, the population size should not be too small or too large. For this example, a population size of 16 is better than four or 64. If the population size is too small, the power of the GA to explore several possible locations in the search space in parallel is not fully exploited. If the population size is too large, the relative number of crossover and mutation operations that can be performed is limited because of the large number of chromosomes. At the extreme case in which the population size equals the model execution limit, GA will degenerate into a pure random search. We compared the GA approach to random search for the examples in Tables 1 and 2 and found that for the first three small examples, random sampling of the input space could

GA Population

Number of Successful Runs

Average Number of Model Execution (Compute Time)

4

18/20

1044 (1,000 s)

16

18/20

518 (500 s)

64

10/20

299 (300 s)

56

eventually generate a solution, but with many more executions of the simulation model than required by the GA approach. For the GE Traction Pilot example in Table 2, a random search never produced a suitable input despite very large sample sizes (3,000). Comparisons with approaches based on formal methods are not possible because there is no apparent way to build the required abstract models from the given simulation models, even for the small examples. In summary, the results of this section indicate that our GA method can automate the generation of test inputs for real-world problems. In most cases, test inputs can be generated within a reasonable time even for complex control systems with hybrid dynamics or large-scale productionsize models for which formal methods and random search methods cannot be used.

Conclusions This article presents a new technique for generating test inputs for embedded control systems using existing simulation models. The power of the technique comes from carefully designed elements of a GA. The construction of the fitness function, the chromosomes, and the introduction of special crossover and mutation operators all contribute to an algorithm that finds test inputs much faster than random search. The complexity of the problem prohibits the use of formal methods or traditional gradient-based optimization. The key feature of the proposed method is that no analytical model is required; tests are generated by nothing other than repeated execution of the simulation model. We have demonstrated the effectiveness of the method for large production-size models from industry. There are several directions for future research. The current fitness function is simply the sum of the indicator functions for sets of signals, representing the satisfaction of various criteria for acceptable test inputs. Fitness functions that provide better measures of the proximity of candidate test inputs to a solution might help speed up the convergence of the GA. To make the method effective for designers of embedded control systems, a user interface should be developed that helps with the construction of the fitness function for specific problems. Developing methods for exploiting more domain knowledge in the construction of crossover and mutation operators is also of interest.



August 2003

Acknowledgments The examples in this article were provided by Dr. Suresh Reddy of General Electric Transportation Division and Dr. Ken Butts and William Milam of Ford Scientific Research Labs; we gratefully acknowledge their support of this work. We would like thank the editor and the anonymous reviewers for helpful suggestions. This work was supported in part by Ford Motor Company, General Electric Transportation Systems, the National Science Foundation, Army Research Office, and the Pennsylvania Infrastructure Technology Alliance, a partnership of Carnegie Mellon, Lehigh University, and the Commonwealth of Pennsylvania’s Department of Economic and Community Development. Q.C. Zhao was also supported by NSFC under Grants 60074012 and 60274011, Ministry of Education of China and Tsinghua University project.

[17] M.K. Iyer and M.L. Bushnell, “Effect of noise on analog circuit testing,” in Proc. 16th IEEE VLSI Test Symp, Monterey, CA, 1998, pp. 138-144. [18] The Mathworks-Simulink performance tools—Simulink model coverage [Online]. Available: http://www.mathworks.com/products/slperftools/ [19] R. Alur, T. Henzinger, and P.H. Ho, “Automatic symbolic verification of embedded systems,” IEEE Trans. Software Eng., vol. 22, no. 3, pp. 181-201, 1996. [20] A. Wright, “Genetic algorithm for real parameter optimization,” in Foundations of Genetic Algorithms 1. San Mateo, CA: Morgan Kaufmann, 1991, pp. 105-218. [21] P. Fleming and R. Purshouse, “Genetic algorithms in control systems engineering,” Dept. Automatic Control and Systems Engineering, Univ. of Sheffield, Res. Rep. 789, 2001. [22] F. Herrera, M. Lozano, and J. Verdegay, Tacking Real-Coded GAs: Operators and Tools for Behavioral Analysis. New York: IEEE Press, 1996. [23] T. Back, F. Hoffmeister, and H.P. Schwefel, “Extended selection mechanisms in genetic algorithms,” in Proc. 4th Int. Conf. Genetic Algorithms, San Diego, CA, 1991, pp. 92-99. [24] Mathworks (2002), Stateflow, car simulation demo. [Online]. Available: http://www.mathworks.com/products/demos/stateflow/sfcar.html [25] C. Houck, J. Joines, and M. Kay, Genetic algorithm optimization toolbox (GAOT) for MATLAB 5 [Online]. Available: http://www.ie.ncsu.edu/mirage/GAToolBox/gaot/

References [1] B. Vinnakota, Analog and Mixed-Signal Test. Englewood Cliffs, NJ: Prentice-Hall, 2001. [2] M. Abramovici, M. Breuer, and A. Friedman, Digital Systems Testing and Testable Design. New York: IEEE Press, 1995. [3] M. Burns and G. Roberts, An Introduction to Mixed-Signal IC Test and Measurement. New York: Oxford Univ. Press, 2001. [4] M. Soma, S. Huynh, J. Zhang, S. Kim, and G. Devarayanadurg, “Hierarchical ATPG for analog circuits and systems,” IEEE Des. Test Comput., vol. 18, no.1 pp. 72-81, 2001. [5] A. Gupta, S. Malik, and P. Ashar, “Toward formalizing a validation methodology using simulation coverage,” in Proc. 34th Design Automation Conf., Anaheim, CA, 1997, pp. 740-745. [6] S. Huynh, J. Zhang, S. Kim, G. Devarayanadurg, and M. Soma, “Effcient test set design for analog and mixed-signal circuits and systems,” in Proc. 8th Asian Test Symp., Shanghai, China, 1999, pp. 239-244. [7] T. Henzinger, P.W. Kopke, A. Puri, and P. Varaiya, “What’s decidable about hybrid automata,” J. Comput. Syst. Sci., vol. 57, no.1, pp. 94-124, 1998. [8] J. Tofte, C.K. Ong, J.L. Huang, and K.T. Cheng, “Characterization of a pseudo-random testing technique for analog and mixed-signal built-in-self-test,” in Proc. 18th IEEE VLSI Test Symp., Montreal, Canada, 2000, pp. 237-246. [9] A. Gupta, S. Malik, and P. Ashar, “Toward formalizing a validation methodology using simulation coverage,” in Proc. IEEE/ACM Int. Conf. Computer-Aided Design, San Jose, CA, 2001, pp. 286-292. [10] P. Amman, P.E. Black, and W. Majurski, “Using model checking to generate tests from specifications,” in Proc. 2nd IEEE Int. Conf. Formal Engineering Methods (ICFEM’98), Brisbane, Australia, 1998 pp. 46-54. [11] P. Ammann and P.E. Black, “Abstracting formal specifications to generate software tests via model checking,” in Proc. 18th Digital Avionics Systems Conf. (DASC99), St Louis, MO, 1999, vol. 2, pp. 10.A.6. [12] A. Gargantini, and C. Heitmeyer, “Using model checking to generate tests from requirements specifications,” in Proc. Joint 7th Eu. Software Engineering Conf. and 7th ACM SIGSOFT Int. Symp. Foundations of Software Engineering, Toulouse, France, 1999, pp. 146-162. [13] M. Rudnick, J. Patel, G. Greenstein, and T. Niemann, “A genetic framework for test generation,” IEEE Trans. Computer-Aided Design, vol. 16, no. 9, pp. 1034-1044, Sept. 1997. [14] M. Srinivas and L. Patnaik, “A simulation-based test generation scheme using genetic algorithms,” in Proc. 6th Int. Conf. VLSI Design, Bombay, India, 1993, pp. 132-135. [15] A. Schultz, J. Grefenstette, and K.D. Jong, “Learning to break things: Adaptive testing of intelligent controllers,” in The Handbook of Evolutionary Computation, T. Bäck, D. Fogel, and Z. Michalewicz, Eds. New York: IOP Publishing and Oxford Univ. Press, 1997, G3.4 . [16] The Mathworks: Developers of MATLAB and Simulink for technical computing [Online]. Available: http://www.mathworks.com

August 2003

Qianchuan Zhao received the B.E. degree in automatic control, the B.Sc. degree in applied mathematics, and the Ph.D. in control theory and its applications in 1992, 1992, and 1996, respectively, from Tsinghua University. He is currently an associate professor in the Department of Automation, Tsinghua University. He also holds visiting positions at Carnegie Mellon University and Harvard University. He is an associate editor for the Journal of Optimization Theory and Applications. His current research interests are DEDS theory and the optimization of complex systems. He can be reached at the Department of Automation, Tsinghua University, Beijing 100084, China, and the Department of Electrical and Computer Engineering, Carnegie Mellon University, 5 0 0 0 F o r b e s Av e , P i t t s b u rg h , PA 1 5 2 1 3 , U . S . A , [email protected]. Bruce H. Krogh is professor of electrical and computer engineering at Carnegie Mellon University, Pittsburgh, Pennsylvania. He was an associate editor of IEEE Transactions on Automatic Control and Discrete Event Dynamic Systems: Theory and Applications and founding editor-in-chief of IEEE Transactions on Control Systems Technology. His current research interests include discrete event systems, hybrid dynamic systems, and synthesis and verification of embedded control system designs. Paul Hubbard received the B.Sc. and M.Sc. degrees, both in mathematics and engineering, from Queen’s University, Kingston, and the Ph.D. degrees in electrical engineering from McGill University, Montreal. He has worked previously with Lockheed-Martin Canada on electronic counter-measures, with BBN Technologies on command and control systems, and at Carnegie Mellon University on automatic test generation. He is currently a defense scientist at Defense Research and Development Canada, Ottawa, Ontario, where he works on the autonomous intelligent control of unmanned systems.



57