Generating Tests by Exploiting Designed Behavior - Association for ...

From: AAAI-86 Proceedings. Copyright ©1986, AAAI (www.aaai.org). All rights reserved.

Generating

Tests by Exploiting Mark

Harper

MIT Artificial

Designed

Behavior

Shirley

Intelligence

Laboratory

545 Technology Square Cambrzdge, MA 02139 ([email protected])

Abstract One of the hardest pattern

generation

problems

for a complex

because it requires reasoning behavior can be extremely operations

that

behavior.

in digital device.

circuit

design

This is difficult

about how to control complex. Knowledge

the device was designed

is test in part

a device whose of the specific

to perform

can help solve

this problem. The key observation is that a device’s designed behavior is often far more limited than the device’s potential behavior. This limitation translates into a reduction of the search necessary to achieve planning goals. We describe an implemented program

based

on this idea.

When

this

to generate

tests

where

illustrates both We represent graph, through

the operations

defined

tion set defined to be complete.

When

reasoning

specializing

about

how to manipulate

complex

systems (e.g. for test generation or diagnosis), system is designed to behave is an important to apply. This used in several

knowing how the kind of knowledge

paper demonstrates how this knowledge can be ways to produce an effective problem solver for

one of the hardest problems in electronic design: generation for VLSI devices. Test generation encounters several traditional including

conjunctive

planning,

and remains

test

pattern

AI concerns,

an important,

un-

solved problem: existing programs are notably unsuccessful large circuits. Human test programmers are more successful, part because

ning

engineered

they use knowledge

from many

on in

and a variety

sources

of reasoning techniques. Here we focus on one kind of knowledge - designed behavior - which they find important. At the core of our test generation one key observation: tions that ited than of inputs When signed

a device’s

a device

is intended

to carry

its potential behavior. to a disk controller

solutions behavior,

system

designed

is a planner

behavior,

based

This

to planning problems can be found within this limitation can translate into a reduction

Hypothesis: efficiently

work that

relies every

exercised

on what component

while

staying

deof

compo-

we call the Designed-Behavior in the device within

can be fully and

the device’s

designed

This report describes research done at the Artificial Intelligence Laboratory of the Massachusetts Institute of Technology and at GenRad Inc. Support for the laboratory’s Artificial Intelligence research on digital hardware troubleshooting is provided in part by the Digital Equipment Corporation, and in part by the Advanced Research Projects Agency of the Department of Defense under Office of Naval Research contract N00014-80-C-0505.

884

National

Conference

/ ENGINEERING

method

is most

interfaces,

ating a behavior lieve the method of the building

e.g. the instruc-

The list of operations

value,

a testing

against

matches

to create

appropriate

and

section

for devices

characteristics of this problem strategy. Section 4 introduces examples throughout tails of our method.

This

whose

and plan-

command

of distinct operations they the non-trivial effort of cre-

of microprocessor

3 describes

graphs

solutions.

circuits, we betests for many

systems

(e.g. processors,

UART’s, graphics controllers, and the like). The next section gives an overview of the problem,

is assumed

goal, like loading

the behavior

graph. In the domain of digital is appropriate when generating blocks

which

with a behavior

behavior

interfaces are narrow in the number admit, since each operation requires

the

test

generation

consequences

for representation a simple processor

the paper, while section The method’s performance

of several

and reasoning which provides 5 contains the deon the processor

is shown in section 6. Finally, a simple account of processor design is presented in section 7, which provides justification for the performance

2

achieved.

on Artificial

intelligence

The

Task

We are interested

in the task

of test

generation

for VLSI com-

ponents. This task is part of the quality control phase of circuit manufacturing where we verify that the physical circuit in fact Test generation takes a design produces the designed behavior. description for a circuit and produces a set of tests. Taken together the tests must specify sets of inputs and predicted outputs such that, puts

AAAI-86

successful

is shown

tests fails

how data is supposed to flow these graphs by simulation of

by matching

a certain

neces-

out, is often far more lim-

For example, very few sequences correspond to valid commands.

the search necessary to achieve planning goals. In testing, the goals involve manipulating individual nents.

on

i.e. the opera-

operates

with

An example

at the device’s

for a processor.

the search

successfully generates is valid, and (quickly)

of designed

a structure that describes We generate the device.

a register

Introduction

we can reduce

it is not.

cases. the space

The planner

1

is true,

sary to generate tests. Our program in situations where this assumption

match

when

the inputs

the predicted

are applied values

to the circuit

(i.e. the circuit

we can be confident the circuit was manufactured Tests are traditionally created by partitioning

and all out-

passes

all tests)

correctly. the design

into

components and generating a test for each component by (1) working out how to test the component as if it were alone, and (2) working out how to execute that test within the context larger circuit. The second problem is conjunctive planning,

of the since

it involves

achieving

multiple,

potentially-interacting

have to work out how to manipulate to cause

a desired

The

classical

pattern

of activity

approaches

goals:

the circuit’s

(e.g.

around

primary

a component.

[Roth66])

use

a combination

of heuristic

search and constraint propagation using representationsof the circuit’s structure and behavior. runtimes are excessive for anything cuits and simple state machines,

we

inputs

gate-level Current

other than combinational cirand we think this is a direct

consequenceof the use of gate-level

representations. More recent to hiresearch (e.g. [SinghG]) appl ies similar search techniques erarchical models of device structure and behavior, yet still does not take advantage of knowing the designed behavior. For com-

plex circuits,

the human experts any existing test generation

than

describes

several

limiting

3

ways

remain much more successful program, and the next section

- inspired

by what

the

experts

Figure

In this

section

of the Problem

we describe

two

domain and the effects and reasoning strategy.

they

3.1

Designed

Purposefully

characteristics

have on our choices

designed bewhich essen-

tially

circuit

are annotated

or the

notion

contain expressions described in terms

problem

The point tained within

of representation

has

been

used

in several

unspecified,

analog

circuit

we need

to

was sufficient

worked.

know what

We require that

purpose

to aid in determining more

the relative algorithms

behavior

of the behavior graph representation the set of graphs are many patterns

information

is, i.e. the tasks

than

sizes of solution spaces search. The sets contain

graphs

can provide interfaces.

generated

behavior. Hence, dicates that there

this;

the system

the designer

behavior

how an

opergraphs

is this: conof activity

a complete Because

from them

list of operations

de-

this list is complete,

the

represent

all normal

circuit

failure to find a useful pattern of activity is no solution within the normal operations

inof

the circuit.

is

supposed to accomplish. Our program then uses this information to focus on solutions to testing problems. Figure 1 illustrates different test generation

For efficiency,

with variables in them, e.g. a READ cycle is of a symbolic address and symbolic data.

fined for the circuit’s

programs for reasoning about physical systems. In [DeKleer79], for example, the knowledge that a device had some purpose, even though

traces.

and the corresponding

around every component in the circuit. “All our planner must do” is find the patterns of activity that are useful in testing. We

Systems

of purpose,

simulation

are parameterized,

assume Teleology,

states

We explicitly represent the space of a circuit’s havior with the behavior graphs mentioned above, ations

Domain of the

sizes of sets of circuit

do - of

search.

Characteristics

1: Relative

that pos-

Extreme

3.2

A second

and

Complexity

dominating

characteristic

of this

domain

is the

sible states of a circuit, i.e. the different ways of consistently assigning values to the circuit’s nodes. If the components com-

extremely complex behavior of modern VLSI components. Algorithms which manage this complexity are essential. One way to

prising the circuit are disconnected, then there is no constraint between their states, and the set of possible circuit states is the

do this is to identify

cross product of the sets of individual component states. Connecting the components together makes many potential states inconsistent, resulting in a smaller set of possible states. This is

one that

focuses

closely

involves

the rest

of the circuit,

the space that a planner with perfect ability straints could search. Since local constraint

to propagate

con-

propagation tech-

The

traditional

build our problem ing at a different micro-planner

useful

abstractions2.

partitioning

of the problem

on a single

into

component

suggests

that

two parts,

and another we might

that

usefully

solver in two parts, each part potentially workabstraction level. And this is what we do: a

is responsible

for solving

conjunctive

goals around

niques are incomplete in their ability to detect global conflicts, algorithms using these techniques search a somewhat larger space

individual components, and a macro-planner is responsible for taking the output of the micro-planner and then controlling grosser

and backtrack

aspects of the circuit We must describe

when

inconsistencies

are found.

Within the space of consistent states space of behaviors the circuit was designed

of the circuit lies the to perform. This is the

space our test generator searches. The ratio of designed behavior to possible behavior depends upon the particular circuit, but the more specialized

and complex

the circuit,

the smaller

it tends

to

be. 1 ‘In the case of the MARK-2

processor,

this ratio

is on the order of 22”“.

Naturally this number says nothing about how the spaces are searched nor about the frequency and distribution of solutions within the spaces. We haven’t yet done the performance measurements for some typical circuits that would yield these numbers. But, as suggested by the performance of our method on a microprogrammed processor, we expect our method to be a useful addition to the spectrum of techniques helpful for solving test generation problems.

state, namely its data registers. the circuit structure and behavior

two parts of our program. system models the circuit nents,

roughly

for these

For simplicity’s sake, the prototype structure as a flat network of compo-

at the level of the block

diagrams

in databooks.

3

The system uses four levels of behavioral description. The simulation models for the individual components are the foundation. We only require them to predict outputs from inputs, 21n fact, because are abstractions designer.

the circuits are designed, it is not surprising that there readily available; they are the abstractions used by the

3Note that, although individual gates.

the model is flat, it is considerably

above the level of

AUTOMATED REASONING

I 885

EXAMPLE

INPUT

SEQUENCES

+ COMPONENT

1

VIA

BEHAVIOR

SIMULATION

.

tion after

GRAPHS

~UMMIRIIAT~VIA

REGISTER

accomplished using Design for Testability techniques. The next section introduces a simple processor which we use as an illustration throughout the rest of the paper, and the sec-

BEHAVIORS

SUMMARIZATION COMPONENT

CHANGES

that

4

The

MARK-2

The

circuit

shown

2: Behavior

fictional

Representations

two are summaries Activation havior example,

as Lisp functions. mentioned above.

of the behavior

Sunzmary

graphs

directly graphs

which

links

contain

if the simulator

graphs.

component

notices

that

sometime during an instruction, is linked to the behavior graph

The next The final

instances

MAC-l

the ALU performs

registers. Figure 2 shows tions are generated.

3.3

Summary

When

reasoning

VLSI circuits,

an add

the cuinternal

Characteristics

The first observation

gives us a way to overcome

culties resulting from the second. These observations implications which together guide us to the strategy representing

5

The

out:

first, that they are engineered artifacts, designed to accomplish a purpose, and second, that their behavior can be extremely complex.

be controlled

each part

stand

the space of designed

behavior

the diffi-

The

ways

(e.g. by observing

We use simulation

because

which

fails

but

quickly

on the

it is much

amount

worse

of time with

Our designed present,

method

rest

so that

it is undesirable

quickly

and

either

terested simulate

other

methods

can

generator

to fail,

be

and unpredictable

using

could variables

The inputs erations

the circuit’s

behavior for solutions to testing goals. If a solution is then our program will find it, and it does so quickly

and

components.

add,

of be-

flows through the it could be done

version create

of the circuit). behavior

graphs

we need

only

simulate

Rather than of numbers the instruction

for data.

of operations

are the complete

set of circuit

plus perhaps

provided

inputs

by test

as programs,

experts.

because

op-

some interestthat

We repreworks

well

with another approach to test generation discussed in [Shirley85]. We divide the circuit state into two components: control state and data state4. Control state is initialized the same way for each simulation variables. For example,

is an effective test generation strategy. shows the performance of our program

1).

6

a database

data, thus reducing the number simulation we need. For example, suppose we’re in-

because the search space is limited. To the extent that the Designed-Behavior Hypothesis is true, i.e. that efficient tests for components exist within the designed behavior, then our method The example in section on a simple processor.

a working

from the databook,

sent these simulator searches

is to create

it can easily

to the simulator

collected

ing sequences

way.

exhaustively

busses inputs

accessible

”

intermediate

in the ADD instruction of the MARK-2. the instruction once for each combination

the processor

-

for a test

for it to run for a large no result

are not directlv

through

of the preprocessor

once,

Naturally,

23 instructions; and data primary

graphs which describe how information We do this using simulation, although

some

applied.

nodes

in

in turn.

purpose

in other

graphs

as important

is a lgbitincluding a

for managing global aspects of the circuit’s captured in the behavior graphs. We discuss

solves

- and just

of this circuit is a sequencer,

implementing

of the

Details

containing symbolic runs (or observations)

quickly

portion portion

indirectly

and of searching those graphs for solutions to testing goals. We recognize that test generation is a very difficult problem: partial ‘solutions are likely to be the norm in this field for many years to come. Consequently, rather than trying to build a test generator which solves all problems, we are aiming for one which of the problems

example

Preprocessor

havior circuit.

have several of explicitly

with behavior

version

Our method has’three parts: (1) a preprocessor for creating the for finding patterns of acbehavior graphs, (2) a micro-planner tivity among the behavior graphs which solve testing goals, and

representa-

two observations

as a teaching

The internal

must

micro-

presented

processor

(3) a macro-planner state which aren’t

of the Domain

about

behavior

horizontally

modified

80 lines of microcode

and outputs.

For

then the ALU’s add operation generated by simulating that in-

how the various

5 is a simple

It is a slightly

The central the righthand

ROM holding

to the beof them.

The Register Transfer Summary captures effects each behavior graph has on the circuit’s

struction. mulative

I

and the lefthand portion is a RAM. The address and their associated signal lines are this circuit’s

Component

The operations

one or more

.

of our method.

Microprocessor

in figure

processor.

[Tanenbaum84]. wide datapath; so we can implement them level is the set of behavior

the details

ACTIVATIONS programmed

Figure

describes

and is left by most The simulator

run, while the program

instructions

data state is initialized with counter is initialized to ?PC

holding

itself is event-driven,

the expression where

an event

(+ ?PC is a circuit

The question of what the designer should do when the test generator fails can be answered by stepping back from the test generation problem and considering its place within the larger

and simplifying symbolic expressions and (2) recording justifications between events. These justification links form a flow graph

context

which

of circuit

but one part of testability

design

and manufacturing.

of the design task. against performance

Test

generation

Thus, if generating tests for a given circuit requires effort or is impossible, then the designer can consider the

886

circuit

to make

the

/ ENGINEERING

problem

is

It is possible to trade off ease and reliability considerations.

easier.

Any

redesign

too much modifying can

be

node changing

its value.

is one component

Its essential

features

of the behavior

are (1) propagating

graph.

The expressions

‘This division depends on the level at which the circuit is described, but for any given level it is fairly clear and generally corresponds to the division of controller and datapath. For the MARK-2, the control state is the microprogram counter and the micro-instruction register, and the data state is everything else (e.g. the accumulator and the program counter).

Specific Patterns IrJPUT-1 II/PUT-2 0 65535 65535 0 1 65535 2 65535 4 65535 8 65535 16 65535

A composite event with subevents: AT ?Time = :PLUS OP IIIPUT-1 = ?Value-1 AT ?Time Ilr'PUT-2 = ?Value-2 AT ?Time OUTPUT = ?Output AT ?Time and constraints: (and (controllable? ?Value-1) (controllable? ?Value-2) (observable? ?Output))

Time

-------------

----------_

98 99 100 -> 101 ?AC) 102

:PLUS

cuit

the values

inputs

the data

during

registers.

to say where another,

are functions

simulation

The variables

they

much

of nodes the

originated,

simplified,

the

initial

in these expressions

consequently, flow graph

node back to primary inputs. of these two flow graphs.

5.2

of the values

run and

The behavior

.----------_

---------???

?Addend

OUTPUT

i+ ?Addend

?AC

of

of ADDD’s

Figure 4: Portion

behavior

graph

(shown

as a table)

are marked

the values graph

form at each

is the union the matcher and irrelevant

The Micro-Planner

The micro-planner’s

II:PUT-2

on cir-

states

the expressions

linking

IrlPUT-1

??? indicates an unknown value . indicates an unchanged value (i.e. ditto) -> indicates the match ?Addend = Contents(RAId,?ADDR)

Figure 3: An event pattern

describing

OP

to divide the set of behavior subsets depending on whether

graphs into relevant the component “did

anything interesting” during a particular behavior. then tries each potentially relevant behavior graph

task is to take a test for a single component,

The matcher in turn until it

expressed in terms of the component’s I/O pins, and to rewrite it in terms of the circuit’s pseudo-inputs and outputs (i.e. pri-

The matching finds a situation which meets all of the constraints. process itself is straightforward unification augmented to check

mary

the additional

I/O

pins

plus data

registers).

The core of this planner

is

a matcher which takes component tests and finds them in the behavior graph database. If successful,

instances of this planner

produces

will do most

a sequence

of the work

needed

of inputs to cause

The remainder of the work Where do the component two sources: for standard devices, we ask the domain

for the circuit

which

the test to occur

within

though we have not yet tried to do this. Component tests are expressed in two parts: of prototypical operations that the component (e.g. a register

test data provide

(e.g. have

must

try to load these

language

data

are just

sequences

in figure

3 describes

The pattern

for describing

and (b) actual numbers).

prototypical

We

operations;

of numbers. a prototypical

of the MARK-~‘S ALU. The four subevents between place, value and time that define tion. The additional match only instances

we could method,

(a) descriptions must be able to

be able to load data)

the register

a rich

the actual

from

combinational devices and sequential experts. For other combinational de-

vices, we use a conventional test generator. In principle, also use a recursive application of our test generation

perform

the circuit.

is done by the macro-planner. tests come from? We get them

constraints further of add operations

pattern

The matcher

constraints.

finds numerous

havior graphs i.e. when the

the behavior

graph

(ADDD)

, which

amount

stored

in main

?Time ?Value-1 ?Value-2 ?Output

specialize this pattern to that are useful for test-

inputs.

As

The

program

portion

to apply matching binding pattern

by an of this In all of

specific test data an event pattern environment. The in figure 3 against

75 Contents(RAId, ?Addr) ?AC (+ Contents(RAM, ?Addr) ?AC)

address

set up so that

from section

3.2 allows

which

At this point

determine what inverse function

came out of the ALU, by applying the that we observe at the circuit output.

can then

(i.e. ?Value-2) instruction Moreover,

Summary

relevant

== == ==

back-solve

to determine

properly. Similarly, there must be a output of the circuit which is an invertible function of the ALU's output. With that we can

Activation

The

==

some

into the two primary

test data moves through the circuit to the ALU, it is back to the desired value, thus exercising the ALU

The Component

memory.

instruction

of the accumulator

add operation

describe the relations the ALU’s add opera-

then

actually to data

the value

Recall that our purpose here is to the ALU. An additional result of against a behavior graph is a variable bindings resulting from matching the the behavior graph in figure 4 are:

isters in order to apply In this case, one addend

data

be-

graph appears in figure 4, displayed in tabular form. the other potential matches the ALU is either incrementing the in all of these cases, one program counter or the stack pointer: of the ALU’s inputs is a constant, and thus is not controllable.

of two distinct primary inputs. This allows us to apply test data to the ALU by transforming the data using the inverse functions, the inverted

the MARK-~‘S

for the ADD-Direct-from-memory

increments

environment

feeding

where

many situations in which the ALU performs an addition. But only one of these potential matches meets the additional restrictions needed for it to help test the ALU. This match occurs in

ing the ALU. In order to help, the expressions on the two input nodes must be controllable, i.e. they must, be invertible functions

the inverted transformed

places

match the subevents of the pattern in figure 3, processor is “put through its paces,” there are

must

the equations

in this binding

how it must set up the circuit’s regparticular values to the ALU’s inputs. (i.e. ?Value-1) must be in the RAM at

can be specified

later,

and the other

addend

be in the accumulator.

the micro-planner

has now identified

the ADDD

as the one to use to test the ALU’s addition operation. it has also determined how the data registers must be the ADDD

instruction

the job of the macro-planner operations which will initialize

can do this job.

It is now

to work out a sequence of circuit the data registers properly.

AUTOMATED

REASONING

/ 887

5.3

The Macro-Planner

The

purpose

of the

operations

macro-planner

(in this case

the circuit’s micro-planner

descriptions is to generate

sequences

of

set up

leave important data in one or more registers the ALU’s output in the accumulator), a sec-

ond job of the macro-planner to one of the circuit’s

is to work out how to move

outputs

The macro-planner

where

searches

based

on the sequence

contained

through

length

in the registers.

results

it can be observed. the space

sequences to find one which moves data planner is implemented as straightforward a metric

which

state so that the single test operation found by the will start with the right data. Since a test opera-

tion will sometimes (e.g. ADDD leaves

data

sequences

of instructions)

and

pattern

designed

simulation

method)

(a well-behaved

operations for testing

macro-planner, set up circuit

The with

of useful

descriptions

(i.e.

6

inputs ,I II

5

---->

on an Example

Figure 6 shows the fault achieves with the MARK-2 The

chose the ADDD information for the

program

coverage that processor6.

was able to generate

while

to

(PRELOAD-IIEIYIORY ?Ac ?ADDR) (LODD ?ADDR) (PRELOAD-MEMORY ?vAm ?ADDR) (ADDD ?ADDR) (STOD ?ADDR) (OBSERVE-14~140~~(+ ?VALUE ?AC) ?ADDR)

Performance

components,

of all of this is the sequence

which subsequently included three operations state and two operations to observe it.

1: 2: 3: Test ------------> 4: Observe outputs -> 5: 1, 6: Setup

it failed

our

7

Explaining

Circuit prototype

system

method

operations

Note

We believe

Consider starts

testing

the datapath

to realize despite

trolled almost completely of the program’s success

that

the fact

the program that

should of the

some ality.

of the mapping

is easily

between

able to

component

following

account

a specification

of processor

for an abstract registers abstract

design:

machine

and’the machine

a designer

which

includes

instruction set. His in hardware while

is con-

by 80 lines of intricate microcode. Part can be attributed to its use of abstract

describing capabilities of the tester two special operations oper .ations and observe-memory hardware. These are the preload-memory which write data into and read data from the RAM. has 1G components and 91 nodes. Each simulation run is approximately 50 clock cycles long, taking about 10 seconds of real time on a Symbolics 3640. Generation of prototype tests, i.e. tests with symbolic data, for this circuit takes 1 minute, including both the time taken for successfully creating tests for some components and failing to do so for

instruction,

from

register

these

describing

how data

to register.

dataflow

graphs

Naturally directly

To do this, he adds identity transformations. in one graph

in hardware:

as can’t

without

boxes

gle multiplexor. collects

to the graphs components which perform For example, he might insert a reg-

to shift

allowing

some

components

via time-multiplexing.

of its operations

to be shared

In another

into two graphs When

all the control

signals

from

By this account, tal refinement controller chine).

and

them

large

portions the fact

of individual

plus

that

is designed

different

change means

existing components that many datapath

processor

the merging

and creates

a

these signals at the right any one of the well-known

from

is designed (stylized implementation The availability of components which

chips),

a sin-

the designer

all the graphs

is very

graph

introduce using

fit together,

the way the datapath merging)

in time,

another

he might

and implement

the flow graphs

later

with

situation,

finite state machine (FSM) to provide times. This FSM is implemented using methods (e.g. with a PLA).

ment

is transformed the designer

sharing there would be a wasteful duplication of functionSo he looks for ways of merging the graphs together.

identity in

‘We include

GThis circuit

the circuit

varies over different parts of the circuit, is a direct result of the design process.

the programmer accessible job is to implement this

implement

that

succeeds

the datapath

the directness

the

with

thereby

It is also important

between

of the components that implement between the more “direct” the relationship operations and circuit operations, the bet-

and circuit operations and that this variation

ister

be modified. Thus our program’s failure points out areas circuit that a redesign program should focus on.

on the relationship

of the processor, and consequently, our method solve testing goals involving that ALU operation.

the expert

but really

depends

ter our method performs. For example, the ADD operation of the ALU very directly implements part of the ADDD instruction

since data paths are typically much wider than control paths. ’ Our program also quickly fails on the controller, an area which with effort,

the Performance

and the operations

one for each

for the highlighted

to do so for the others.

testable

to solve problems

side.

meeting myriad performance, reliability, and cost constraints. The specification can be viewed as a set of dataflow graphs,

the program can quickly work out how to test virtually all of the datapath, which includes more than half of the physical circuit,

says is partially

on the other

method is that it components using

Why is there a radical difference in coverage between the datapath and the controller ? We conjecture that the performance

it moves tests

through

of instruction

and the amount

below. The micro-planner the ALU and provided

reasoning

the components

them. In particular, individual component

of circuit operation

use of simulation backwards

of our complex

from the behavior

graphs.

the

reasoning

would try to do. A clear advantage can work forward through extremely

of our

of the end product

Also,

is critical:

sequencer is very difficult, yet this is exactly what planners and classical test generation algorithms

descriptions of the effects of instructions) are contained in the Register Transfer Summaries, which are generated by our system An example

behavior.

matching

the microcode goal-directed

involving

around properly. best-first search

The operator

of the circuit’s behavior (e.g. while controlling the The program generates these abstract descrip-

register state). tions from the

(incrementhe way the

of a state madirectly imple-

operations

process

doesn’t

(it just adds new identity component operations tend

(e.g. ALU normally boxes), to “very

directly implement” circuit operations. The state machine design process, however, need yield no such simple relationships. The behavior of the whole different from the behavior register,

888 / ENGINEERING

ROM,

or MUX).

controller (a state machine) of any controller component

is very (e.g. a

CA

I

MFIRK-2

Figure

5: The MARK-2

Processor

HA ,I I, 1 1 I

MARK-2

Figure

6: Coverage

of processor

components

AUTOMATED

REASONING

/ 889

This account

also suggests

a difference

between

References

our approach

to test generation and the classical approaches. The usual result of the design process, a circuit topology, is the superposition of the dataflow graphs, plus the identity boxes added to allow sharing, plus the controlling state machine. include the list of operations the circuit

This topology doesn’t can perform, nor does

it make explicit

the circuit,

how data

moves through

e.g. timing

[DeKleer79]

[Roth661

attempt

nents

to generate

and topology.

tests

Testing

directly

experts

from the circuit

perform

compo-

well, because

8

are an approximation

of the flow graphs.

[Shirley851

they

8

Conclusion

We have demonstrated how knowledge of a system’s designed behavior can help solve test generation problems, or more generally, problems involving manipulation The key to our method is that be far more limited can translate planning

than

of complex engineered systems. a device’s designed behavior can

its potential

into a reduction

behavior.

of the search

This limitation

necessary

to achieve

goals.

A prototype system has been implemented and run on a small number of examples. This system has 3 components: (1) a preprocessor to create the behavior graphs using symbolic simulation of input sequences out of the databook; (2) a micro-planner to match simulation tivation aspects planner’s

testing goals gotten runs (the matcher

from domain experts against is guided by the Component

the Ac-

to control,global Summaries); and (3) a macro-planner of the circuit’s state (the behavior graphs also supply the operator

descriptions).

The system has generated tests for a simple apath. It was successful despite the fact that

processor’s datthe datapath is

controlled by 80 lines of intricate microcode. A clear advantage of our method is that it can, by using simulation, reason forward through complex components, such as the controller, in order solve problems involving the components on the other side.

9

to

Acknowledgments

The following to the contents Yehudah

people

have read

of this paper:

Freundlich,

drafts

Randall

Jeff Van Baalen,

or otherwise Davis, Walter

contributed

Gordon

Kramer, Howard Shrobe, Ralil ValdCs-Perez, Brian Peng Wu. I would especially like to thank Gordon who is the testing

expert

whose

methods

Robinson,

Hamscher,

inspired

Glenn

Williams, Robinson,

much

of this

work. ‘The domain experts report an interesting dichotomy. They say that, for humans, test generation problems are either very easy or very hard. The problems are easy when they can see clearly which manipulations of the device’s interfaces are useful for exercising components inside the circuit, and hard otherwise. In our work, we have have made some effort, albeit informally, to match the performance of our test generation system to the expert’s intuitions about which problems are easy for humans and which are hard. And the expert does report that the results on the MARK-2 match his expectations fairly closely.

890

/ ENGINEERING

of Technology,

Roth,

J. P., “Diagnosis

1979, AI-TR-529. of Automata

Failures:

A

278-291. Shirley,

M.,

“An Automatic

proach to Testing,” IEEE lation and Test Generation

know the same things designers know and can reconstruct and use the designer’s flow graphs. Finally, our program is successful for the same reason, it reconstructs and uses behavior graphs, which

stitute

Calculus and a Method,” IBM Journal of Research and Development, Vol. 10, July 1966, pp.

and selection relationships, both of which were explicit in the flow graphs before they were merged. Classical test generation algorithms perform poorly, because they

de Kleer, J., Causal and Teleological Reasoning in Circuit Recognition, Diss., Massachusetts In-

Programming

Ap-

Workshop on SimuSept Environments,

1985. [Singh85]

Singh, N., Exploiting Design Morphology to Manage Complezity, Diss., Stanford Department of Electrical Engineering, April 1985.

[Tanenbaum841

Tanenbaum,

A. S., Structured

nization, Prentice-Hall, NJ, 1984.

Inc.,

Computer Englewood

OrgaCliffs,