Aug 8, 1991 - by. Thomas R. Gruber and Daniel M. Russell .... [Ullman, Wood, Craig, 1989]) for a designer to use when considering its implications, and to ...
Knowledge Systems Laboratory Technical Report KSL 90-45
August 8, 1991
Design Knowledge and Design Rationale: A Framework for Representation, Capture, and Use by Thomas R. Gruber and Daniel M. Russell
KNOWLEDGE SYSTEMS LABORATORY Computer Science Department Stanford University Stanford, California 94305
Design Knowledge and Design Rationale: A Framework for Representation, Capture, and Use Thomas R. Gruber Knowledge Systems Laboratory Stanford University Daniel M. Russell Systems Sciences Laboratory Xerox PARC July, 1990
Abstract Knowledge about the rationale for a design—how and why a device is designed as it is—can be valuable, but is difficult to capture in reusable form. This paper presents a view of design knowledge capture and the use of design knowledge for design rationale. We define design rationales as explanations in response to questions about the design. These explanations are generated from knowledge of artifacts and design activities. We characterize design activity in terms of observable changes to design descriptions, and present a theory of design knowledge in the form of an ontology of concepts about design descriptions and operations on them. The theory unifies artifact description and decision-making views of design. Based on the theory, we characterize different methods of acquiring design knowledge and design rationale in the context of integrated design support environments. We then analyze in depth two design knowledge capture techniques: a semiformal representation tool and a model-based explanation system, and we explain their functions and how they work.
Introduction Designers create descriptions of artifacts to achieve functional requirements. During design they choose structures to achieve intended functions, and evaluate design decisions against criteria. In the process they modify the descriptions of artifacts, and they clarify and refine the intended functions and requirements. Valuable design knowledge underlies this process. A design rationale is an explanation of how and why an artifact, or some portion of it, is designed the way it is. A design rationale is a description of the reasoning justifying the resulting design—how structures achieve functions, why particular structures are chosen over alternatives, what behavior is expected under what operating conditions. In short, a design rationale explains the “why” of a design by describing what the artifact is, what it is supposed to do, and how it got to be designed that way. Design rationale represents design knowledge that is used by people— possibly assisted by interactive computer aids—for engineering tasks including manufacturing, verification, diagnosis, and redesign. In current engineering practice, this knowledge is often lost because designers forget it, leave the project, or are inaccessible in a large organization. The task of eliciting, recording, and organizing design knowledge is called design knowledge capture (DKC). But collecting design knowledge is difficult, while creating and collecting design rationales is even harder. If we are to build computer-based tools to support design rationale use, we need to understand what kinds of knowledge to capture, how to represent the knowledge, and how it can be used to generate design rationales. DKC tools have to go beyond the services provided by traditional Computer-Aided Design (CAD) systems. Traditional CAD drawing tools capture only data about the physical or logical aspects of an artifact, and require detailed information that is available only after the design is essentially finished. DKC tools capture and use information such as the intended behavior or function of a device, and the justification for design decisions. They support the conceptual, decision-making stages of design when the knowledge underlying design choices is available. This paper presents a view of design knowledge capture and the use of design knowledge for design rationale. We first discuss the nature of design rationale, the problem of capturing the underlying knowledge, and the way the knowledge can be used. This motivates a theory of design knowledge, which we present in the form of an ontology of interrelated concepts. Based on this theory, we characterize different methods of acquiring design knowledge in the context of integrated design support environments. We then analyze in depth two DKC techniques: a semistructured hypertext tool and a model-based explanation system, and we explain their functions and how they work.
2
1. Design Rationales Throughout the life cycle of an artifact, from early conceptual design through diagnosis and repair to redesign, people need information about the artifact’s design. They commonly ask questions about why an artifact is designed as it is, such as • What is the purpose of a piece of artifact structure? How does it work? • What is its behavior? • Why was it designed this way? What were the requirements and assumptions? • What other alternatives were considered? The first two questions ask about the artifact — what is it for, how does it work, and how does its form relate to function? The second pair of questions ask about the design process — how and why did the designers come to the decisions resulting in the design? Design rationales answer questions such as these. Design rationales are explanations. They explain the relationships between the structure, behavior, and function of artifacts, such as how structure implements function, or how behavior emerges from structure. They also explain the decision-making process. This second kind of rationale is based on the history of design decisions, describing how alternatives were considered and evaluated, and how decisions were made over time. Designers create these two kinds of design rationales, using them frequently throughout the design process. [Goel & Pirolli, 1989] In practice, design rationales serve as an externalization of a design (e.g., a sketch [Ullman, Wood, Craig, 1989]) for a designer to use when considering its implications, and to communicate design information to another person. In response to specific questions, design rationales seem to fulfill Grice's maxims for conversation [Grice,1975] — they are meant to answer a question with respect to a given problem-solving context clearly and concisely. As a consequence, rationales are typically not absolute, context-free statements about an artifact. Instead, they answer specific questions, and are based on a great deal of knowledge about the design, the domain, and the design problem-solving context. Difficulty of capturing design rationale Collecting design knowledge in the form of design rationales is difficult. When rationales are generated by designers, they are usually created with reference (and limited scope) to the problem(s) at hand. They are typically very incomplete, falling short of the information required to replicate the decisions made. Rationales offered during designing protocols do not begin to cover the full range of decisions actually made, nor do they elucidate all of the factors that must be considered to actually solve a design problem. 3
Designs typically proceed without explicit knowledge of all the design variables or options available at any single step. Much of the knowledge about design practice is held inarticulately and implicitly by practitioners [Brown, 1990]. Furthermore, when designers create artifacts, they do not solve problems in a neat, linear, top-down style. Instead, they seem to work all over the design, making decisions about aspects of the design that might have little obvious relationship to the overt topic at hand. In practice, low-level structural decisions are intermixed with high-level abstract decisions about function and purpose [Guindon,1988]. Reconstructing a design rationale from an audit trail of design activity seems difficult. However, it is unrealistic to expect the designer to rationalize a design during the design process without automated assistance, since the creation of good rationales is a difficult cognitive task. We know that it is simple to derail the problem solving process by extraneous activity. We would prefer to gather design rationale knowledge in as light-weight and simple a way as possible. The trend in current engineering design tools is to reduce the cognitive overhead of the tools on the designer, freeing up the designer to work on direct problem solving, rather than worrying about tool use. (A wish dearly expressed by [Hewson,1990] for typographic design tools and [Ullman Wood, Craig, 1989] for sketching to support mechanical engineering.) Uses of design rationale Despite the difficulties in collecting the knowledge needed to create them, when rationales are available they open up an entire range of possibilities in supporting design activities. Here are a few examples. Checking work: Since a rationale explains portions of a design, verifying a design becomes far easier. [Conklin, 1989] reports that in their use of an explicitly rationalized design, several serious errors were found that would have otherwise have probably gone undetected until much later in the design process. Clarification of design arguments: Sometimes the overhead of stating a rationale for a design decision can be worth it. Very often, being explicit can force design issues into concrete terms that support problem solving [Russell, 1990]. Argumentation systems [Streitz,1989] [Newman, Marshall,1990] rely on explicitly constructed arguments to improve the quality of the designed artifact (in these cases, a text and a legal argument, respectively). Information sharing: In a complex design task, making information about the design and the design process available is difficult. Rationales explaining design process steps and design history can ameliorate some of the ineffective wandering observed in design problem solving [Guindon, 1988]. For
4
example, a design history can avoid revisiting previously explored alternatives or redefining decision criteria redundantly. Standards committee have used design histories to help track the decision-making process (e.g., the Common Lisp committee [Steele, 1990]). Institutional memory: In large engineering organizations, hundreds of designers work simultaneously over long periods of time. They have to share models of the common artifacts, communicate with each other across group and discipline boundaries, and coordinate with downstream activities such as manufacturing and testing. Capturing the knowledge underlying design rationales in explicit, machine-readable form is important to the accumulation and sharing of expertise within the institution. Requirements for design support systems A system for supporting the creation and use of design rationale clearly would be beneficial to designers. Because it is difficult to gather design rationale knowledge, the challenge for design problem-solving support is to understand how to create design rationales from underlying design knowledge, rather than simply to record user rationales. We need to provide computational support for design knowledge capture and use, rather than simply providing slots for attaching rationales to artifact descriptions. Central to the use of design rationale is the user’s ability to ask questions which do not have pre-stored or pre-computed answers. It is fairly difficult (if not impossible) to predict all of the design questions that may be asked. Yet, this is what a design rationale system must provide: the ability to answer a wide range of questions about the design, perhaps especially those questions that have not been anticipated in advance. It is our belief that this capability rests on capturing design knowledge, rather than design rationale, per se. Design tools can help with this knowledge acquisition, and with using design knowledge to create design rationales. But in order to determine what to capture and what to do with it, we need a representation that can support the knowledge that underlies design rationales.
2.
A Descriptive Theory of Design Knowledge
In this section we offer an ontology for representing design knowledge: a coherent set of terms, explicitly defined and related among themselves. The ontology embodies a conceptualization of the processes, products, and uses of design. Our intent is not to describe how humans think when they design, but rather to lay out a representation that can be used to capture publicly accessible knowledge about design in a useful, reusable form. (Our bias is toward a machine-intelligible form—a knowledge representation.) A representation of design knowledge is a language that lets us — and our programs — capture (record) and reuse information that is generated and 5
needed by designers during design. Such a design knowledge representation needs to be grounded in a theory of design problem solving. Studies such as [McGinnis and Ullman, 1988] and [Goel & Pirolli, 1989] tell us about the kinds of information relevant to the human designer. Sources such as [Mostow, 1985], [Brown, 1982], and [Brown & Chandrasekaran,1987] tell us about constructs needed to automate design. Our analysis of the uses of design rationale tells us what such a representation is for—to capture and reuse design knowledge. To capture and reuse design knowledge we must be able to account for concepts such as: • physical and logical structure of artifacts • behavior of artifacts • function and purpose of artifacts • validation, justification • design requirements and specifications • design decisions, alternatives • issues, arguments, criteria These terms are overlapping and ambiguous in common usage, yet critical to discussions of design rationale. That is why we lay out a conceptualization of designs, framing a coherent ontology, and covering each of these concepts. Let us begin by defining some more basic terms. Design description: a description of a designed artifact, a specification of what the artifact is or should be. 1 A design description may include the information sufficient for realizing (implementing, constructing) the artifact, as well as requirement specifications. In some cases (e.g. art, software prototypes, some kinds of plans), the design description is the artifact. Designing: the process of creating and editing design descriptions. “Editing” includes adding, augmenting, deleting, modifying, refining, transforming, etc. Designing is an observable behavior, the result of cognitive activity. Design activities: what designers do (including cognitive activity) that affects the evolving design descriptions. We will list some specific kinds of design activities later, but we will not attempt to represent design activities except via the notion of design moves. Design move: an action in the process of designing reflected in a change in the design description. Specifically, design moves are 1Although we may use “design” interchangably with “design description,” we intentionally avoid
definitions such as “a design is a set of decisions” or “a design is an intention” (i.e., unobservable entity).
6
changes to structure, behavior, or function descriptions (including requirements), changes to design criteria, explanations of analyses, and justifications for decisions.
2.1
Artifact Descriptions
Designing is a process of making and changing artifact descriptions. We will see that design rationale is a special kind of description. To be clear about the form and content of design rationale, we first discuss what design descriptions describe. From engineering we have the concepts of structure, behavior and function. The structure of an engineered artifact is usually given in terms of parts composed into wholes: connections of electrical components, assemblies of mechanical parts, organizations of software modules, and so forth. 2 The behavior of an engineered artifact is what it does. For example, the behavior of electrical devices is typically described in terms of signals, voltages, currents, etc.; the behavior of mechanical devices includes position (motion), forces, temperatures, etc; the behavior of software and digital circuits include input/output mappings. The notion of function can be problematic. For our purposes, we will talk about intended function as a prescription of behavior, such as the input/output specification of a program (which in simple cases can be described as a mathematical function, but is not the same thing). 3 In sum, artifact descriptions are of three types: Structure description: (notation: S) a description of the physical and/or logical composition of an artifact, typically in terms of the composition of parts. A structure description is about the thing itself and not what it does. Usually a structural description is the basis for realizing the artifact. Behavior description: (notation: B) a description of something an artifact might do, in terms of observable states or changes. If observable states are specified in terms of the values of physical quantities Q 1...Qn , then behavior descriptions are relations over these quantities, where Qi’s may include initial conditions and time. Intended function description: (notation F) a description of a subset of possible behavior that is supposed to be achieved by an artifact. F can be described as a subset of the B space Q1 × Q2 ... × Qn. Intended function is a constraint on behavior. The intended function description is not expected to represent the “real intent” of humans, but is, rather, an operationalized specification.
2 “Structure”
includes design parameters such as color, size, materials, etc, not just components and connections. 3 Intended function descriptions are one kind of design requirement , which is defined in the next section.
7
n
tio
Structure
a lid va
jus tifi ca tio n
Function
prediction
Behavior
Figure 1: Types of artifact descriptions. Structure descriptions specify the constituents of an artifact and relations among them, typically physical or logical components and connections. Behavior descriptions represent possible states of an artifact and how they change. Function descriptions are constraints on behavior; they are one type of requirements. In addition, mappings between structure, behavior, and function—such as justification, validation, and prediction—are pertinent artifact descriptions. These mappings have an explanation-like quality and are typically produced by inference.
These three types of artifact descriptions form the vertices of the triangle shown in figure 1, with three directed arcs of interest. The mapping from structure to behavior is probably most familiar: the prediction of artifact behavior from its structure. Behavior prediction is a routine part of design, particularly in later stages. Doing the analysis or simulation requires one to construct a model, perform some computation on it, and interpret the results. The result of analysis or simulation is an inferential path from descriptions in structure space to descriptions in behavior space. Just as a path of deductive inference can be presented as a proof, the result of analysis or simulation forms an argument structure, showing how behavior arises from the interactions among structural constituents. One can view this mapping as an explanation of how the artifact works.4 Two other kinds of inference / explanation are found on the arcs of figure 1. A design justification maps structures onto functions; it is an inference that shows the functional purpose or role of structure. A design is justified if all its designed structures contribute to functional requirements. We view 4The term “explanation” has several senses that are so closely related that we deliberately blur the
distinctions in this paper. One sense is the presentation medium: An explanation of how something works may be presented in the form of pictures with circles and arrows and a paragraph on the back of each one; explanation presentation is an area of work in AI and HCI [explanation workshop]. Another sense is explanation as inference (abduction and related types of reasoning), which is subject matter for philosophical analysis [Hempel, 1965] and automated reasoning [abduction workshop]. A third sense is explanation as a rhetorical entity, an argument constructed to communicate something; these are the stuff of books such as The Way Things Work [Macaulay, 1989]. In knowledge-based systems that interact with people to acquire and generate explanations, all three senses are relevant and related (e.g., a generated English explanation usually corresponds to the internal path of inference, which is guided by some abductive goal.)
8
justifications as explanations of how structure achieves function. And as we shall see, design justification is an important kind of design rationale. Finally, determining whether given set of behaviors are consistent with the prescribed behavior—the intended function—is called validation. Validation is an explanation of how a specific behavior, such as that predicted by simulation or analysis, satisfies some behavioral specifications. Behavioral prediction: (notation: S → B) a description of the behavior enabled by (a part of) artifact structure, typically produced by simulation or analysis using behavior models. Formally, a prediction is a mapping from S to B. In general, all behavior descriptions are inferred; prediction is one method. Design justification: (notation: S → F) an explanation of the intended function achieved by (a piece of) artifact structure, explaining its “purpose”, “functional role”, etc. Formally, justification is a mapping from S descriptions onto F descriptions. Functional validation: (notation B → F) a proof or argument that a set of behavior achieves the intended function description. Formally, a validation is a mapping from B to F. Since descriptions in F and B are both behavioral descriptions, validation is an explanation of how some specific B description—such as that predicted by a simulation—is a subsumed by a F description.
In sum, the important properties of these mappings are: • Mappings among S, B, and F are usually inferred as part of design activities, either by people or by their tools. • The mappings are analogous to proofs or arguments, leaving behind a justification structure (i.e., derivations or proof steps). • The mappings naturally correspond to explanations, what the artifact will do, how its structural constituents contribute to its functions, and whether the predicted behavior achieves the prescribed behavior. (As we shall see, these mappings are one source of design rationales.) 2.2
Requirements
Design requirements are specifications that the designed artifact must satisfy. We do not distinguish requirements as a distinct kind of knowledge about artifacts. Instead, we characterize requirements as prescriptions over artifact descriptions—structural and functional—and other specifications used in decision making—namely, design criteria and objective functions. Thus, we identify three kinds of requirements: • Functional requirements: constraints on B — that is, F descriptions. These described what the artifact is supposed to do, couched in terms of behavior (not teleology). • Structural requirements: constraints on S, such as those imposed by safety codes or customer requests. 9
• Design criteria and objective (utility) functions — to be defined below. 2.3
Decision Making
Although artifact descriptions are the product of design activity, changes to artifact descriptions do not represent the design process itself. One can conceptualize design activity as a process of decision making. In this view, design activity consists of generating alternatives and deciding among them. To formalize this notion we need to define what it means to be an alternative and identify parameters that characterize a design decision. We take design alternatives to be versions of artifact descriptions, specifically S or F descriptions. Alternatives may be “complete” descriptions, or views onto the descriptions that focus on a part of the description and/or present abstractions of detail. There are two subtypes of alternative descriptions. Structural alternatives are proposed structural designs, which may be generated by creating or modifying S descriptions. For example, adding a component creates a new version of the structural description (i.e., a version including the proposed component), and an alternative could describe the new component in its structural context. Of course, alternatives can be viewed as “stand alone” descriptions if the “previous version” is empty.5 Functional alternatives are proposed functional requirements, which are also created and refined during design. Design alternative: a proposed version of an artifact description, possibly filtered or abstracted to show a specific fragment of the description. Structural alternative: a proposed version of (part of) an S description. Functional alternative: a proposed version of (part of) an F description.
A design decision is a choice among design alternatives. Utility theory offers an ontology of basic concepts for representing decisions, and a normative basis for inference. The basic concepts are alternatives (outcomes), criteria, and the objective (evaluation) function. Criteria are features of alternatives that are relevant to the decision, that is, that contribute information to an evaluation of the utility of alternative. Some example design criteria are “manufacturability,” “keep the weight under 10 pounds” and “minimize cost up to some upper limit, at which point no alternative is acceptable.” The objective function provides for alternatives a measure of utility in terms of their criteria. 6 The concepts are straightforward; much of
5Representing alternatives as versions of design descriptions does not commit one to a theory of how
alternatives are generated (e.g., by modifying existing designs or synthesizing from scratch). This representation is not committed to design by applying operators to search a design space. In fact, we believe this is an inappropriate model. 6The evaluation function can be further parameterized by objectives , functions over criteria that measure the utility of alternatives, and a combining function, which determines total utility given the utility
10
the machinery of decision theory is concerned with how to combine the information provided by multiple criteria and objectives into a single utility value. For example, a common objective function takes the form of a weighted sum of criteria; the criteria functions and weights represent knowledge about what is valuable in a design. Design decision: a choice among design alternatives. A description of a design decision includes the set of alternatives, relevant criteria, and salient properties of the objective function. Design criterion: a feature of a design alternative that can be used to evaluate the utility of the alternative and make decisions. A criterion can be formalized as a function of an alternative, e.g., C(a), where a ∈ S. The satisfaction of functional requirements can be represented as hard or soft constraints that serve as design criteria. Objective function: a function that evaluates the utility of alternatives with respect to some set of criteria. An objective function can be formalized as U(a) = f(C1(a), C2(a), ...), where Ci are criteria and a is an alternative. 7
Justifications A justification of a decision is an explanation of the alternatives and criteria considered, and why the chosen alternative is preferred. Justifications are part of the design description, since they represent what is relevant and valuable about an artifact. The criteria considered and objective function used in a particular decision capture important knowledge about the design. Although the formalism of utility theory makes it possible to state the objective function and criteria as mathematical functions, such information is rarely available in such a precise and complete form. Informal arguments in favor or against an alternative are more common. If the task is to justify rather than automate the decision, or to ensure that it is normative then decisions can be explained without specifying an operational objective function. In particular, one can represent partial, symbolic forms of the objective function. Nonnormative decision justifications can be organized by argument structures. For example, Toulmin argumentation concepts can be used as justifications for decisions that are formulated in decision-theoretic language, essentially replacing the objective function with an argument [Lee, 1990]. The identification of relevant and irrelevant criteria in the context of a decision is valuable knowledge even without an objective function that evaluates and combines them. values returned by objectives. For instance, this formulation would call “cost” and “weight” criteria and “minimize the cost/weight ratio” an objective. For simplicity, we fold objectives into criteria. 7It is also reasonable to define preference functions that compare two alternatives at a time, rather than returning a single absolute utility estimate. The utility of a design alternative can also be formulated as a second order function that is parameterized by a set of criteria Ci , taking them as inputs in addition to the alternative a: that is, U(a) = f(C1 , C 2 , ... C n, a). A second order formulation allows the use of knowledge about criteria, such as their conditional independence, relative weights, etc.
11
Justifications of this form have been used as a technique for automated knowledge acquisition; domain experts list “relevant features” of alternatives in specific problem solving situations, and the knowledge acquisition tool uses them to form pattern matching clauses in operational rules [Gruber, 1989]. Decision justification: an explanation of a choice among alternatives in terms of the utility of those alternatives with respect to a set of criteria. A normative decision can be explained in terms of utility theory. Informal explanations, such as lists of alternatives and criteria considered, can also serve as justifications for design decisions.
12
Decision
O
C
O
C
C
F
C
S
C C
C
C
F
C
B
O
S
C C
B
C
F
C
S
O
C
C C
B
C
C
F
C
S
C
B
Alternatives U utility
assessment objective function
O
C C
S
C
F
design criteria
C
decision justification
C
B
artifact descriptions
Figure 2: The structure of a design decision. A design decision is a choice among alternatives, which are versions of artifact descriptions. The utility of each alternative is evaluated by an objective function that combines the values of design criteria. Criteria are functions over artifact descriptions; some criteria are based on functional or structural requirements. A decision justification is the explanation of why the alternative is (most) useful; the structure of the explanation includes mappings from structures and functions to criteria, and combinations of the values of criteria.
2.4
Unifying the artifact-description and decision-making views of design
Both artifact descriptions (S, B, F descriptions and the mappings) and decisions (alternatives, criteria, objective functions) represent useful design knowledge. As they have been formalized here, there are strong relationships
13
between the concepts from the artifact-description and the decision-making perspectives. To wit: • Alternatives are artifact descriptions in S or F. Versions of artifact descriptions represent alternatives to decide among. Functional requirements as well as structure can be evaluated as alternatives. • Functional requirements — descriptions in F — can be made into criteria with a satisfaction function. C i(a) = satisfaction(F i(a)), where C i is a design criterion, F i is an intended function, and satisfaction is a function that measures the degree to which structural alternative a∈S causes behavior that satisfies the the behavior prescribed by Fi . Note that the operationalization of satisfaction may require S → B and B → F reasoning. • The equation Ci (a) = satisfaction(F i(a)) does not imply a design strategy. Design moves may describe changes in structural alternatives (the a), functional requirements (the F i), or criteria that are deemed relevant (the Ci ). Indeed, a designer may simultaneously explore artifact description and decision spaces. • There are some criteria that do not correspond to functions, such as “make it manufacturable”, since they do not refer to artifact behavior per se. However, these criteria do operate on artifact descriptions. The satisfaction function is an important special case. Justification Revisited Just as structure and function descriptions are related by explanation-like mappings, alternatives and criteria are related by argument-like objective functions, as in figure 2. Whether an objective function is represented with a numeric criteria/alternative matrix or a decision-tree-like enumeration of choices and outcomes, the underlying knowledge can be interpreted as an argument or justification for the decision. When the criteria are based on the satisfaction of intended function, the corresponding explanation is a functional justification. A functional justification may be constructed by composing a behavioral prediction and a functional validation: S → B, B → F. One structural alternative might be compared with another on how well they implement the same intended functions. For example, a functional justification for using lead-acid batteries over NiCd batteries (S) might compare their charge and discharge rates (B) against the requirements of the application (F), finding that the NiCd batteries discharge too quickly. Functional justification: an explanation of a choice among structural alternatives in terms of how well the functional requirements are achieved.
14
2.5
Design History and Design Rationale
Design history is a record of what happens during design. Design rationale is usually defined as a representation of the “reasoning behind (underlying) an artifact,” including the design’s history of changes. Unless one builds an AI program that can automate design (and can deduce the designer’s intent from the design history), it is rather tricky to capture the designer’s reasoning. What is sometimes offered under the name of design rationale is a history of what designers do during design, or what they say about the decisions they make. This paper presents the position that design activity is reflected in the creation and editing of design descriptions. Thus, design history is what designers “do” and “say” while they design, reflected in the observable changes they make to design descriptions: Design history: a recording of design moves in a reusable form. Types of design moves include: • proposing structural alternatives (S descriptions) • proposing modifications to requirements (F descriptions or design criteria) • evaluating alternatives. In the process of evaluating alternatives, one can create a decision justification, mentioning criteria and objectives, or a functional justification (a S → F mapping), possibly via simulation or analysis of behavior. • changing previously determined portions of the artifact description or its justification (e.g., adding new criteria, altering an artifact description)
Can design history alone be a sufficient representation of design rationale? Not necessarily, because the rationale is used for information access and communication, and the sequence of design moves may not provide a helpful presentation of the information. What makes a design history reusable may be its organization: the structure of the argument is made clear from the structure of the history. Now we have the foundation for defining design rationale. Design rationale: an explanation of how or why an artifact (or some part of it) is designed the way it is. A rationale is an answer generated in response to a query grounded in an artifact description. It is constructed from information recorded in a design history.
Following are some classes of design rationale, characterized by generic queries and the information source used to answer them. • What is the function of this piece? (design justification, S → F) • What does this part do? (behavior prediction, S → B) • Why was this part chosen instead of that one? (decision justification) • Why was this part chosen to implement that function? (functional justification in the context of a decision justification) 15
• What is the intended context of use of the artifact? (based on either or both a behavioral prediction S → B and a functional validation B → F). One cannot explain how behavior arises from structure without making assumptions about operating conditions. Prescribing behavior (F) also flushes out intended contexts of use. • What criteria were considered in choosing this part? (decision justification) • What decisions does a change in this functional requirement impact? (functional justification) Thus, design rationale explanations are based on all of the kinds of descriptions created and modified during design, and not just on explicitly articulated arguments about the design. In the next section, we analyze approaches for capturing the design knowledge underlying design rationale.
3. Design Knowledge Capture in Design Support Environments In this section we use the design knowledge ontology presented in the previous section to characterize the roles of design support tools, and in particular, different approaches to design knowledge capture. 3.1
Conventional design support tools: Support for artifact descriptions and decision (tradeoff) analysis
Design support tools help designers create and modify artifact descriptions. Artifact description editors such as familiar CAD drawing tools allow the user to manipulate S descriptions. Artifact description editors provide operations for creating, viewing, and maintaining descriptions of objects and relationships among them. Analysis and simulation tools help designers construct S → B mappings, giving information about the behavioral implications of a design. Validation tools construct S → F mappings, verifying that functional requirements are achieved by a proposed structure. Design critics and constraint checkers make similar arguments, verifying that an S description satisfies design criteria (or evaluating the degree to which criteria are satisfied, which is equivalent to what objective functions do in the decision-theoretic paradigm). In general, design support software provide two classes of computational services to support design activities. The first is memory and communication: machine-readable design descriptions can be stored and transmitted efficiently and completely. That is the primary contribution of conventional CAD drawing tools. (The fact that these tools are so widely used attests that description making is a large component of the work in design.) The second kind of service provided by software tools is feedback on the implications of a design (as modelled). This is what analysis and simulation are all about;
16
running formal models of artifacts gives the designer data on the behavioral consequences of structural choices. Conventional decision support tools also help create descriptions— justifications of decisions. For example, it is common in large-scale engineering projects for designers to perform “tradeoff studies”— explicit enumeration of design alternatives and an analysis of the tradeoffs among design criteria. Decision support tools can be used to construct tradeoff studies [Shema,1989], [Lee,1990]. They augment human communication and memory capabilities by eliciting and recording alternatives and criteria. By imposing a normative decision-making paradigm (e.g., expected utility), these tools can compute the utilities of decision alternatives implied by (consistent with) the value model given by the decision maker. Conventional design support tools are not adequate to support design rationale, because they do not capture enough of the design knowledge that underlies design rationale. First, they only capture part of the relevant artifact descriptions. For instance, a structure description captured with a CAD drawing program can give detailed information on the geometry of a physical device, but is not sufficient to answer queries about the intended function of the device. Second, the descriptions are isolated at tool boundaries, which hinders the construction of rationale explanations. For example, most simulators do not operate on the S descriptions created by CAD systems; instead, they take mathematical models constructed by the user. Another example is version management and decision support: the structural alternatives represented by versions of a design are not linked to decision support models of those alternatives. 3.2
Tools for design knowledge capture and use
Design knowledge capture tools also help designers create, modify, and maintain design descriptions. Some operate on artifact descriptions and some on design history. The primary requirement of design knowledge capture tools is that they capture design descriptions in a form that supports the communication and reuse of design knowledge. In particular, DKC tools should operate on representations that are useful for constructing design rationale explanations. Many DKC tools can be seen as natural extensions of traditional artifact description editors. Electronic designer’s notebooks [Hwang & Ullman, 1990; Lakin et al, 1989] allow the creating and editing of S and F descriptions at the early, conceptual stage of design, where details required by traditional CAD tools are not available. Issue- and argumentation-structured hypertext tools [Conklin & Begeman, 1988; Fischer, McCall, & Morch, 1989a; Marshall, 1987] represent design moves as decisions and help elicit documentation that informally justifies the decisions by presenting the designer with a semi-structured form to complete. Much of this style of design rationale research is concerned
17
with the structure of the justification (what constitutes an alternative is left to the user) and the organization of the decisions (chronologically, tracking the design history, or logically by goal hierarchies). Some tools record design moves in the artifact description space directly, organizing the design history by the structural alternatives considered at each juncture [Sommerville et al., 1989; Mark & Schlossberg, 1990]. Decision support tools have been customized for design decisions, where decision alternatives and criteria are explicitly associated with design alternatives (in S space) and criteria are associated with design requirements. Such tools can help users explore the space of alternatives defined by a set of design criteria and structural options [Shema, et al. 1989; Pugh,1981; Lee, 90]. Some DKC tools integrate artifact description editors with other design support tools. The JANUS system [Fischer, McCall & Morch, 1989b] integrates a CAD-like editor with a rule-based design critic and an argumentation-structured hypertext documentation environment. Note that the three tools are integrated via design descriptions: S descriptions are syntactically linked to S → F and requirements-checking constraints, and textbased justifications of decisions are linked to the same structures (decision alternatives = S descriptions). Design (case) memory systems [Mark & Schlossberg, 1990; Delisle & Schwartz, 1986] provide access to institutional design experience by intelligently indexing versions of related designs (S descriptions), where the indexes are structured justifications mentioning functional requirements and other design criteria. Model-based DKC tools treat design knowledge capture as a knowledge acquisition problem, where the task is to acquire operational device models that can be used to infer S → B and B → F explanations. They emphasize explanation generation capability rather than rationale replay. Model formulation assistance [Abelson et al., 1989; Iwasaki et al., 1989] helps engineers build mathematical models of physical devices by applying libraries of ways to model the behavior of standard components and taskspecific knowledge about modelling strategy and the use of behavior models in analysis and simulation. Once models have been constructed, model-based simulation, analysis, and verification tools can infer S → B and B → F mappings. Integrated explanation systems can then generate design rationale that explain the purpose (F) of a piece of structure (S) [Baudin, 1989; Gruber, 1990; Kellog, Mark, and Sullivan, 1988]. Explanations can also be used as human-computer interaction media for iterative knowledge acquisition [Bareiss, 1989; Gruber, 1989; Mark, 1988]. In the case of acquiring design knowledge, the user selects from or configures machine-generated explanations that correspond to rationales, telling the machine which of many possible rationale are salient in a given situation [Kellog, Mark, and Sullivan, 1988; Gruber, 1990]. We describe an example of this approach to DKC in section 4.3.
18
4. Analysis of two approaches to design knowledge capture In this section we analyze examples of two design knowledge capture techniques: a hypermedia tool incorporating semiformal models of the domain and design of artifact, and a model-based explanation approach. The hypermedia example is IDE [Russell, 1988], a system to support the design and creation of instructional materials. IDE represents a growing class of tools for human communication and cooperation that emphasize the acquisition of and access to knowledge primarily for human consumption [Conklin & Begeman, 1988]; MacLean, Young & Moran, 1989; Malone, 1986; Marshall, 1987]. IDE has been used for several years by instructional designers in several different kinds of settings (e.g., technical industrial training, graduate and undergraduate university education, high school mathematics, vocational education). We explain how an early version of the system (IDE 1.5) was used for design knowledge capture with great difficulty, and how a newer version (IDE 2.0) based on our theory of design knowledge representation facilitates design rationale use. This is illustrated with a pair of examples between the older and newer versions of IDE. The second example is a model-based approach to design knowledge capture, representing what can be done with knowledge-based simulation systems that can explain how behavior arises from structure (cf. [Forbus & Falkenhainer, 1990]). Using an example from the device modeling environment DME [Gruber & Iwasaki , 1990], we describe a scenario for a new kind of design rationale capture based on comprehensive models of the devices being designed. In this approach, the user demonstrates the function or purpose of a piece of structure by setting up a behavior scenario in a simulation environment. Given restrictions on the relevant structure, behavior, and context of use of a device, and a model sufficient for generating a simulation, the system can generate an an executable demonstration of how a specified intended function is achieved. The simulation scenario can be used to generate explanations in natural language. We show how the modelbased explanation technique can support interactive documentation of the design rationale, in which users can ask a restricted set of questions about the rationale rather than just read what has been written by designers. We also discuss the practical limitations of this approach. 4.1
Attached Rationales in IDE 1.5
In IDE 1.5, (built and used 1987-88), we created a design rationalization system designers could use to express why a course of instruction was constructed in a particular way. [Russell, 1988]
19
Figure 3: In the IDE 1.5 design rationalization room, instructional designers could create arguments that would explain a body of instruction. The rationale was expressed as arguments explaining why design decisions were made with respect to an established body of design principles.
Our goal with IDE 1.5 was to facilitate courseware construction and instructional domain analysis (e.g. task or needs analysis). Hence, we built a combined CAD / knowledge representation and acquisition system for instructional designers. Since part of our goal was to improve design re-use and re-engineering, capturing design rationales became an important objective. How IDE 1.5 design rationale creation worked In IDE 1.5, a course was represented by a hypermedia structure that linked together multimedia presentations and tests into a complex, usually nonlinear, sequence.
20
Figure 4: IDE 1.5's default course structure is a nonlinear sequence of multimedia presentations similar to a guided tour. To create courseware artifacts, an instructional designer creates this graph structure, annotating each presentation node with information describing how, when, and where each activity occurs.
To rationalize an IDE 1.5 course, the designer had to suspend work on the course, move into the rationalization room, and then create an argument structure that would document the underlying principles, the decision made, the argument used to reach that decision. The rationale was captured by having the user create the argument and explicitly attach it to the structure (in the artifact) and to the representation of the decision that lead to that structure's design. We call this style of design rationale representation attached rationale, since the user must explicitly create each design rationale argument, and then attach it to the artifact description. (This is the design rationale representation style used by [McLean, Young & Moran,1989; Conklin & Begeman, 1988] and others.) Thus, each time the user did a significant amount of work on the artifact, it was necessary to stop and document it. The effect was rather like doing unpleasant documentation of a large software program without any clear notion of the community standards for such documentation.
21
Evidence 1 Evidence 2
R:
Argument
Decision
o o o
Evidence 3
pointers into affected artifact structures
Figure 5: IDE 1.5's rationale representation ties the artifact description to an argument structure documenting the reasoning behind the decision. “Evidence” is primary data acting as backing assumptions; “arguments” (or counterarguments) present a case (as text) for (or against) the “decision,” which is linked into the artifact description.
What went wrong with IDE 1.5's design rationale mechanism? We noted several problems in our use of IDE 1.5 design rationales. Foremost was that the rationale creation was separated from design practice. The user had to move from the design room (or mode8 ) into a rationalization room each time a new design argument was constructed. Although IDE 1.5 used multiple, sharable workspaces (the Rooms system [Card & Henderson, 1987]), there was still a large overhead in moving from actual designing (i.e., writing materials down) to rationalizing the artifact just created. Improved user interfaces can help reduce this problem significantly, but can't make it go away. Secondly, the rationale language was too disconnected from the domain of discourse of the designers. (For details of the rationale language, see [Russell, 1988].) The rationale representation in IDE 1.5 had three types of decisions: Epistemology Decisions, Cognitive Objectives, and Presentations and Test Decisions. The structure of the rationale required that the user create arguments to explain each decision made in terms of underlying design principles (which came in several types as well). In our real-world tests we found that this representation had little (if any) connection with the actual decisions made by course designers. Instructional authors spend more time with day-to-day tradeoffs such as "how much time is required to read through a section of text" than in making Epistemology Decisions. An important user perception problem was that design rationale creation had no obvious benefit to the users. The benefit was too far in the future. We had planned on using the rationale for course re-design and re-use. And 8 In IDE, a "mode" is a particular view onto the design database associated with a given design task. In some versions of IDE 1.5, each mode corresponded to a different "room." The argument holds in both cases: rooms or modes, the user had to switch contexts dramatically in order to create a fragment of design rationale.
22
although we had one experience with courseware re-design that worked out well 9 , we didn't continue the use of IDE 1.5 for long enough—for the years necessary—to see how well the rationale would have worked as an aid for reengineering. Finally, creating design rationales is very revealing. We found that, on occasion, users are reluctant to write down "I don't know" or "historical precedent" as the argument behind a design decision. In requiring a complete rationale argument for decisions, the rationale quickly became an onerous task that exposed inadequacies and uncertainties. While complete rationale elicitation is difficult for daily production use, it is tremendously useful for educational settings. (See [Russell, 90] for a use of this property as a tool to teach reflective instructional design.) Summary: IDE 1.5 was a successful design environment for creating instructional materials. However, the design rationale component of IDE 1.5 suffered from a disconnection with the users and with their daily work practices. Building rationalizations in this explicit fashion can be done. We did it and they were useful. However, it requires that the users be extraordinarily disciplined and scrupulous in creating the rationale. Such annotations can be useful, particularly if they are the ultimate object of study,10 but this approach is difficult and doesn't lend itself to automatic analysis and design rationale rendering. 4.2
Semiformal Representations and Weak Models: Design Knowledge in IDE 2.0
During 1989, we began porting IDE from a Xerox Lisp machine environment onto the Macintosh to create IDE 2.0. With that shift, we also took the opportunity to re-think our design knowledge representation for IDE . Representing design knowledge in IDE 2.0 In our domain of instructional design, we came to realize that many of the courseware systems we were designing have a simple artifact structure. For these artifacts, a small set of design rules describe common practice within this style.
9 In this case, a videotape was created using IDE 1.5. One year later, a second, much shorter videotape was created from the original tape by re-editing the material as recorded in the original IDE representations. By re-using the IDE rationalizations, recovering the organization of the original tape was simplified, and the time to create the second tape was reduced significantly. 10 For example, there is one on-going project with IDE 1.6 that is rationalizing a successful multimedia foreign language course in the "attached rationale argument" style. This project is reverse-engineering the courseware, and then analyze the rationale. The goal is to look for a coherent set of design principles describing the courseware design that may be reapplied to guide new courseware authoring for multimedia foreign language instruction.
23
As an example, consider the "Minimalist/Example-based" instructional design paradigm illustrated in figure 6 [Pirolli & Russell, 90]. At the risk of over-simplification, the task of the instructional designer is to determine what must be taught (in terms of tasks, skills, concepts, etc.) and to map those ideas into a courseware structure. A Minimalist/Example-based artifact is designed in IDE 2.0 by creating a structure that explicitly represents each of the modules, its constituents, the problem and domain spaces (and all of their substructure, which has been suppressed here for clarity).
24
Artifact Description
Domain Model
Example Activity Questions
Problem Space
Element1 Example Activity
Module
Questions
Domain Space
Element2
Artifact Design Rules (S-> F) (F -> B)
Figure 6. The Minimalist/Example courseware structure representation. A course artifact is a sequence of example, activity, questions triples with links to concepts in the domain and problem analysis spaces. A set of 13 rules describe the relationships between concepts introduced, where and how they are used, and to what level of detail concept use or reference should be made.
25
Figure 7. A small piece of a Minimalist/Example-based course represented in IDE 2.0. Each module (“M” node) has a set of elements (“E”), each of which comprises an Activity, an Example and a Question. Here a browser (above) shows a small introductory module, with the element node expanded below. Each slot shown points to a card with additional information about this part of the course.
A course designer using IDE 2.0 instantiates the structures of figure 6 to create a piece of courseware. Design knowledge in this representation is partially captured by a set of design rules that detail how, when, and where a concept can be introduced and used in the course design. These rules are explicitly encoded in IDE 2.0 as search mechanisms written in IDE's underlying query language. Thus, an entire class of design rationale explanations are available to the designer by simply asking. The IDE user can ask for the behavioral rationale for a given piece of course structure by running the query: for example, "Explain the structure to function relationship of the ‘Row-Column’ section of this course." Rationalization queries are stored as cards in IDE, each card accepting a parameter (e.g., a pointer to a fragment of artifact description) and generating (on demand, in real-time) a brief explanation about the structure, function, or behavior in question.
26
In figure 7, the course is designed to teach Excel©, the spreadsheet system, through a sequence of examples, activities and questions. A structure-to-behavior rationale can be obtained for any structure in the course. In figure 8, we see the result of asking for the rationale of the example "select column." The explanation shown in the card is generated by the rationale query, which encodes an inference procedure for this representation. In essence, the procedure checks to see if the design guideline "refer to important concepts with increasingly abstract references" is followed. If so, then an explanation is created by composing templates and values into an explanatory response. In this IDE 2.0 course design, the notion of presentation abstraction is well-defined, allowing the query to operate over the course representation, constructing an explanation.
Figure 8. An explanation for a fragment of course structure is created in response to a query: “Explain the structure to function mapping of the ‘Row-Column selection’ element.” The answer is created semiformally, by composing templates with the values of card slots that are retrieved by standard relational queries.
The design rationales created by IDE 2.0 are driven by a weak model of the design process, which is primarily semi-formal in nature. In this example, the system reasons about the structure of the artifact, and provides an explanation in terms of its underlying function. In the artifact description
27
language of IDE 2.0, the artifact and its design knowledge are recorded as a part of ordinary work. The rationale for S → B mapping is contained within a piece of code that knows how to make this deduction from the artifact description; it must, by the nature of the artifact description in IDE, be able to handle semi-formal representations. 11 4.3 Model-based Explanations: Design rationale in DME One class of explanations are provided by simulations, which map descriptions of physical structure to descriptions of behavior. (S → B) When people explain how things work, they often employ informal simulations, describing what happens to what aspects of a system under what conditions over time. For example, to explain the purpose of training wheels on a child’s bike one might describe a scenario where the bike starts tipping too far off axis, but is kept from falling over by the training wheels, which transfer the tipping forces to the bicycle frame. A more complete rationale would specify the operating conditions under which the wheels are used, such as the forces applied (based on assumptions about the weight of the rider and the speed), and verify that the materials and structural support for the training wheels could withstand those forces. Engineers use similar explanations to justify and explain their designs. In engineering practice, simulation is used to predict the behavior of engineered artifacts. A simulation requires behavior models, typically in the form of equations, that describe the behavior of individual components and general processes; the simulation predicts the system behavior that arises from the interactions of components and the operating environment. The formulation of behavior models from structural descriptions is not automated; simulations are set up by engineers to answer some class of questions about artifact behavior. Using simulation to answer questions about the behavior of an artifact reveals knowledge about the designer’s intent. The specific models that are created and the input conditions that are specified describe a behavior of interest and intended context of use. The simulation serves as a demonstration of intended device behavior; it is an F description. One can employ a computer simulation as a communication medium for design rationale. The basic approach is to arrange for a knowledgeable user to set up simulation scenarios, configuring and guiding an simulation tool so that the predicted results account for the phenomenon to be communicated. In particular, in the technique called rationale by demonstration, the user
11
This is “semi-formal” because the explanation mechanism works by making standard relational queries over the artifact description database, and then knowing enough about the structure of the representation to incorporate fragments of data structures it can’t interpret. That is, the explanation is a composite of inferred information with pieces that are extracted from semi-structured representations. As [Malone, 1986] discovered, a surprising amount of useful work can be done in this style.
28
documents the intended function of a bit of artifact structure under given conditions. [Gruber, 1990] If a person can demonstrate a behavior by constructing an operational simulation scenario, then the machine has enough information to replay the scenario to another person. Instead of writing text describing a device, one engages in a dialog with a knowledge-based modeling and simulation environment, telling the machine what it needs to know to generate a demonstration of some behavior of interest. The user identifies the relevant structure and intended behavior by constraining the possibly relevant structure and behavior within the closed world defined by the modelling primitives. The machine generates explanations based on the outcomes of simulation. The user essentially manipulates the simulation scenario until what the machine says corresponds to the intended message. The result is a machineintelligible model of design rationale. Because the simulation is generated from underlying models, it can be used to answer questions not typically anticipated by static documentation, and can be effectively indexed with other knowledge about the device. Two extensions to conventional modelling and simulation technology are required to support rationale by demonstration: model formulation assistance and machine-generated explanation of simulation results. Model formulation assistance can be provided by an interactive, knowledge-based modelling environment. Consider a domain in which artifacts are designed from standard components, such as wires, switches, motors, and control circuits. A knowledge-based modeling environment for such a domain includes a library of modelling primitives with which to construct behavior models of the devices. For electrical circuits, the model library describes components such as voltage sources (emf seats), resistors (ideal loads), ideal switches, etc. The model library also contains descriptions of physical processes (or mechanisms), such as current flow, electrical-tothermal energy conversion, etc. To analyze the behavior of a designed artifact, the designer constructs a model by associating primitive components from the behavior modelling library with the constituents of the physical structure being designed. For example, in one scenario she might choose to model a real motor as an idealized resistor with constant resistance, and treat the actual wires as perfect conductors. Given a behavior model composed this way, a model formulation tool can generate the information required for a simulation engine, such as equations. Then the simulation tool can be invoked. Typically a simulation scenario requires more information than originally anticipated by the modeler, such as initial states of components, and desired ranges for behaviors of interest (which voltages to track and for how long). These parameters can be interactively elicited from the modeler, in the context of the scenario. Note that such an interactive modelling and simulation system provides feedback on the completeness of a device model with respect to a specific modelling purpose. Although such knowledge29
based CAD environments are not yet in common use (except for electrical circuits [Tuinenga, 1988]) several systems have been developed and commercial development is proceeding. The technology for machine-generated explanation derives from work in Qualitative Physics in AI [Bobrow, 1988; Weld & DeKleer, 1989]. Qualitative physics models describe qualitatively distinct regions of device behavior, rather than numeric behavior trajectories defined by the time scale of numeric simulation. For example, a qualitative simulation of water being heated in a boiler represents the temperature of the water in terms of whether it is below, at, or above the boiling point, and the direction in which the temperature is moving. Qualitative models also provide information that can be used to infer causal order among modelled quantities, whereas conventional mathematical modelling is acausal. For example, boiling may be viewed as a process by which the volume of liquid water decreases and the volume of steam increases. The notion of process is used to ascribe causality: the increase in stream pressure is caused by the boiling, which is caused by the heat source, etc. Using these two properties of qualitative models, an explanation based on qualitative simulation can mention abstract processes causing qualitative changes in the state of the modelled system. For example, Forbus and Falkenhainer [1990] have developed a tool called SIMGEN that can generate explanations such as those shown in figure 9. 12 The boiling temperature of WATER and the temperature of the water in CAN become equal, which leads to S1. In S1, heat is flowing from STOVE to the water in CAN and the water in CAN is boiling. The amount of steam in CAN becomes positive, which leads to S16. In S16, heat is flowing from STOVE to the water in CAN, heat is flowing from STOVE to the steam in CAN, and the water in CAN is boiling. The amount of water in CAN becomes zero, which leads to S4. In S4, heat is flowing from STOVE to the steam in CAN. The temperature of the steam in CAN and the temperature of STOVE become equal, which leads to S5. In S5, nothing is happening.
Figure 9: An explanation generated by SIMGEN from a qualitative model.
An Example of Rationale by Demonstration Consider the following fragment from a NASA document explaining the purpose of a piece of a circuit in the electrical power system of the Hubble Space Telescope: 12
A CAN with WATER is being heated on the STOVE. Qualitative states are labelled by S.
30
5.1.5 Voltage Limiter (1) (2) (3)
(4)
(5)
(6)
A voltage limiting circuit is provided in each diode bus feeder to limit the voltage at the using equipment to 32.0 V. With the SSM design change to six 23-cell, type 44 batteries the user input could exceed the 32 V user specification. To prevent this, an additional diode pair (parallel 100A diodes) was added in series with the load and bypassed by a relay that is operated by a voltage sensing circuit. When the diode bus voltage exceeds 34.15 ± 0.2 V the bypass relay opens and inserts the diode in the circuit as a voltage dropping element. When the voltage falls to 31.41 ± 0.2 V the relay will reclose, bypassing the diodes. The voltage sense circuit board used in this circuit is essentially the same as the circuit used in the charge current controller except for the set point voltage. Figure 5-1 shows one of the voltage limiter circuits and the associated controls.
31
Figure 10. The voltage limiter rationale as given in NASA design documents.
Several kinds of design knowledge are represented by this text. Sentence (1) summarizes the teleological function of the device. (2) explains why the design change was made, indexing it to another design change. (3) describes the relevant structure. (4) presents an informal simulation of how it works. (5) relates a standard component to other contexts of use, and provides an index to more documentation. (6) links the text to a graphical rendition. We are developing the rationale by demonstration technique in the DME environment [Gruber, 1990; Iwasaki et al., 1989]. We will show how the technique can be used to capture, in machine-intelligible form, knowledge of how the voltage limiter works in its intended operating conditions. The knowledge acquired will be sufficient to generate the information in sentences (1), (3), and (4), answer questions not covered in the paper documentation. The steps for rationale by demonstration are: 1. identify relevant structure 2. formulate behavior model 3. identify behaviors of interest 4. set up initial conditions 5. simulate until desired state is reached 6. generate explanations 7. set up alternate conditions 8. simulate until desired state is reached 9. generate explanations 10. repeat 7-9 as needed 1. Identifying relevant structure In this first step, the user abstracts, filters, and aggregates structure descriptions. In a modern design environment, most of the physical structure of engineered artifacts is available in CAD databases. However, for a typical simulation scenario only a fraction of the entire artifact is considered. Also, complex subassemblies are often treated as single objects in a model. In voltage limiter example, the user replaces all of the using equipment with a virtual “diode bus” terminal (abstraction), eliminates several auxiliary components such as the left relay (filtering), and treats the subcircuit for voltage sensing as a black box (aggregation). The resulting structure description is illustrated in figure 11.
32
100A DIODES DIODE BUS
VOLTAGE SENSING CIRCUIT
K112
Figure 11: Identifying relevant structure.
2. Formulating a behavior model In the knowledge-based modeling environment, behavior models are constructed by associating off-the-shelf models of idealized components with the components in the refined physical structure. In the voltage limiter, the 100A diode pairs are associated with “voltage-droppers” from the model library. The physical wires among the components in the real circuit are associated with idealized electrical connections (zero resistance) in the behavior model. The pairs of wires that energize the relay in the physical system are modeled as a single control line that triggers an idealized switch corresponding to the relay. The resulting behavior model is shown in figure 12
33
Input voltage
V1
V2
T1
Voltage dropper
Voltage dropper
VDC1
VDC2
Output voltage T2
S1
VOLTAGE SENSING CIRCUIT
T4
V4
T3
V3
VSC
The voltage limiting circuit VLC Figure 12: The behavior model of the voltage limiter.
3. Identifying behaviors of interest Many possible behaviors can be simulated from a given behavior model. The user identifies behaviors of interest by a) selecting from among all modelled quantities posited by the behavior models (e.g., voltages, currents, states of the relay); b) indicate whether each is endogenous or exogenous; and c) select from among those physical processes that can possibly involved the modelled objects (e.g., in this case, electric current flow through a circuit and not any thermal or magnetic processes). The behaviors of interest are reflected in the modelled quantities such as V1 in figure 12 4. Establishing initial conditions The user has to specify initial values for the exogenous modelled quantities. In addition, the simulation requires initial values for internal states and output states of components, such as whether the relay is open and the initial output from the voltage sensing circuit. In the example session, the initial value for the input voltage is specified with a symbolic constant. 5. Simulating Given the models and initial conditions, the simulator can propagate constraints and generate a sequence of states of the device.13 The simulation proceeds interactively. The user is responsible for 1) stopping the simulation
13DME
is currently using the QSIM [Kuipers,1986] simulator, which works by propagating constraints.
34
when it reaches a desired state and 2) resolving ambiguous state transitions due to nondeterminism in the model. 14 6. Generating explanations When the dust settles on a simulation session, the system can generate causal and process-oriented explanations of the data. A summary of the events in a simulation can follow the history of activated processes and the movements of modelled quantities of interest. The explanation can be general at first, and then elaborated in response to follow-up questions by the user. The following classes of questions can be answered with explanations based on the data produced by a qualitative simulation. An example explanation of “what happened” is shown in figure 13. A purpose of the voltage limiting circuit VLC is to prevent the output voltage V2 at diode bus terminal T2 from exceeding the maximum allowed using equipment voltage MAEV, which is 32.0 V. The voltage limiter VLC works as follows: When output voltage V2 > threshold voltage TV1, and the bypass relay S1 is closed, there is current flow from input terminal T1 through relay S1 through voltage dropping circuit VDC2 to terminal T2, and the input voltage V1 = voltage V2 + voltage drop VD2 across voltage dropping circuit VDC2. Since terminal T2 is electrically connected to the terminal T3 of the voltage sensing circuit VSC, the voltage V3 at terminal T3 = voltage V2. Since voltage V3 > threshold TV1, the voltage sensing circuit VSC outputs voltage V4 > 0V at terminal T4, which causes relay S1 to open. There is now current flow from terminal T1 through voltage dropper VDC1 through voltage dropper VDC2 to terminal T2. Assuming voltage V1 is constant or decreasing, then voltage V2 will decrease by at least voltage drop VD1 across voltage dropper VDC1.
Figure 13: An example explanation for the voltage limiter can be generated from underlying models of the designed artifact.
Since explanations like this are generated from an underlying model, the user can ask follow up question, such as
14Qualitative models often do not contain enough information to determine which qualitative transition
occurs next. For example, if two quantities are increasing toward landmark values (where qualitative changes occur), a purely qualitative mode cannot predict which will reach its landmark first. In “envisionment” systems such as QPE [Forbus, 1984] the entire space of possible states is computed; in time-oriented simulation, the simulation states branch into possible future worlds. In an interactive simulation, the user can guide the system down the desired path of possible futures.
35
What happened? A summary of the behavior of a device during a simulation scenario. By default, this means listing the processes that started and stopped and the values of the modelled quantities that “caused” them to occur and that they influenced. What caused this to happen? If the user selects a description of a value of a quantity or process in a given state, then an explanation of the causal precursors and influences can be given. What is it? A definitional or similar description of a modelled construct, such as a component, a connection, a physical process, a landmark value, etc. What if? A “what happened” explanation for an alternative path in a simulation. Show more detail. This operator applies to any of the query types. Answers bottom out at knowledge that is entered by model builder or is found in the libraries. The ability for users to ask follow-up questions in response to a machinegenerated explanation is the essence of interactive documentation. Unlike static documentation, which must completely anticipate the specific information needs of the reader, interactive documentation achieves the flexibility of model-based reasoning. The cost, however, is the knowledge acquisition effort. Interactive documentation will not be practically useful until the majority of artifact descriptions are routinely created and used in machine-intelligible form.
5. Discussion DME and IDE 2.0 implement radically different approaches to design rationale. DME uses deep models as a probe to elicit explanations of system function, structure and behavior. IDE 2.0, by contrast, has no strong model of artifact or design decision-making. Yet both systems provide the designer with ways of asking questions whose answers must be inferred from design knowledge. As the IDE 1.5 example illustrates, simply collecting and attaching design rationales doesn't provide a great deal of leverage in supporting rationale use. Such rationales are incomplete, difficult to obtain, and hard to work with computationally. To be useful, attached rationales must have a finer representation structure than simple attachments of explanations provide. IDE 2.0, by contrast, has a weak model of how artifact structures implement underlying functional requirements. It can use this weak model to compute explanations about an artifact description. Attached rationales are used in such a system (to explain exceptions to design rules), but such explanations are integrated in a design knowledge system that can use them. Semiformal reasoning can produce rationales by using the design rules to 36
create explanations that are able to incorporate and use the exceptions as needed. From a more formal perspective, DME shows how a deep model of a domain can support design activity by incorporating explanation into the design process. Iterating between working with artifact functional models and the artifact description, a model-based design system maintains a functional / structural representation of the artifact that can be used to answer an extremely wide range of questions about the artifact. The simulation technology that drives DME also gives it the ability to create rationales in response to many functional questions. In both cases — deep models of artifacts and semiformal representations of design — design rationales are bound up in the relationship between artifact and underlying design knowledge. A spectrum of representational techniques form formal to semiformal can support design rationales, the important point is that models and inferential mechanisms exist. As a consequence, design rationale is not a specific property of a design process, representation, or problem solving technique. It's answering questions. If you ask hard questions over a wide range of topics about a design, you need to provide fairly general question answering mechanism. Here, we have described an ontology that attempts to capture the salient features of design explanations over a wide range of differing designs, domains and design questions. Design rationale is best viewed as explanations that are generated rather than recorded. This approach gives the designer a great deal of leverage. After all, the point behind design rationale is to inform someone about a design. Since we don't want to unnecessarily limit the range of those questions, creating design rationales in response requires that models of the structure, behaviors, functions and design decision making be represented. In our view, this implies that design rationales are dependent on first capturing design knowledge. Capturing design knowledge in a way that it can be used and accessible to the user as explanation is fundamental and prerequisite to design rationale. As we have discussed, both IDE 2.0 and DME operate on artifact descriptions. However, this ontology also suggests that analogous research can be done in reasoning with design histories. Just as we have shown both semiformal and strong models of artifact design rationalization, we believe that rationales can be created to explain decisions made during the course of a design. Such a system would neatly integrate both the artifact and decision-making aspects of design problem solving.
37
Acknowledgements We would like to thank many people for many long discussions on design knowledge, rationale, and related topics. In particular, Sanjay Mittal, Brian Falkenhainer, Ken Forbus, Ralph Ginsberg, Yumi Iwasaki, Pete Pirolli, Marty Tenenbaum, and Brian Williams have been very helpful and influential.
References Abelson, H. et. al. (1989). Intelligence in Scientific Computing. Communications of the ACM, 32:546-562. Bareiss, E. R. (1989). Exemplar-based Knowledge Acquisition: A Unified Approach to Concept Representation, Classification, and Learning. Boston: Academic Press. Baudin, C. Sivard, C. and Zweben, M. (1989). A model-based approach to design rationale conservation. IJCAI-89 Workshop on Model-based Reasoning, Detroit. Bobrow, D. (ed) (1988) Qualitative reasoning about physical systems. Bradford Books, Cambridge, MA. Brown, D., Chandrasekaran, B. (1987) Design Problem Solving. Morgan Kaufmann, Los Altos, CA Brown, J. S. (1990) Dialectics for design: Expanding the role of design in implementation. “Technology and the future of work” Conference, Stanford, CA. Brown, J. S., Burton, R., and deKleer, J. (1982). Pedagogical, natural language and knowledge engineering techniques in SOPHIE I, II, and III. in Intelligent Tutoring Systems. Sleeman and Brown (Eds.), Academic Press. Card, S., Henderson, A. (1987) A multiple, virtual-workspace interface to support user task switching. CHI+GI Conference, ACM. Conklin, J. & Begeman, M. L. (1988). gIBIS: A hypertext tool for exploratory policy discussion. Proceedings of the 1988 Conference on Computer Supported Cooperative Work (CSCW-88), Portland, Oregon. Delisle, N. & Schwartz, M. (1986). Neptune: A hypertext system for CAD applications. SIGMOD-86, Washington, D. C. Fischer, G., McCall, R., & Morch, A. (1989a). Design environments for constructive and argumentative design. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI-89), pages 269-275, Fischer, G., McCall, R., & Morch, A. (1989b). JANUS: Integrating hypertext with a knowledge-based design environment. Hypertext '89, pages 105-117, Douglas, S. A. and Liu, Z. Generating causal explanation from a cardio-vascular simulation. IJCAI-89. Forbus, D. K. and Falkenhainer, B. (1990). Self-explanatory simulations: An integration of qualitative and quantitative knowledge. AAAI-90. Franke, D. W. Proposal for research: Representing, utilizing, and acquiring teleological descriptions. AI Lab, University of Texas, Austin, TX. Computer Science Department technical report AI89-112. Goel, V. & Pirolli, P. (1989) Generic design problem solving. AI Magazine. Grice, H. P (1975) Logic and conversation. In Cole, P., Morgan, J., Syntax and Semantics: Speech Acts, p 41-58, Academic Press, NY.
38
Gruber, T. R. (1989). The Acquisition of Strategic Knowledge. Boston: Academic Press. Gruber, T. R. (1990). Model-based explanation of design rationale. Technical Report KSL 9033, Computer Science Department, Stanford University . Extended abstract in Proceedings of the AAAI-90 Workshop on Explanation. Gruber, T. R. & Iwasaki, Y. (1990). How things work: Knowledge-based modeling of physical devices. Knowledge Systems Laboratory technical report, Stanford University. Guindon, R. (1988) A Framework for building software development environments: System design as ill-structured problems and as an opportunistic process. MCC Technical Report STP-298-88, Austin, TX. Hempel, C. (1970) Aspects of Scientific Explanation. Free Press, NY, NY Hewson, R. (1990) Sketching by numbers: is typographic design possible in the electronic paradigm?, Xerox PARC SSL Technical Report XXX-XX-XXX, Palo Alto, CA Hwang, T. S. & Ullman, D. G. (1990). The design capture system: Capturing back-of-theenvelope sketches. International Conference on Engineering Design (ICED 90), Dubrovnik, Yugoslavia. Iwasaki, Y. and Simon, H. (1986). Causality in device behavior. Artificial Intelligence, 29:63-72. Iwasaki, Y., Doshi, K., Gruber, T.,Keller, R., and Low, C. M. (1989). Equation model generation: Where do equations come from? Proceedings of the 1989 Workshop on ModelBased Reasoning at IJCAI-89. Iwasaki, Y. (1990). Reasoning with multiple abstraction models. Fourth International Workshop on Qualitative Physics. Kellog, C. Mark, W., Sullivan,(1988). Interactive acquisition and explanation of design knowledge. Proceedings of the AIAA Aerospace Sciences Meeting. Kuipers, B. (1986). Qualitative simulation. Artificial Intelligence, 29:289-338. Lakin, F., Wambaugh, J., Leifer, L., Cannon, D., & Sivard, C. (1989). The electronic design notebook: Performing medium and processing medium. Visual Computer: International Journal of Computer Graphics, Lee, J. (1990) SIBYL: A Tool for Managing Group Decision Rationale. To appear in Proceedings of CSCW '90. Macaulay, D. (1989) The Way Things Work. Houghton-Mifflin, New York, NY. MacLean, A., Young, R. M., & Moran, T. P. (1989). Design Rationale: The argument behind the artifact. CHI-89, Austin, TX. Malone, T. (1986) Semi-structured messages are surprisingly useful for computer-supported coordination. CSCW Conference, Austin, TX. Mark, W. (1988). Explanation and interactive knowledge acquisition. AAAI-88 Workshop on Explanation. Mark, W. & Schlossberg, J. (1990). Design Memory. Lockheed AI Center, Sand Hill Road, Menlo Park, CA. Marshall, C. (1987). Exploring representation problems using hypertext. Hypertext '87, pages 253-268, Chapel Hill, NC. McGinnis, B., Ullman, D. G. (1989) The Evolution of commitments in the design of a component. ICED Conference, Harrogate, UK. Mostow, J. (1985) Toward better models of the design process, AI Magazine, vol 6, no 1. Newman, S., Marshall, C. (1990) Pushing Toulmin Too Far, Human Computer Interaction (this issue)
39
Pugh, S. (1981) Concept selection -- A method that works. Proc. ICED, WDK 5 Paper M3/16, Rome, Italy, pages 497-506. Russell, D. M. (1988). IDE: The Instructional Design Environment. In J. Psotka, D. Massey, & S. Mutter (Eds.), Intelligent Tutoring Systems: Lessons Learned. Hillsdale, NJ: L. Erlbaum Associates. Russell, D. M. (1990) Reflections on the IDE Project, Instructional Design Systems Conference, Logan, UT (in press) Shema, D., Bradshaw, J., Covington, S., & Boose, J. (1989). Design knowledge capture and alternative generation using possibility tables in CANARD. Knowledge Acquisition for Knowledge Based Systems Workshop, Banff, Canada. Sommerville, I., Haddley, N., Mariani, J., & Thomson, R. (1989). The designer's notepad: A hypertext systems tailored for design. Hypertext II Conference, York, UK. Steele, Guy L. (1990) Common LISP: The Language (Second Edition). Digital Press, MA Streitz, N. (1989) From ideas and arguments to Hyperdocuments: Travelling through activity spaces. Hypertext ‘89, Pittsburgh, PA Tenant, H. R., Ross, K. M., Saenz, R. M., Thompson, C. W., & Miller, J. R. (1983). Menu-Based Natural Language Understanding. Proceedings of the Association for Computational Linguistics, pages 151-158. Tuienga, Paul. (1988) SPICE: A guide to circuit simulation and analysis, Prentice Hall Ullman, D., Wood, S., Craig, D. (1989) The importance of drawing in the mechanical design process. Design Process Research Group Technical Report, Oregon State University, DPRG-89-1 Weld, D., deKleer, J. (1989) Readings in Qualitative Reasoning about Physical Systems. Morgan Kaufmann, Los Altos, CA.
40
