Derivation and Use of Design Rationale Information as Expressed by ...

Knowledge Systems Laboratory Technical Report KSL 92-64

December 1991 Released July 1992

Derivation and Use of Design Rationale Information as Expressed by Designers by Thomas R. Gruber Daniel M. Russell

Derivation and Use of Design Rationale Information as Expressed by Designers Thomas R. Gruber Knowledge Systems Laboratory Stanford University 701 Welch Road, Building C Palo Alto, CA 94304 [email protected]

Daniel M. Russell Systems Sciences Laboratory Xerox Palo Alto Research Center 3333 Coyote Hill Road Palo Alto, CA 94304 [email protected]

Abstract A design rationale is an explanation of why something is designed as it is. In this paper we analyze the sources of knowledge and inference underlying design rationale. We examine protocols of people talking about designs in several domains to identify kinds of design information that are requested and used. We classify the information types along dimensions including the source of the information, how the information is or could be captured, and whether it is retrieved or inferred. We find that the sources of knowledge used to explain a design are manifold, including models of the artifact and how it works, models of design methods and decision making processes, and understanding of the intended functionality and other requirements. We observe that design rationale explanations are often constructed from several of these sources, supported by activities such as information retrieval, simulation, hypothesis testing, and decision making. These observations can inform the design of design rationale tools. One implication is that the “capture and replay” approach is incomplete for acquiring many kinds of rationale information. Since rationales are often inferred from available information, rather than stored and looked up en masse, the space of possible information requests is difficult to fully anticipate. Furthermore, preformulated or stored rationales become obsolete when assumptions and constraints change. A related conclusion is that the usefulness of the information that is captured in a design history depends on how it is indexed and integrated with existing engineering tools. Another consequence is the need for software support for the capture and management of dependencies among several kinds of design elements: design decisions, alternative design choices, requirements, other constraints, evaluation criteria, artifact structure, expected behavior, and assumptions about the operating environment.

1 Introduction A design rationale is an explanation of why an artifact, or some portion of it, is designed the way it is. Such an explanation draws on a variety of sources. In the service of explaining a design, a rationale may clarify information about the design often left implicit in artifact descriptions, such as what behavior is expected under which operating conditions, or how elements of the design are related by constraints and other dependencies. A design rationale may also describe a reasoning process that justifies the design, such as why particular structures are chosen over alternatives, how required functionality is achieved by prescribed behavior, or how behavior is expected to arise from the structure of the artifact. People and their programs use the information provided by design rationales for many engineering tasks, from early conceptual design through diagnosis and repair to redesign and maintenance. Design rationale information helps designers reuse, modify and maintain existing designs. Creating and maintaining an explicit record of design rationale can help support collaboration among groups, such as concurrent product engineering teams standards specification committees, and producer-consumer negotiations . In current engineering practice, the information is often lost because designers forget it, leave the project, or are inaccessible in a large organization. A technology for software support of design rationale is just beginning to be developed. A range of approaches have been proposed with varying cost and performance profiles.1 A number of fundamental questions have yet to be answered, such as what kind of design information is useful to capture, and how it can be used to support engineering practice. In this paper we analyze the sources of knowledge underlying design rationale and the types of inferences made from that knowledge, considering the ways in which the resulting explanations are used. We examine protocols of people talking about designs to identify kinds of information that are requested and used. We classify the information types along dimensions, such as the source of the design information, how the information is or could be captured, and whether it is retrieved or inferred. We then analyze how these observations might inspire the design of design rationale tools. Section 2 describes the survey of protocol studies and the analysis method, and summarizes the observations. Section 3 contains the analysis of uses of rationale information, sources of knowledge, and types of inferences involved. Section 4 gives our conclusions on the implications for the design of software support. First, let us reveal a guiding bias.

1

Many of the approaches are represented in this volume. [Gruber, Boose, Baudin, & Weber, 1991] offers a preliminary analysis of the tradeoff space.

3

1.1

Design rationale as explanation in response to information needs

There are several definitions of design rationale in the literature, and each reveals a different perspective on what is salient. We find it useful to view design rationale as an explanation in response to an information need about the design. By explanation we mean a structured communiqué rather than a simple statement of fact. For instance, an argument consisting of a set of reasons for and against a position is an explanation, where the individual facts that are offered as evidence are not. By information need we mean a need to know something in order to accomplish some task. The need may be explicitly requested or anticipated by the explainer. By saying that a rationale conveys information we can ask what is conveyed and how; focussing on task-situated needs leads us to analyze the purpose of the information. Viewing the rationale as a response suggests that the explanation is constructed to communicate the desired information. This implies an author and intended receiver, and an active process of putting together the necessary information. We might imagine technology for capturing every utterance and gesture of a designer, but what matters is that the relevant information be made available in the right form for the person who needs to know the rationale for a design. Our research methodology follows from this perspective. Our objective is to inform the design of tools for the capture and use of design rationale in support of engineering practice. So we look for information needs in the form of questions and conjectures about a design, and analyze the answers in terms of the information used to construct them and whether it could be captured. We consider the ways this information might be used, and in this context, what computational services are required to retrieve or infer the answers.

2 Survey of Empirical Studies on the Use of Design Information For data on the kinds of information needed, under what conditions, and how it is provided, we turn to protocol studies of people talking about designs. We examined a set of empirical studies of people requesting, communicating, and using the kind of information associated with design rationale. The data include protocols of individual designers thinking aloud during innovative design, teams of designers discussing a previous design for the purpose of redesign, designers asking for information during a design, and designers negotiating with customers and other stakeholders about a design in progress. The studies covered designs in several domains: architecture, electromechanical devices, heating and air conditioning systems for buildings, userinterfaces in software and hardware, and the design of instructional courses. Our goal is to characterize a space of design rationale information at an abstract level by identifying questions and statements used in these settings. We studied existing protocols in a variety of domains not as a representative sample of what designers do or think, but as a source of exemplars of the kinds of information needs and uses, and means for providing the answers. 2 2

Given the variety of domains and the artificial settings of some of the experiments, it would be inappropriate for us to draw quantitative conclusions about the design process, such as the relative frequency of various kinds of information requests. We refer the reader interested in these kinds of empirical results to the reports on the individual studies.

4

2.1

The empirical studies surveyed

We obtained a variety of protocols, and where possible, published summaries of the data, from the following sources. We are very grateful to the researchers who shared the results of their work (see acknowledgements). The numbered bullets (➀–➆) will be used throughout the text to refer to these sources. ➀ Kuffner and Ullman’s studies of the information requests of mechanical engineers [Kuffner, 1990; Kuffner & Ullman, 1990]. Three subjects, practicing mechanical engineers, were videotaped thinking aloud while doing redesign. Two subjects redesigned a piece of manufacturing equipment for coating plates, and the third redesigned a battery enclosure for a portable computer. The subjects were given complete specifications and drawings of a previous design of the artifact, and instructed to make a series of changes to the design (new or modified requirements and operating conditions). The experimenter was present to answer questions about the design. The experimenter made transcripts, and identified segments corresponding to information requests. Segments were of two types, questions and conjectures. Questions were direct inquiries to the experimenter or utterances indicating a search for information by the subject, including references to documents, personal notes, and memory. Conjectures were conclusions about the design inferred from incomplete information, including assumptions, suppositions, and interpretations. We used the experimenter’s segmentation and included both questions and conjectures as segments. ➁ Ana Garcia’s study of design rationale in civil engineering [Garcia, 1991; Garcia & Howard, to appear]. Garcia has studied designers of heating and air conditioning systems for buildings (HVAC). She took verbal protocols of designers discussing their work, and also collected written rationale documents. We used the data from the written documents only.3 The documents are “requests for rationale,” forms and letters in which stakeholders in the design such as the building owner, architects, and construction designers challenge and negotiate the design decisions. The stakeholders have access to drawings and documents such as building codes. Questions include requests for clarification and information, requests for justifications of decisions, and proposed changes to designs. The request for rationale documents are naturally structured into issue/response transactions, so no segmentation was necessary. We used the experimenter’s sampling of salient questions and answers from a larger corpus; she picked representatives to cover a range of request formats, request types, and answer types. ➂ Bellotti and MacLean's study of two designers discussing a proposed redesign an automated teller machine. The designers’ goal was to solve the problem of long lines of customers waiting to use the ATM [Bellotti & MacLean, 1989]. The designers are professional software designers who have worked together in the past. They were videotaped working on the problem for about forty five minutes. The protocol was transcribed and annotated with any significant non-verbal activities to aid in understanding the text without reference 3

Garcia is currently analyzing the protocol and document data to build a predictive model of the design process in the HVAC domain. The model will be used as part of a design rationale system she is building. A preliminary report in the representation and architecture of the system may be found in [Garcia & Howard, to appear].

5

to the video. The authors segmented the protocol into 385 pieces, each with an ID number. We have used their segmentation and numbering scheme. ➃➄➅ Pirolli and Goel’s studies of designers in three different domains [Pirolli & Goel, 1989]. Their transcripts cover ➃: an architect designing a small automated postal service center for a university, ➄: an instructional designer creating a detailed course design for a complex office workstation and ➅: a mechanical engineering design of an automated teller machine to be placed in a post office. In each of these cases, the designer had a problem specification to read, and some reference materials at hand -- the actual site on campus for the architect, documentation and earlier course materials for the instructional design, and a set of design documents for the location of the automated teller. The designers were instructed to “think aloud,” and an experimenter was physically present to encourage a naturalistic flow of speech, answer simple questions, and to prompt delicately when the subject became quiet. The sessions were videotaped, transcribed, and segmented. We have adopted the authors’ segmentations. ➆ Catherine Baudin et al.’s study of the information requests of mechanical engineers redesigning a damper for a computer-controlled automobile shock absorber [Baudin, Gevins, Baya, Mabogunje, & Bonnet, 1991; Leifer et al., 1992]. In this case, a graduate student with experience in mechanical engineering was given a set of design documents for the existing damper and a problem statement describing a set of changed requirements. An experimenter was present to prompt for thinking aloud during silences. The designer had access to the (extensive) design documentation on the existing artifact. In addition, one of the designers of the original artifact was present to answer questions about how to find information in the design documents (e.g., “where is the calculation of the maximum arm length?”) and to answer questions that were not answered in the document set (e.g., “Subject: I don’t see how you’ve conducted [the heat]. Original-designer: Yes, we did not do any elaborate design efforts toward conducting it away.”) The protocol was transcribed and segmented by the experimenters into information requests. The segments represent information needs of the designer, which included direct questions and indirect expressions of needs for information. We used the experimenters’ segmentation and rephrasing of fragments into complete sentences.

2.2

The analysis method

We followed the following procedure in analyzing the data. 1. Identify and extract segments of protocols where speakers are questioning, answering, conjecturing, or describing the design. Some of these extractions of information transactions were performed by the original researchers.4 2. Transform each segment into a generic question using a limited vocabulary of abstract terminology (the glossary is given in Appendix 1). The transformation involves three 4

The method of transforming segments of think-aloud protocols into information requests is due to Kuffner and Ullman [1990 ], based on [Kato, 1986 ]. They used this technique to measure whether information was conjectured or confirmed, derived or introduced, and whether the answer was or would be correct, incorrect, or unknown.

6

operations. (a) Rephrase the segment as a question. In cases where a segment is not an overt request for information, infer a generic question from the segment and the context in which the mentioned information is used. (b) Transform each concrete, domain-specific term in the question into an abstract term from the glossary, such as “parameter” or “requirement.” (c) Use the adjective “this” to mark constants in the query, and articles ‘a’ and ‘an’ for typed variables. For example, the segment “The section is ... basically a cantilevered roof...”➃ becomes the generic question “What class of device or component is this part?” 3. For each segment, characterize the possible types of answers, and the possible information sources for answering the questions. The answer field should contain a description of the form of the answer and the means for obtaining it. This may require inference from knowledge and information not explicitly present in the protocol data.5 For example, an answer could be “retrieval of a requirement statement” and the sources could be “design or requirements document.” It is permissible to list more than one information source. For those segments where there was an explicit answer given with an identified source, then it should be listed. In all cases, one should also consider answers which could be given if the appropriate information source was available. For example, for the generic question “Is this explicit constraint on structure satisfied?” we identified two kinds of answers. One is “a confirmation of a checked constraint” based on formal simulation or constraint models. Another is “a justification of the decision about the structure in question” which could be provided by a design decision history. 4. For each question/answer pair, determine whether the following properties hold. • Looked up (retrieved) or Inferred (computed). If the necessary information were recorded, would the answer to the question be a simple retrieval of the requested information as is, or would the answer be computed (e.g., from equations) or inferred (e.g., by a rule). • Formalizable or Semiformal or Informal. Could the information be formalized in a way that could reused under software support, or it is something that would require human interpretation. This requires some subjective judgement. If there exists a formalism in the literature or supported by tools in which the information could be expressed, then code it formal or semiformal. If the knowledge is representable in principle by something like predicate calculus, but is of a domain that it would not be practical to cover with a knowledge base, then label it informal. We iterated over the segments, trying to converge in a vocabulary for generic questions, answers, and information sources that would adequately cover all of the examples.

5

We did not restrict our analysis to the actual information sources used by the subjects in the studies, because we are looking for possible ways to support design rather than to account for how it is done today.

7

2.3

An example analysis

We will walk through the analysis for one of the generic questions, to illustrate how the analysis proceeds. Consider the following question from study ➀. We’re not worried about color, for instance.

We transform this segment into a generic question by turning the statement into a question (“is color something to worry about?”), and replacing “color” with “this constraint” and “is it something to worry about” “is it a requirement” producing Is this constraint a requirement? This transformation is a simple kind of induction, called turning constants to variables. In this case, the variables are the terms “constraint” and “requirement” from the generic vocabulary (Appendix 1). Since this was a conjecture, the “answer” is really an assumption that the color is not a constraint specified in requirements. It wasn’t specified in the requirements for the device being designed. The designer assumed that color didn’t matter because the battery case being designed is inside the computer. We generalize this kind of answer as: • requirement inferred from background knowledge by designer

Now we label the source of such an answer, reusing a category already defined for an earlier answer: Human: human designer or client (domain knowledge)

Finally, we code the other properties, lookup/inferred and formality. We say it is inferred because determining whether a specific constraint is a requirement from experience is likely to involve some reasoning (i.e., it is not just a fact that is recalled from memory). We label it semiformal because we could imagine a tool capturing the statement that color is not a constraint and explicitly linking it to formal requirement and constraint objects. • requirement inferred from background knowledge by designer ........................................................................................................ Human Inferred Semiformal

Now we consider other ways that the question might be answered, and the sources of knowledge. We decide that the answer could have been retrieved from in a requirements document, which we code as: • retrieval of requirement statement ............................................................... DesDoc Lookup

Semiformal

In the format of Appendix 2, the complete entry now looks like this:

8

Is this constraint a requirement? We’re not worried about color, for instance. • requirement inferred from background knowledge by designer ........................................................................................................ Human Inferred Semiformal • retrieval of requirement statement ............................................................... DesDoc Lookup Semiformal

2.4

Summary of Questions Asked

Following the procedure outlined above we analyzed all of the protocols collected and produced a set of 63 generic questions. These questions define a space of design information, which we analyze as a grid in Section 3. Each generic question represents a kind of information need or use, and a potential opportunity for computational support. The questions are grouped into 14 question classes. There are several dimensions one could use for classification, such as design vs. redesign, conceptual vs. detailed, and so forth. We grouped questions by topics, guided by the question-forming vocabulary, to help in making sense of the data. Generic questions may plausibly fall under more than one class, but are grouped under exactly one class in the following list. This list of questions is surely incomplete, yet represents a fairly large space of design information. Appendix 2 contains the complete set of generic questions, answer types, with examples from the protocols. This section lists just the generic questions. 1

Requirements What are the given requirements? Is this constraint a requirement? Give more detail about this parameter of the operating environment. Can I assume this fact about the operating environment? What are the requirement constraints on this parameter? Is this parameter constrained by external requirements? What is the expected behavior of this artifact in the scenario of use? Should I assume that this functionality is required? Can I modify this requirement?

2

Structure/form What are the components? What class of device or mechanism is this part? What is the geometry of this part? (qualitative) What material is this part made of? How do these components interface? What are the locations of parts, connections, etc (for constraint checking)? What are the known limitations (strengths) of this part/material class? What affects the choice of artifact components?

3

Behavior/operation: what does it do What is the behavior of this parameter in the operating conditions What is the behavioral interaction between these subsystems? What is the range of motion of this part? What is the cause of this behavior? What are the expected failure modes in the scenario of use?

4

Functions What is the function of this part in the design? What is the function of a feature of a part in the design?

5

Hypotheticals

9

What happens if this parameter changes to this new value? What is the effect of this hypothetical behavior on this parameter? Adapt equation to this changed parameter and recompute What will have to change in the design if this parameter changes to this new value?

6

Dependencies What are the known dependencies among the parts? What are the constraints on this parameter? Is this parameter critical (involved in a dominant constraint)? How is this subassembly related to this parameter? What is the source of this constraint?

7

Constraint checking Is this constraint satisfied? Does this structure have this behavior which violates this constraint? What are the known problems with this design? Would a part with this functionality satisfy this constraint?

8

Decisions What are the alternative choices for this design parameter? What decisions were made related to this parameter? What was an earlier version of the design? What decisions were made related to satisfying this constraint? Which parameter, requirement, constraint, or component should be decided first? What design choices are freed by a change in this input parameter? What alternative parts that satisfy this constraint could substitute for this part? Where did the idea for this design choice come from?

9

Justifications and evaluations of alternatives Why this design parameter value? Why is design parameter at value V1 instead of normal value V2? Why was this alternative chosen over that alternative? What is person P’s evaluation of these alternatives? Why not try this alternative?

10 Justifications and explanations of functions Why is this function provided? Why is this function not provided? Why can’t the current design achieve this new value of this functional requirement parameter?

11 Validation explanations How is this requirement satisfied? How is this function achieved? How is this functional requirement achieved? How will this part be maintained?

12 Computations on existing model Compute a parameter value given other parameters What are the trajectories of parameters

13 Definitions What does a term in the documentation mean?

14 Other design moves Search for information expected to be in documentation, eg, equation or diagram. Change this requirement constraint and update design. Have all the arguments for/against this alternative been checked?

Table: List of Generic Questions

2.5

Summary of Generic Answers

For each generic question, there are one or more generic answers. Here we summarize the range of generic answers, grouping them by the format of the answer. The classes of answer formats

10

correspond to the procedures that would automate the generation of these answers. The simplest answers to generate are yes or no confirmations and fill-in-the-blank (atomic) answers. More sophisticated procedures would be needed to generate explanations and other richly structured answers. These generic answers were extracted and generalized from specific answers. The answers are stated here without the context of the generic questions and specific protocol segments. The context, along with the format and content of the answers, contributes to their interpretation. Confirmations “Yes or no.” • permission to make assumption about requirement • confirmation of function achieved / structure present / behavior performed • confirmation of a checked constraint • confirmation that information is in documentation

Atomic answers “The answer is X.” • clarify ambiguity in description / text / picture • qualitative or quantitative parameter value • parameter value, under default / normal / special conditions • classification of device or mechanism in standard language • definition of term

List answers “The answers are X, Y, ...” • list of requirements / assumptions / parts / parameters on parts / current design goals / design dependencies / alternate parts / possible choice of {materials, parts, param values} / decisions made that affect a constraint / criteria used to make decision / possible criteria

Retrieval of Historical Record • design record or earlier version of component / system / design decision / evaluation model / design strategy used / how decision was reached

Structured Descriptions • informal sketch showing connections between components • CAD rendering of geometry or connections between components • description of parameter range • expression of a constraint on a parameter • detailed description of material / process / procedure • description of where information came from • background data on reliability / customer satisfaction • summary of information found in document

Explanations of how it works “It works like this: ...” • causal story to explain behavior / how function is achieved / how structure and behavior satisfy constraint / how requirement is achieved by behaviors and structures • explain how behavior violates criterion • causal / dependency path between model of system and parameter • scenario of use / envisionment of scene / scenarios of failure modes • description of maintenance procedure • a set of behavior equations • explanation of expected use or behavior of part / system

Justifications “Because ....” • justify criteria uses / not used • justify design strategy in terms of cost / complexity / relevancy to design problem

11

Response to Imperatives “Okay, it is done.” • check (validate) requirement • simulation of part role / system / subsystem with variance • reformulation of the equations accommodating the new param value

Response to Hypothetical “If X then Y.” • analysis of effects of change on constraints • sensitivity analysis of a parameter • analysis of limits on possible behavior • results of a verification analysis

Evaluations • evaluation of alternatives against criteria • verify that all arguments for / against an alternative have been checked • assessment of overall strengths / weaknesses of design • methods used in evaluation

2.6

Summary of Sources of Knowledge

In analyzing the protocol segments, we identified the knowledge sources used in answering each generic question. Where it was not possible to determine the knowledge source, we made our best guess of the most likely sources based on our experience. We then grouped the sources into four major categories, which are analyzed in Section 3. Each source of knowledge used in creating design rationale statements is categorized and given a code for use in the protocol analysis. We merged and generalized knowledge sources until the set stabilized at that listed below. Most knowledge sources were used in several generic answers. There are roughly three times as many generic answers as sources of knowledge. The number beside each knowledge source points to an generic question and answer in Appendix 2. Task-specific information — documentation and data available about the design, in humanreadable text and graphics. DesDoc: design document or notebook (text and graphics) ReqSpec: formal requirements specification (mix of formal and informal) Handbook: a component description from a handbook or parts database Glossary: reference glossary for domain UsageData: external reports on artifact use (data on reliability, etc.) StdPrac: codification of standard practice in domain

Design history — a record of the design process over time History: design decision history (alternatives, criteria, decisions; audit trail or checklist) Cases: explicit links to decisions in previous designs using parts of this class

Engineering models and tools — machine-interpretable formalisms Decision: decision model comparing alternatives on criteria CAD: conventional CAD model or engineering drawing SimMod: simulation model of artifact behavior in operating environment Constr: constraint model relating formal design parameters FunMod: a formal model of functional-dependencies Demo: demonstration scenario recorded by designer DepMod: a model of dependencies between parameters FunLib: library of functionally indexed component models (eg, of nand gates, amplifiers, valves) KB-CAD: knowledge-based CAD model

12

Strategy: model of design problem solving strategy (used by automated design system)

Background and domain knowledge — implicit, not formally represented Human: human designer or client (domain knowledge) Author: human author of design document (design specification knowledge) Envis: scenario of use envisioned by designer (knowledge about artifact in use or environment of use)

Table: Sources of knowledge used to answer questions about design

2.7

Uses of the Information

By examining the collected protocol statements in their original context, we found several major categories of use. These categories are derived by considering the role each answer played in the design setting.6 Below is a list of general uses or roles of information during design. Following each use category are examples of the kinds of statements supporting these uses. The numbers in parentheses are examples from Appendix 2 illustrating representative uses from the protocols.

1. clarifying requirements or assumptions about operating environment • restate requirements in more detail or in current design context (1.3) • specify conditions under which the designed artifact must operate (1.1 - 1.9)

2. assisting in making a design decision among alternatives • what criteria are used to make a design choice (2.8, 8.9, 8.10) • how a part choice is evaluated on design criteria (9.3, 9.4, 9.5) • alternatives for the part, including those already considered (2.8, 9.5)

3. understanding the existing (given) design • the artifact itself: its parts, behaviors, connection topology (2.1-2.8, 3.1-3.5) • the functional role of a component in the system (4.1-4.2) • the constraints that a component satisfies and doesn’t satisfy (7.1-7.4)

4. explaining the effects of changes to the design or a requirement • • • •

the constraints to consider when making the change (5.1-5.4, 6.1-6.4) the impact of a change in the design on requirements (1.5, 1.6, 1.9, 6.3, 6.4, 11.1, 11.3, 14..2) the things to change in the design given a change in requirements (1.8, 1.9, 6.4, 9.2, 9.5) alternatives for the part, including those already considered (7.4, 8.1, 8.7, 9.3, 9.5)

5. explaining the expected operation of the device • what-if: the behavioral ramifications of a scenario not yet considered (1.7, 1.8, 3.5, 5.2, 9.3, 11.1, 11.2, 11.4)

6. verifying and validating the design • constraint checking and verification of requirements (5.4, 6.2 - 6.4, 7.1-7.4, 11.1 - 11.4) • optimality: what are the evaluation criteria? How does the design measure up? How does it compare with alternatives? (2.8, 8.9, 8.10, 9.3 - 9.5) • monitoring the design process: have all the plausible alternatives been considered? what’s the next step to pursue? (8.10, 14.3)

Table: Roles of Design Information in the Context of Use

6

There are other uses of talk during design than the ones we examine here. A statement may be used to assert authority or establish control in a design session, as suggested by anthropological studies of group work. In this text, however, we focus on the information of the protocol segments related to the design itself.

13

We find it difficult to associate any single category of use with design rationale. All of these uses of design knowledge can contribute to an explanation of why something is designed as it is.

3 Analysis of Sources of Knowledge and Inference Underlying Design Rationale This section presents our analysis of the observations presented in Section 2 and Appendix 2. We examine each of the basic sources of knowledge that were used, or could be used, to find answers to the generic questions. For each source of knowledge we will analyze (a) how the knowledge is used in design, (b) the role of inference in obtaining the answers, and (c) ways in which the knowledge could be formally represented.

3.1

Domain- and Task-specific Documentation

Much of the information mentioned in the protocols can be made available as documentation. Documentation includes general references such as handbooks of components or engineering textbooks containing standard models. A large amount of project-specific documentation is also relevant: requirements specifications, guidebooks or codes on standard practice, reports on previous designs, and notebooks. We include both paper documents and databases (e.g., of parts) in this category. Uses of documentation. Information obtained from documentation permeated many of the activities of designers. Many of the questions were about the requirements. The existing requirements documentation was often incomplete, prompting the subjects to ask a human for clarification. The graphics in a figure or the symbols in an equation were often ambiguous. Subjects conjectured about whether information was present in documentation, and asked for help in finding information. Decisions were justified by reference to facts potentially stored in documentation, such as “this person ratified this decision.” In one case, designers had to justify the document set they were using (“Why do you list the Uniform Building Code, 1985 Edition, when we should be designing to the 1988 Edition?”➁). The role of inference in using documentation is the problem of information retrieval. Obviously, computers can search large information sources efficiently, if they are given operational descriptions of what to look for (patterns to match). There is an inherent tradeoff between the quality of the indexing and the utility of the search for getting the answers sought. Much current research addresses the issue of retrieval once a complex body of knowledge has been captured on-line. (See, for example, [Boy, 1991; Fischer & Stevens, 1990].) Beyond the cost of indexing, there is a limit to what is captured at all in documents. Many questions in the protocols were about details that are difficult to anticipate when writing documents. For example, the battery case designer wanted to know whether the color mattered, because it influenced a decision about materials.➀ Roles of formal representation. In the context of retrieving information from design documentation, the most important knowledge provided by the human is what is relevant. The

14

author tries to predict what information will be relevant to the reader. The reader wants to know what factors were considered relevant by the previous designer. This is more important than the details of the facts, which can be looked up. Formal representation can help capture relevance information in two ways. The first way is rather simple: documents can be structured by templates that represent institutional memory of what is relevant. For instance, a form with slots for cost, environmental impact, etc. forces the author to anticipate what the institution deems important. Checklists are another kind of structured documentation, where the designer is required to evaluate the design against a set of constraints. A second role for formalism is in indexes. Facts are indexed because it is predicted that it will be useful to retrieve them that way. An important kind of index that would address many of the questions we saw is the functional index — a mapping from functions (amplify, actuate, ...) to components or subsystems. Except in a few special domains such as digital electronics, functional indexing is an open problem and opportunity for computational support. 7 Even a semiformal representation of function would be useful. For example the answer to the question “Why the 2 inch tubing?”➀ might have been recorded in a design notebook if there was a built in link type for dimensions and features of parts to lists of requirements.

3.2

Decisions and Evaluations

A large part of what the designers talked about in the surveyed protocols was decision making. Decisions ranged from fine-grained choices of parameter values to high-level decisions about which technology to use. A complete decision involves a set of alternatives, a set of criteria, evaluation of those alternatives against those criteria, and justifications for those evaluations. Justifications can make reference to almost any knowledge about the design, from appeals to authority to simulations of artifact behavior. Typically, information requests were about some approximation to this ideal. For example, designers often asked just for the set of alternatives, or the set of criteria. This is another case where it is important for the designer to know what is relevant, even without knowing how. Knowing how an alternative stands up to evaluation criteria is a stronger source of rationale information. Uses of decision-related information in design. Decisions are so ubiquitous in design activity that some researchers define design in terms of decision making. We found two basic uses for information explicitly related to decisions. First, decisions are described as events (what we call “design moves”) that serve as loci for considering alternatives and linking dependent elements in the design. For example, a designer will ask what decisions in the previous design were related to a specific parameter. The answer is a set of clusters of information (alternatives, criteria, etc) that identify places in the search space where the new design might differ, and what other parameters should be re-examined if the design took a different path. A second use of decisionoriented information is evaluation. The evaluation of an entire design is decomposed into evaluations of subdesigns. Decisions encapsulate the context in which local choices can be evaluated against subsets of all the criteria. When a decision is revisited, information about the

7

See [Sticklen & Bond, 1991] for an introduction to some of the research.

15

decision helps the designer predict the potential impacts on criteria, and tells him or her which criteria to consider in carrying out an evaluation. Inference in decisions means choosing alternatives based on evaluation against criteria. Automated inference can help the designer be consistent with normative ideals (i.e., overcoming human bias in judgement). For example, some tools offer spreadsheet-like services for experimenting with tradeoffs among alternatives and criteria in design [Boose, Shema, & Bradshaw, 1991; Shema, Bradshaw, Covington, & Boose, 1990]. Inference is also involved in evaluations, and tools can help with evaluation tasks such as calculating costs, and checking constraints. Answers to information requests of the form “why is this alternative chosen” often require inference from structure to behavior and function (“because this part does that action which achieves this function”) and inference from features of alternatives to requirements (“because it is the cheapest part that can achieve this requirement”). Roles of formal representation in decision making. Decisions can be structured with a range of representations, from hypertext webs to decision-theoretic models. At the least structured end of the continuum are formalisms such as IBIS [Potts & Bruns, 1988], PHI [McCall, 1986], QOC [MacLean, Young, Bellotti, & Moran, 1991], and DRL [Lee & Lai, 1991]. IBIS and QOC structure deliberative argument in general, where DRL is based on a decision making model. These sorts of representations are semiformal in the sense that they provide types of decision elements (“issue”, “option”, etc.) and relations (“supports”, “is a good alternative for”), yet they allow arbitrary human-readable text or graphics as nodes. Rationales encoded in these formalisms would answer many of the questions we encountered in the protocols—the questions about relevance mentioned above. A slightly more structured formalism would be useful for indexing cases of previous designs. A case library of designs is a set of existing designs that are variants on a theme, such as a drawer of plans for suburban houses. If the cases in the library could be recorded as a series of decisions, then the context of each decision can serve as a powerful index for case retrieval [Mark & Schlossberg, 1991]. The trick is to pick decisions that can be reused, or, to put it another way, to identify design contexts that will be similar to future contexts. We found questions in the protocols that we imagined could be answered by looking up similar decision contexts in a library and retrieving design cases in which those decisions were made. For example, when designing the battery case and considering the types of contacts, one designer said that he might take apart several of his household appliances to see how they implemented the contacts.➀ The most structured formalism for decision making is based on the mathematics of maximizing expected utility, given a network of conditional probabilities (see [Howard & Matheson, 1984] for an introduction to the literature on decision analysis). Such a network could tell one exactly how good a decision would be given information about the probabilities and utilities of consequences of the decision. None of the information questions or answers in the protocols would have required or utilized this degree of precision in decision making. However, even seemingly small design decisions can have amplified monetary consequences in products designed by large

16

teams. In such situations, the costs of representing design decisions with the rigor and completeness required of normative formalisms can be outweighed by the benefits.

3.3

Engineering Models and Tools

Increasingly, engineering information is produced, processed, managed, and distributed via computer-based tools. The experimental setting of the protocol studies precluded much actual tool usage. However, many of the kinds of information mentioned in the protocols are the domain of existing engineering tools, or plausibly, of advanced tools that are being explored in research laboratories (and have been proposed in the literature). Engineering tools operate on models. We identified several kinds of models as sources of knowledge for the protocol answers. They include CAD, simulation models, libraries of composable behavior models (e.g., components in electrical engineering, mechanisms in mechanical engineering), mathematical analysis models (closed form), and constraint models. Use of models in engineering. Naturally, engineering models are used to support engineering tasks such as analysis, simulation, testing, and so forth. In the context of the studies surveyed, we found that models were or could plausibly be used to answer many of the more difficult design questions. Questions about structure/form would be most directly answered by structure descriptions, such as feature-based CAD and component/connection models. Questions about the behavior of artifacts or parts would be naturally answered by simulation and analysis models. And questions about dependencies and constraints were often answerable only by invoking engineering models (or asking the original designer). Model-based inference. The striking thing about the use of engineering models as information sources is that answers are inferred from models in multiple ways. The same model that is capable of answering the question “what is the effect of this hypothetical behavior on this parameter” can be used to answer questions such as “What are the constraints on this parameter?” and “What are the alternative choices for this design parameter?”. The first question could be answered by running the tool associated with the model (e.g., propagating a change in values through a network of constraints over design parameters). The latter questions could be answered by reasoning about the model analytically (e.g., following links among constraints, and looking at the range of permissible values for a parameter). Most answers based on models were inferred rather than retrieved, with a few exceptions “What is the source of this constraint? [a manufacturer’s constraint?].” Heterogeneous formalisms. Every type of engineering model has its own formalism. Representations range from geometric primitives to lumped-parameter equations to composable component models. Algebraic constraints may be expressed in unidirectional form, or as sets of simultaneous equations. Many of the questions required crossing model type boundaries. For example, “What decisions were made related to satisfying this constraint?” and “What design choices are freed by a change in this input parameter?” require analysis of both constraint and decision models.

17

3.4

Implicit Domain and Background Knowledge

As became apparent during the analysis of the protocols, many answers are based on knowledge that is not explicitly captured in any medium, including informal documentation and engineering models. This is true even when experimenters cajole their subjects to constantly “think aloud” and when source materials have been carefully prepared. The observation also holds when considering what might have been recorded, even if it is not usually recorded. In these cases, the desired information is inferred by humans from background knowledge. Usage. Background knowledge is used when the information cannot be found in existing materials, for whatever reason. This was a common case in our survey. Kuffner and Ullman [1990] found that conjectures were frequently not confirmed, even though the experimenter was present to act as a surrogate information source (but not as a surrogate for the original designer). It appears that the designers were comfortable with making informed guesses, based on their own experience. We found two areas where information was heavily based on inference from implicit background knowledge. One is requirements: designers made guesses about what the requirements should be, based on a strong set of expectations. Some examples: “I don’t know if you expect some receipt of something like that for your transaction. ... so, and the receipts will be for parcels and registered letters.”➅ “pick and place [using] this area here. I'm assuming the [assembly robot] can do that.”➀ A second heavy use of background knowledge was in envisioning. Envisioning is mental simulation: imagining what would happen under hypothetical conditions. In domains without computable behavior models, envisioning was a common way for designers to predict what would happen or what would work in for a specified design. Goel & Pirolli [1989] report the ubiquity of “scenario immersion.” We found that envisionment was a source of knowledge for questions about behavior (especially relating to the operating environment), function, and validation. For example, the designers of the fast ATM envisioned failure modes: “What are the mistakes going to be in? ... The mistakes are going to be typing the PIN number; getting the card upside down, or the wrong card at the start...”➂ This required knowing how people interact with such machines, some of which is based on experience (e.g., entering passwords into computers) and some of which can be derived from common sense knowledge (that people will sometimes use the wrong card without noticing). Inference. In using background knowledge as an information source, the inference is done by people, of course. It is worth noting that in the protocols surveyed, when background or domain knowledge was used to answer a question, some inference was usually involved. The cases involving pure retrieval, like asking for the identity of a shape in a drawing, tended to be clarification questions. Formal representation and background knowledge.

18

In addition to domain knowledge, commonsense knowledge is in use throughout a design. We find these kinds of inferences are often made while envisioning the design and in explanations about the results of simulations. For example, a designer of the manufacturing equipment in study ➀ needed to know that people do not like to be made wet by the machine they are operating all day. The Cyc Project [Lenat & Guha, 1990; Lenat, Guha, Pittman, Pratt, & Shepherd, 1990] is producing formalisms for representing this kind of knowledge, but it would be hard to imagine that the Cyc KB would be comprehensive enough to cover the space of background knowledge used by designers. The problem of background knowledge, then, is an inherent limit to the capabilities of design rationale tools to provide the information used by designers.

4 Implications for the Design of Software Tools The major conclusion of our analysis is that the scope of design rationale is very broad—broader than the current literature on design rationale would suggest. We found many different kinds of information about design that could be part of explanations of why a design is as it is. Furthermore, if we examine the generic questions and answers in isolation, we are hard pressed to find properties inherent in individual information requests that would allow one to categorize them as relevant to design rationale or not. The implications of this analysis are not to revise the definition of design rationale; in a sense, the definition becomes irrelevant. The impact is on the design of software tools to support engineering. If design rationale knowledge is multifaceted, then tools that support the use of design rationale knowledge should not be limited to a particular facet, such as argumentation, but should support a broad base of design information. In this final section we will elaborate this conclusion using an example taken from the survey data, and follow with specific implications for the design of software support. Consider the following series of questions and answers from the damper redesign study ➆. D-35: What is the problem with [the existing equipment]? -> high force at high stroke generate[s] a lot of heat. D-37: Did you anticipate that problem? -> Yes....The heat problem is associated with the solenoid. ... -> We looked at various different kinds of actuators. Solenoid is just an actuator which gives you a certain force when passed it a certain current and voltage. ...there’s a tradeoff that we have so little power and we have to generate high force. The solenoid was chosen because with little power it can generate high force at various small stroke length. ... D-40: If I get a solenoid with very good insulation, that would be too expensive[?] -> No, the best you can get can go up to 175 degrees centigrade. The designer was given a changed requirement on the damper that, in the previous design, would have required running the damper at a higher temperature. He is trying to understand why the previous damper cannot be run at a higher temperature, and is told that the solenoid

19

employed would not tolerate the heat. In exploring alternatives, he wonders why the previous design did not use a solenoid that is adequately insulated. He hypothesizes that it might have been the cost. This hypothesis was incorrect. The dominant constraint was availability of such a solenoid. This example illustrates a few points worth highlighting. Rationale explanations are not naturally stored as such; they are constructed from information that is stored. For roughly half of the answers to the generic questions in the survey, answers were computed or inferred from more basic information sources. In these situations the information presumed to be available was not sufficient to answer the question via a look-up procedure; what is stored is not the same as what is needed to answer the questions. In the solenoid example, the designer needed to know why a solenoid was used in the previous design, and whether it could be reused in the new design. The answer is distributed over a series of responses to probing questions. The data to support those individual answers might be available online, in a catalog or parts database, but the rationale explanation is constructed dynamically. Explanations generated in response to information requests are robust over changes in dependencies. What if a solenoid of the desired type became available in the interim since the previous design? This is a common situation in modern re-engineering settings. If the explanations were generated from the pieces, in response to the designer’s series of queries, then the change in part availability would pop out and the new design opportunity could be uncovered. A design rationale is a self standing explanation, not a reconstruction of the design process. MacLean, Young, and Moran Moran [1989] argue that a rationale is constructed for its own sake, and that rationalization should not be confused with the design process itself. The preceeding solenoid question and the survey results illustrate the point. The original designers considered a series of alternatives in some order, but the answer given to the question was not a chronology of decisions. The explanation of why the solenoid was chosen is formulated in terms of functionality and constraints involving other parameters, not in terms of a hypothetical story of how the decision was made. Some questions in the protocols are about the decision or design process, such as “Did you anticipate that problem?” However, the information is requested to inform the new design, which is why it is helpful when the answer is formulated in terms that map to the new design problem. These claims, and the analyses of the previous section, support the following prescriptive implications on the design of computational support for design rationale information.

4.1

Capture and replay is incomplete for many types of questions.

Several design rationale tools provide the service of capturing design rationale information so that it may be replayed for consumers [Conklin & Begeman, 1988; 1991; 1991; Fischer, McCall, & Morch, 1989; Lee, 1990; 1991]. They emphasize the interesting problems of capturing knowledge that is often tacit, difficult to formalize, and generally incomplete. Given the range of information that is needed for coherently explaining the rationale for a design, it is not surprising that capture

20

is difficult. However, our analysis suggests that for many types of design information that are used, the critical computational service is not in the capture of complete explanations but in the search for pieces of information and the generation of answers from that information. A simple capture and replay of design process/history is inherently incomplete for the range of information requests observed. If many kinds of rationale explanations are inferred from more basic information, then a capture and replace approach will require the designer to make and record those inferences at design time. It is not inconceivable that knowledge about the properties of solenoids would be accessible on-line, perhaps retrieved from a catalog or parts database. However, the knowledge that the absence of high temperature solenoids is relevant to the design is inferred. In the example protocol, the original designer was available to explain the reasoning that links the “problem with the existing equipment” and the availability of solenoids. The original designers actually explored a space of several competing factors, including heat transfer. The path from overall functionality to solenoid characteristics is just one of many possible explanations. If rationales like this were to be captured and played back, then the space of plausible explanations would have to be anticipated by the design rationale producer. Furthermore, a capture and replay recording of rationale would have had to anticipate the assumptions that might change, and explicitly parameterize them (e.g., identify the availability of high temperature solenoids as a supporting argument for some issue). If the rationale only included the history of decisions as they actually occurred, the resulting record could end up reflecting an opportunistic hopping among constraints and alternatives (as has been observed in software design [Guindon, 1990]), and would capture only a fraction of the information requested about a design. We conclude that design rationale tools must go beyond the capture and replay paradigm if they are to cover the space of information needs we have identified.

4.2

DR support should be integrated with engineering models and tools.

Many of the sources of knowledge we analyzed were engineering data and models, often associated with inferences performed by engineering tools (section 3.3). The relationship between power, stroke length, force, and temperature in the solenoid example can be described by an analytic model, and computed with a spreadsheet. These relationships can be described informally (sketched graphs, boxes and arrows, etc.), but then they cannot be reused to compute answers to unanticipated questions. For domains such as human-computer interface, these kinds of models are rare. 8 And in all domains, designers choose to develop models for some phenomena and not others, because modelling takes time and effort. It would seem that the same will hold for building design rationales: some factors are worth making explicit. If software tools are to be successful in helping people obtaining and using design rationale, then they must be worth the effort. Ideally, the information needed for rationale should be captured as a byproduct of doing design, and the construction of rationale explanations should be 8

although HCI designs can be seen as embodying such models [Carroll & Rosson, 1991].

21

supported with automation. What kind of automation? If our analysis is correct, then the very tools that engineers use to do design can be the basis for the support of design rationale. We saw how several kinds of models and types of inference are involved in answering the generic questions. Tools exist, or are being developed, for operating on each kind of model (we are assuming the existence of knowledge based CAD and online access to engineering databases). To bring these tools into the service of design rationale support requires integration. The relationship between solenoids and other kinds of actuators might be found in a functionally indexed library of components and mechanisms. The information on properties of available solenoids might come from a different source. And the constraints on power, force, and stroke length might be found in a requirements document. To construct the entire explanation in response to user questions, a design rationale tool would need to make the connections among these information sources. To reconsider the question when assumptions change, the various tools would have to be invoked in a coordinated fashion. New functionality would be required for constructing explanations. But this, too, can get leverage from existing models and tools. For example, one way for a designer to communicate the intended functionality of a device in its operating environment (see questions 3.1 - 3.5) is to demonstrate the expected behavior in a simulation scenario. Augmented with explanation capabilities, a simulation system can be used in this way as a communication medium [Gruber, 1990; Gruber, 1991].

4.3

Dependency management in semiformal representation is an important service.

In the protocols surveyed, designers often ask questions or make conjectures about the effects of changes to the design. A few examples: “What will I have to do if I try to extend the force range from 500 to 750?”➆. “What can I do with more current which could not be done with less current?”➆ “Now if we changed GH, would we have to change anything else in the frame?”➀ “The other thing that would have to change would be the spring, like I said, you adjust the spring constant if you are damping this.”➀ “Supply chilled water temperature should be changed to 45 degrees F. All coils need to be recalculated at 45 degrees F.”➁ The essential information needed to support change in a design is dependency information. We found dependencies among the physical components, model parameters (including those describing assumptions about the environment), functional requirements, other requirements, evaluation criteria, and decisions. Designers are interested in dependencies because they want to understand the rationale for the existing design in order to change it. They want to reuse the good parts of the existing design and modify the bad parts such that the changes don’t undo the careful tradeoffs and choices that make up the design. The rich webs of interconnected dependencies are a source of complexity. Managing this complexity is a problem for designers and an opportunity for software support. Many of the dependencies can be expressed as operational constraints. One kind of constraint is a

22

unidirectional functional dependency determined by the formula for computing a parameter value. For example, the designer knows that the force produced by the solenoid is a function of current and voltage. Another kind of constraint is a limit on possible behavior, such as the maximum stroke length produced by the solenoid. A third kind of constraint is a requirement or satisficing criteria. For example, the choice of ABS plastic in the battery case design➀ was checked against the “UL flammability” safety requirement. When constraints are operational, that is, when they are formulated in a representation in which they can be automatically calculated, propagated, or checked, then the engineering tools that do the computations are powerful aids to dependency management. Furthermore, operational constraints can play a central role in explaining the rationale of a design [Baudin, Sivard, & Zweben, 1989]. Our observations about the broad scope of dependencies suggest extending dependency management services along two dimensions. First, when dependencies are not operational, tools can help elicit them. Much of the computational service provided by deliberation and decision support tools is in prompting the designer to enumerate and record possibilities, and providing external memory for them. The link types in representations of argumentation [Kunz & Rittel, 1970; McCall, 1986; Newman & Marshall, 1990] design space possibilities [MacLean et al., 1991], and decision making [Lee & Lai, 1991], are explicit representations of dependencies that are often implicit. When a designer wants to reconsider a decision or design option, he or she can browse the links in the dependency network. An important future direction for such tools is to augment them with more operational dependency management techniques, such as rule-based constraint checkers [Fischer, Lemke, Mastaglio, & Morch, 1990; Fischer et al., 1991; Fischer, McCall, & Morch, 1989], truth maintenance [Lubars, 1991], and mathematical modelling tools. Second, a comprehensive representation of dependencies is needed that can tie together the many kinds of dependencies and the inferences underlying them. One can view design in any of its many facets: as decision making, as argumentation, as search, as constraint satisfaction. Corresponding to many views of design are the many elements of a design description among which dependencies are important: design decisions, alternative design choices, requirements, other constraints, evaluation criteria, artifact structure, expected behavior, and assumptions about the operating environment. For each dependency path between elements there is a corresponding kind of inference in the design. For instance, dependencies between alternatives and evaluation criteria are the backbone of decision making, and dependencies between design parameters represent the paths of inference in constraint models. The ubiquitous design rationale question “what is affected by a change in this element?” can only be answered by a program to the extent that the program can represent the dependencies and invoke the reasoning services that correspond to dependency paths. To cover the space of dependencies found in our survey, a procedure for generating explanations must be robust with respect to the degree of formality of the design elements. For example, when no formal model of behavior is available, a weak explanation can be provided, such as “this parameter has something

23

to do with these decisions.” When more formal models are available, the generation procedure should compose the output of other tools, such as spreadsheet calculations or simulation results, into a coherent explanation. Such explanations are strong to the degree that they can give details about how or how much a change in one design element affects another. For example, conventional programming environments allow the programmer to change a fragment of code in a file and have the system determine the minimal set of files that have to be recompiled and relinked. An analogous service for design in other domains, considering a broad range of dependency types, would be extremely valuable.

Acknowledgements We are very grateful to our colleagues who generously allowed us to analyze their hard-earned data, much of which was previously unreleased. By offering us their data the authors of the protocols do not implicitly endorse our conclusions. Our analyses are not necessarily based on the identical datasets reported on in their publications. • Ana Cristina Bicharra Garcia very kindly offered her data and notes from her ongoing study of design rationale in civil engineering. • Catherine Baudin, Larry Leifer, Vinod Baya, Ade Mabogunje, Jody Gevins, and Jean-Charles provided us with their protocol data, processed segments, and inspiration on the damper redesign study. • Tom Kuffner was very helpful, sharing the protocol transcriptions and segmented questions from his study of the information requests of mechanical engineers. The original work was conducted by Tom A. Kuffner and David G. Ullman at Oregon State University under NSF grant DMC.87.12091. • Pete Pirolli and Vinod Goel graciously offered their transcribed and segmented protocols. This work was done at the U. C. Berkeley School of Education. • Victoria Bellotti and Alan MacLean, at EuroPARC, kindly provided their transcribed and segmented protocols. Conversations with Tom Dietterich, Bill Mark, Mark Stefik, Marty Tenenbaum, and Jay Weber were enlightening. Thanks for reviews and assistance with the analysis to Amr Assal and Jean-Charles Bonnet at Stanford, and to Tom Moran and the reviewers. Tom Gruber is funded by NASA, DARPA and Boeing Computer Services (NASA Grant NCC2537, NASA Grant NAG 2-581 under ARPA Order 6822, DARPA prime contract DAAA15-91-C0104 through Lockheed subcontract SQ70A3030R and Boeing Computer Services contract W297998).

24

References Baudin, C., Gevins, J., Baya, V., Mabogunje, A., & Bonnet, J.-C. (1992). The variable damper experiment. NASA Ames Research Center, Moffett Field, CA., Technical Report FIA-TR-9208. Baudin, C., Sivard, C., & Zweben, M. (1989). A model-based approach to design rationale conservation. IJCAI 89 Workshop on model-based reasoning, Detroit, pages 88-90, Boeing Computer Services. Bellotti, V. & MacLean, A. (1989). Transcription of: Two designers discussing a proposed design for a Fast Automated Teller Machine. Rank Xerox EuroPARC, Internal Technical Report, Technical Report AMODEUS RP6/DOC1. Boose, J. H., Shema, D. B., & Bradshaw, J. M. (1991). Knowledge-based design rationale capture: Automating engineering trade studies. In M. Green (Eds.), Knowledge Aided Design. London: Academic Press. in press. Boy, G. (1991). Indexing hypertext documents in context. Hypertext-91, San Antonio, TX. Carroll, J. M. & Rosson, M. B. (1991). Deliberated evolution: Stalking the View Matcher in Design Space. Human Computer Interaction, 6(3/4). Conklin, J. & Begeman, M. L. (1988). gIBIS: A hypertext tool for exploratory policy discussion. Proceedings of the 1988 Conference on Computer Supported Cooperative Work (CSCW-88), Portland, Oregon, pages 140-152, ACM. Conklin, J. & Yakemovic, K. B. (1991). A Process-oriented Paradigm for Design Rationale. HCI, . forthcoming. Fischer, G., Lemke, A. C., Mastaglio, T., & Morch, A. I. (1990). Using Critics to Empower Users. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI'90), pages 337-347. Fischer, G., Lemke, A. C., McCall, R., & Morch, A. I. (1991). Making Argumentation Serve Design. Human Computer Interaction, 6(3/4). Fischer, G., McCall, R., & Morch, A. (1989). JANUS: Integrating Hypertext with a Knowledgebased Design Environment. Hypertext 89, pages 105-117. Fischer, G. & Stevens, C. (1990). Information Access in Complex, Poorly Structured Information Spaces. University of Colorado, Technical Report CU-CS-461-90. Garcia, A. C. B. (1991). Selections from Request for Rationale materials in HVAC design. Stanford University, Department of Civil Engineering. Personal Communication, November, 1991. Garcia, A. C. B. & Howard, H. C. (1992). Acquiring design knowledge through design decision justification. Artificial Intelligence in Engineering Design and Manufacturing, 6(1). Goel, V. & Pirolli, P. (1989). Motivating the notion of generic design within informationprocessing theory: The design problem space. AI Magazine, 10, 1, pp 18-38. Gruber, T. R. (1990). Model-based Explanation of Design Rationale. Knowledge Systems Laboratory, Stanford University, Technical Report KSL 90-33. Also appears in Proceedings of the AAAI-90 Workshop on Explanation. Gruber, T. R. (1991). Interactive acquisition of justifications: Learning "why" by being told "what". IEEE Expert, 6(4):65-75. An earlier, longer version appears as: Gruber, T. (1990). Justification-based Knowledge Acquisition. Knowledge Systems Laboratory, Stanford University, Technical Report KSL 91-17.

25

Gruber, T. R., Boose, J., Baudin, C., & Weber, J. (1991). Design rationale capture as knowledge acquisition: Tradeoffs in the design of interactive tools. Machine Learning: Proceedings of the Eighth International Workshop, Chicago, pages 3-12, Morgan Kaufmann. Guindon, R. Sofware Design Tasks as ill-Structured Problems, Software Design as an Opportunistics Process, MCC Technical Report Number STP-214-88, 1988. Guindon, R. (1990). Knowledge exploited by experts during software system design. International Journal of Man-Machine Studies, 33:279-304. Howard, R. & Matheson, J. E. (Eds.). (1984). Readings on the Principles and Applications of Decision Analysis. Menlo Park, CA: Strategic Decisions Group. Kato, T. (1986). What ‘question asking protocols’ can say about the user interface. International Journal of Man-Machine Studies, . Kuffner, T. A. (1990). Mechanical Design History Content: The Information Requests of Design Engineers. Master's thesis, Oregon State University. Kuffner, T. A. & Ullman, D. G. (1990). The information requests of mechanical design engineers. Design Studies, 12(1):42-50. Kunz, W. & Rittel, H. (1970). Issues as Elements of Information Systems. Center for Planning and Development Research, University of California at Berkeley, Technical Report. Lee, J. (1990). SIBYL: A tool for managing group decision rationale. Proceedings of the conference on Computer Supported Cooperative Work (CSCW-90), Los Angelos, pages 79-92. Lee, J. & Lai, K.-Y. (1991b). What's in Design Rationale. Human Computer Interaction, 6(3/4). Leifer, L., Baudin, C., Gevins, J., Toye, G., Baya, V., & Mabogunje, A. (1992). A Tool and Protocol for Developing Reusable Design Knowledge. First AIAA Aerospace Design Conference. Manuscript presented at workshop. Lenat, D. B. & Guha, R. V. (1990). Building Large Knowledge-based Systems: Representation and Inference in the Cyc Project. Menlo Park, CA: Addison-Wesley. Lenat, D. B., Guha, R. V., Pittman, K., Pratt, D., & Shepherd, M. (1990). Cyc: Toward Programs with Common Sense. Communications of the ACM, 33(8):30-49. Lubars, M. D. (1991). Representing design dependencies in an issue-based style. IEEE Software, , July, 81-89. MacLean, A., Young, R., Bellotti, V., & Moran, T. (1991). Questions, Options, and Criteria: Elements of A Design Rationale for User Interfaces. Human Computer Interaction, 6(3/4). MacLean, A., Young, R. M., & Moran, T. P. (1989). Design Rationale: The argument behind the artifact. CHI-89, Austin, TX. Mark, W. & Schlossberg, J. (1991). Interactive acquisition of design decisions. Knowledge Acquisition for Knowledge-Based Systems, Banff, Canada. McCall, R. (1986). Issue-serve systems: A descriptive theory for design. Design Methods and Theories, 20(8):443-458. Newman, S. E. & Marshall, C. C. (1990). Adventures in Representing Arguments: Lessons and Issues for the Design of Hypertext-based Representational Tools. Xerox PARC, Technical Report. Pirolli, P. & Goel, V. (1989). Three Transcripts of design protocols: (1) Architecture, (2) Mechanical Engineering, (3) Instructional Design. UC Berkeley, School of Education. Personal correspondence, December, 1989. Potts, C. & Bruns, G. (1988). Recording the Reasons for Design Decisions. Proceedings of the 10th International Conference on Software Engineering, pages 418-427.

26

Shema, D., Bradshaw, J., Covington, S., & Boose, J. (1990). Design knowledge capture and alternative generation using possibility tables in CANARD. Knowledge Acquisition, 2(4):345364. Sticklen, J. & Bond, W. E. (1991). Function reasoning and functional modelling. IEEE Expert, 6(2), April, 20-47. (A special issue on functional reasoning and modelling).

27

Appendix 1: Vocabulary for describing design information The following set of terms were used to create the abstract, generic questions from the specific information requests. In imposing this vocabulary we are consciously biasing the generalization from the examples, so that the resulting generic questions are comparable within a single, coherent model of design. We do not claim that all design moves fit nicely into this vocabulary, although it serves a descriptive purpose for the data we analyzed. The set of definitions is based on a simple ontology of design knowledge presented in [Gruber and Russell, 1990], and was extended to accommodate the protocol data. behavior -- something an artifact might do, in terms of observable states or changes. If observable states are specified in terms of the values of physical parameters P1...P n, then behavior descriptions are relations over these quantities, where Pi ’s may include initial conditions and time. constraint -- an expression specifying a limit on the range of values of one or more parameters, possible in terms of the values of other parameters. Constraints may be qualitative or quantitative, and may be specified in probabilistic terms. constraint model -- a set of constraints which describe the interrelationships among parameters in a design. A constraint model may be unidirectional, in which case the interrelationships are dependencies that can only be computed in one direction. Another example of constraint model is a set of simultaneous differential equations representing time-varying behavior of multiple parameters. decision -- a choice among alternatives with respect to utility. A specification of a design decision includes the set of alternatives, relevant criteria, and information used to compute the utilities of the alternatives in terms of the criteria (e.g., could be as simple as reasons for and against). dependency model -- a specification of the dependencies among design elements, such as parameters, requirements, other constraints, and decisions. A unidirectional constraint model is a specialization, where the dependencies are over parameters. design alternative: a proposed version of an artifact specification, or fragment of a specification (e.g., selection of a part or material), for comparison with other alternatives. Decisions are choices among alternatives. design criteria -- desiderata specified in terms of measurable properties of an artifact that can be used to evaluate the utility of its design. Example design criteria include “minimize part count,” “minimize cost,” and “be manufacturable.” Requirements of the function of device are usually stated as constraints rather then criteria.

28

design parameter -- a parameter that the designer may vary, modulo constraints on it. In purely parametric design, all designer choices are in terms of the values of predetermined parameters. In other kinds of design, new parameters can be introduced by the decisions to use mechanisms or structures. environmental parameter -- a parameter which is constrained by an assumption or specification about the operating conditions or environment in which the device is expected to work. (eg an input of a black box) functional requirement -- a specification of the behavior that the designed artifact must achieve (or prevent) in a given operating environment. May be operational (i.e. in terms of evaluable constraints over the values of measurable parameters) or nonoperational (i.e. informal description of intended effect, possibly as a demonstration scenario). functional parameter -- a parameter which is constrained by an assumption or specification about the required functionality of the device (eg, an output of a black box) part -- used colloquially in this study to refer to components, subsystems, substructures, course sections, interface features, etc. ... named elements in the design that are composed or connected in some way. parameter -- a variable representing any quantitative or qualitative property of the designed artifact or its interface to the operating environment. Parameters are used for specification of intended functionality, other constraints in the requirements or induced during the design, input / operating conditions, and descriptions of device behavior. “Features” of parts, such as materials and counts, are parameters, as are any objective properties which are the basis for, or object of, design decisions. We use the general term “parameter” instead of terms such as “features,” “quantities,” “inputs” and “outputs” in an attempt to be domain neutral. requirements -- prescriptions concerning the structure, behavior, and/or function that the designed artifact must satisfy. Requirements are often specified as constraints and design criteria. requirement constraint -- a constraint that specifies required limits or values to achieve/prevent, as part of the requirements. scenario of use -- a specification of the intended environment in which the artifact operates and a characterization of how it interacts with the environment

29

specification -- explicit representation of something. Can be prescriptive (what something should be or do), or descriptive (what something is, in appropriate detail). For “input spec” or “design spec” we prefer to use the term “requirement.” structure -- the physical and/or logical composition of an artifact, typically in terms of the composition of parts, and connection topologies. A structure description is about the thing itself and not what it does. Usually a structural description is the basis for realizing the artifact and is depicted with schematics or drawings.

Appendix 2: Question Types, Answer Types, Example Questions This appendix contains the set of generic questions derived from the statements, conjectures, and questions present in the protocols surveyed. Each generic question is followed by an example segment from a protocol, and then by one or more generic answers. Each answer is labelled with the keywords denoting sources of knowledge (defined in Section 2.6), whether the answer is inferred or looked up, and formality of the representation. The entire dataset include several segments for each generic question; we only show a representative. Generic Question Example from protocol [one of several] • form of answer ..................................................................... sources of knowledge inferred

1

formality

Requirements 1.1

What are the given requirements?

D-1: What are the design requirements given initially? • Retrieval of requirement statement .............................................................. DesDoc Lookup

1.2

Is this constraint a requirement?

PATM-142: Register letter? You mean make it a register mail? [Should the PATM provide for registering a letter?] K&U-2141: We’re not worried about color, for instance. • retrieval of requirement statement ............................................................... DesDoc Lookup • requirement inferred from background knowledge by designer ........................................................................................................ Human Lookup

1.3

Semiformal

Informal

Can I assume this fact about the operating environment?

• permission to make assumption (or not) ...................................................... Human Lookup

1.5

Semiformal

Give more detail about this parameter of the operating environment.

ID-106: Is [the class schedule] already pretty fixed? • clarification of ambiguity or missing information in text or graphic .............................................................................................. Human Lookup

1.4

Semiformal

Semiformal

What are the requirement constraints on this parameter?

K&U: I’ve got 11 1/2 inches, it appears, on the interior of this frame. • infer detail of explicit requirement ................................................. Human,ReqSpec Inferred Semiformal

1.6

Is this parameter constrained by external requirements?

K&U: On my table here I would mount a [microswitch], let’s say here. [Let] me think, would it matter [where]? • retrieve and check external requirement ..................................................... ReqSpec Inferred Formal

1.7

What is the expected behavior of this artifact in the scenario of use?

30

K&U: Does (the pivot arm) flip all the way out, or (are there) two positions? • envisionment or demonstration simulation of the scenario ................................................................................... Human,Envis,Demo Inferred Semiformal

1.8

Should I assume that this functionality is required?

PATM-193: I don’t know if you expect some receipt of something like that for your transaction. ... so, and the receipts will be for parcels and registered letters. • inferred or assumed requirement ..................................................................... Envis Inferred Semiformal

1.9

Can I modify this requirement?

D-21: Can I change 80 to -40 in the temperature requirement? • an analysis of the impact of the change on constraints ....................................................................................... DesDoc,Constr Inferred Formal

2

Structure/form 2.1

What are the components?

D-6: What are the components in the suspension diagram? • examination of informal sketch .................................................................... DesDoc Lookup • list of parts from K-based CAD ................................................................... KB-CAD Lookup

2.2

Semiformal Formal

What class of device or mechanism is this part?

D-9: Is the suspension system in the diagram a contemporary suspension system? • classification of device or mechanism in standard engineering terminology (eg classes of control systems) ......................................................................................... Human,KB-CAD Inferred informal

2.3

What is the geometry of this part? (qualitative)

D-16: Is the shape of the volume a cylinder? • classification of part in terms of standard geometric shapes .............................................................................................. CAD,KB-CAD,DesDoc Lookup

2.4

What material is this part made of?

K&U: Is this steel or aluminum? • specification of material .............................................. Handbook,DesDoc,KB-CAD Lookup

2.5

Formal

What are the known limitations (strengths) of this part/material class?

K&U-2141: But let’s start with ABS. It’s a very nice material to use; it molds well, it’s economical..also dielectric...is very good... • list of relevant features of part or material ................................................. Handbook Lookup

2.8

Semiformal Formal

What are the locations of parts, connections, etc (for constraint checking)?

HVAC C1.2: Steam and Chilled Water utilities are missing and connection points to these utilities need to be determined. Please define the points of utilities connection and some specific routings. • the requested data ............................................................................................. CAD Lookup

2.7

Formal

How do these components interface?

K&U: How does this pivot arm seat in (the mounting brackets)? • informal sketch showing connections between components ................................................................................................. DesDoc ? • a CAD model of connections ..................................................................... KB-CAD Lookup

2.6

Formal

Formal

What affects the choice of artifact components?

PATM-201b: [when deciding what interface technology to choose for the PATM.] I don’t know what’s the year its going to be there [when it’s implemented], and what’s the technology that’s going to be applied. ...would like to have more exposure, at least, as little, as little money spent as possible. I would assume that. If

31

so, maybe we use mostly conventional [interface technology], what’s a better choice... with so many changes... equipment, devices available on the market. • list of constraints and criteria affecting selection .......................................... Human Lookup

informal

Not Rec.

3

Behavior/operation: what does it do 3.1

What is the behavior of this parameter in the operating conditions

D-49: What is the input to the Damper from the computer onboard the car? • qualitative value of parameter .......................................................... DesDoc,Human Lookup informal • simulation of the operating environment ..................................................... SimMod Inferred Formal

3.2

What is the behavioral interaction between these subsystems?

D: How is the output of the suspension system used by the damper? • simulation of the two subsystems ............................................................... SimMod Inferred Formal • equations describing the interaction between subsystems .................................................................................................. SimMod Inferred Formal

3.3

What is the range of motion of this part?

D-20: How does the joint between the damper arm and the car wheel arm behave, like this {angled 60 degrees, shown with pen and finger} or like this {180 degrees}? • a specification of the range of a parameter ................................... DesDoc,SimMod Lookup

3.4

Semiformal

What is the cause of this behavior?

FATM-275: Why could queues be so long? • causal explanation of behavior .................................................... ReqSpec,SimMod Inferred Semiformal

3.5

What are the expected failure modes in the scenario of use?

FATM-319: What are the mistakes going to be in [ using the ATM ] ...? • Envisionment or demonstration simulation of the scenario ............................................................................................. Human,Envis Inferred Semiformal

4

Functional Composition 4.1

What is the function of this part in the design?

K&U: [Is this for mounting or for strength?] I don’t see any reason, tell me if I’m wrong, that this has to be so high. • a label for the function of the part ............................................................. Handbook Lookup informal • a simulation of the role of this part in overall behavior, guided by the set of possible functional roles ............................................................................................ FunMod,SimMod Inferred Formal

4.2

What is the function of a feature of a part in the design?

K&U: Why the 2 inch tubing? • a simulation of the role of this part in overall behavior, showing the behavior that arises from the specified feature ........................................................................................ SimMod Inferred Formal

5

Hypotheticals 5.1

What happens if this parameter changes to this new value?

D-13: what will happen if I try to extend the force range from 500 to 750 pounds? • automated experiment, verifying parameter and monitoring changes in other parameters with respect to constraints ................................................................................ SimMod Inferred Formal

5.2

What is the effect of this hypothetical behavior on this parameter?

32

D-36: Does high force at high stroke length generate a lot of heat? • a quantitative value for the specified parameter ......................................... SimMod Inferred Formal • a qualitative value for the specified parameter ................................................ Envis Inferred Semiformal

5.3

Adapt equation to this changed parameter and recompute

D-12: relate the 500 pounds to the suspension system equation. • a reformulation of the equation accommodating the new parameter value .................................................................................. SimMod Inferred Formal

5.4

What will have to change in the design if this parameter changes to this new value?

K&U: The other thing that would have to change would be the spring, like I said, you adjust the spring constant if you are damping this. D-14: what will I have to do if I try to extend the force range from 500 to 750 pounds? • analysis of design parameters that depend on the changed parameter ........................................................................ DepMod,Constr Inferred Formal

6

Miscellaneous Dependencies 6.1

What are the known dependencies among the parts?

D-56: How are the different parts in the picture dependent on each other? • a list of design dependencies ..................................................................... DepMod Inferred Formal

6.2

What are the constraints on this parameter?

D-15: What was the stroke length limit in the previous design? • an expression specifying a constraint on a formal parameter ........................................................................................ DesDoc,Constr Lookup

6.3

Formal

Is this parameter critical (involved in a dominant constraint)?

D-25: Will dealing with the temperature requirement be a critical part of the solution? K&U-2143: The ... material requirement of the case, let’s just say, are, are not very severe. • a sensitivity analysis of the impact of changes in the parameter on constraints ................................................................ DesDoc,Constr Inferred Semiformal

6.4

How is this subassembly related to this parameter?

D-27: How does the electronics relate to the forces? • a causal or constraint-dependency path between a model of the subassembly and the parameter ............................... DesDoc,Constr Inferred Formal

6.5

What is the source of this constraint?

D-39: Is the temperature limit on the solenoid a manufacturer's constraint? • characterization of where the information came from ...................................................................................................................... Human Lookup

7

Semiformal

Constraint checking 7.1

Is this constraint satisfied?

HVAC Dwg E3: Communications closets need to be stacked over each other by floor. Do you agree? -> Yes. Original specifications document allowed for communications to run in dedicated space at alternating sides. Current design document has 4 dedicated vertical chases which accommodate the buildings communication needs. K&U-2140: I’m going to have no problem with uh, UL flammability, of any of this with my ABS [plastic], polycarbonate, or even nylon. • a confirmation of a checked constraint ............................................ SimMod,Constr Inferred Formal

33

• a justification of decision about the structure in question ....................................................................................................... History Inferred Semiformal

7.2

Does this structure have this behavior which violates this constraint?

HVAC AO.1: Loading Doc area is adjacent to windows for offices. Are these windows operable? [if windows near Doc area are operable (could be opened), then exhaust fumes from trucks would enter building.] • a confirmation about the feature of the structure ......................................... KB-CAD Lookup

7.3

Formal

What are the known problems with this design?

K&U: Does the original flipper-dipper work (well)? • results of a verification analysis ................................................. ReqSpec,SimMod Inferred Formal • data on reliability, customer satisfaction, etc. ......................................... UsageData Lookup Semiformal

7.4

Would a part with this functionality satisfy this constraint?

D-40: Would a solenoid with good insulation be very expensive? • List of alternate parts .................................................................................... FunLib Inferred Formal

8

Decisions 8.1

What are the alternative choices for this design parameter?

K&U-2140: [What materials could I use for the case?] ... ABS...polycarbonate... • a set of possible choices of materials, parts, or parameter values for a design parameter .......... Human,Handbook,StdPrac,Cases Lookup

8.2

D-29: How is ground clearance related to and addressed in the previous design? • an expression specifying a constraint on a formal parameter ..................................................................................................... History Lookup

8.3

Semiformal

What decisions were made related to satisfying this constraint?

D-24: Did they do anything to specifically deal with low temperature? • list of decisions explicitly linked to the constraint .......................................... History Lookup

8.5

Formal

What was an earlier version of the design?

FATM-240: Have we still got that thing [the earlier design], we have, haven’t we? [presses whiteboard button to backup to previous screen with design document] • Reveal previous design state ....................................................................... History Lookup

8.4

Semiformal

What decisions were made related to this parameter?

Semiformal

Which parameter, requirement, constraint, or component should be decided first?

D-28: From the following: damping force range, stroke, ground clearance, max volume, temperature range and electronics, what is most important to attack first? • a “best first” strategy or plan for making design commitments ............................................................................................. Strategy Inferred Semiformal

8.6

What design choices are freed by a change in this input parameter?

D-34: What can I do with more current which could not be done with less current? • list of decisions explicitly linked to the constraints involving the parameter ................................................................. DepMod,Constr Inferred Formal

8.7

What alternative parts that satisfy this constraint could substitute for this part?

D-38: What other device could be used instead of solenoid, so that heat would not be a problem? • List of alternate parts .................................................................................... FunLib Inferred Formal

8.8

Where did the idea for this design choice come from?

ID-87a: This is all being pulled right out of the table of contents [of previous existing course]. • List of sources ............................................................................................. DesDoc Lookup

8.9

Informal

Why use these design criteria?

34

HVAC: Why do you list the Uniform Building Code, 1985 Edition, when we should be designing to the 1988 Edition? • justification about design criteria selection .................................................. DesDoc Inferred Informal

8.10 What evaluation model is used to make decisions? D -- [ Which alternative should be chosen? ] [we use a] ... a side by side comparison of wedge and lever designs. Lever design requires less volume and weighs less than the wedge design as well. • description of decision evaluation process ..................................... Human, DesDoc Inferred Informal

9

Justifications and evaluations of alternatives 9.1

Why this design parameter value?

HVAC DwgA3.1: Exhaust stacks are same height as building. Why? Height of stacks discussed with [person] and initially agreed to be adequate. Wind tunnel tests will confirm. • result from authority ...................................................................................... Human Lookup

9.2

Why is design parameter at value V1 instead of normal value V2?

HVAC 4.0-32: Why are you using 15 air changes per hour? Has it been calculated? 12 is fairly normal. Agreed. Facilities manual specifies 15 air changes per hour. Note 6’ fume hood in single lab will require 30 changes per hour. • external requirement ..................................................................................... History Lookup

9.3

Semiformal

What is person P’s evaluation of these alternatives?

HVAC 4.0-39: What is [person]'s view on this point? • replay of other person’s evaluation ............................................................... History Lookup

9.5

Semiformal

Why was this alternative chosen over that alternative?

K&U-2142: [choosing a battery case material] It [ABS] doesn’t absorb moisture, nylon does a little bit. HVAC 4.0-39: System options: we can discuss system alternatives at future project meetings. Right now, I am fairly convinced that should install a constant volume system for labs rather than VAV unless there is some strong argument against constant volume. Yes, we must discuss. OAP is convinced VAV is safer, cheaper like cycle cost and allows additions and modifications and modifications with less disruption. ID-94: Why “yes” and “no” to those things [topic choices for instruction]? Because they are something that don’t have to be learned right now but they can be learned later. And the logon and logoff sheet properties, I, I wouldn’t each teach the secretaries. That’s none of their business. They have no need for that information. • list criteria on which one alternative is better than another ..................................................................................................... Decision Lookup

9.4

informal

Semiformal

Why not try this alternative?

D-41: Why didn't you use the solenoid with better insulation? • external requirement violation ....................................................................... StdPrac ? • list of constraints impacted by alternative .................................................... DepMod ? • evaluation of alternatives against relevant criteria ....................................... Decision ?

? ? ?

10 Justifications and explanations of functions 10.1 Why is this function provided?

35

HVAC 4.0-26: There is no requirement for non-potable water in the building. Why are we still requiring it? We were not informed that non-potable water was not required. It is standard practice to provide non-potable water to laboratories. Does [person] have any comments? • standard practice implies requirement ........................................................ StdPrac Inferred informal

10.2 Why is this function not provided? HVAC 4.0-19: Vibration: because the building will not be structured to handle highly vibration sensitive equipment, this should be explained to the occupants very carefully. The majority of highly vibration sensitive equipment has been located in the basement. The project structural engineer will come to design meeting and discuss with faculty [the user]. • the function is achieved some other way .......................................... Demo,History Lookup

Semiformal

10.3 Why can’t the current design achieve this new value of this functional requirement parameter? D-35: Why can't the existing design go up to 750 pounds? • analysis of limits of existing behavior .............................................. Constr,SimMod Inferred Formal

11 Validation explanations 11.1 How is this requirement satisfied? HVAC DwgA4.1: The area over the Earth Sciences loading dock is covered. How do you plan to exhaust truck fumes from that area? In addition, how do you prevent fumes from entering the building? Ans: Area is covered but open on one side. Exhaust air from subsidiary air plant room is exhausted into loading dock area pushing fumes out of dock to open air. • Explanation of how structure and behavior satisfies the constraint .................................................................................... Envis,SimMod Inferred Semiformal

11.2 How is this function achieved? D-47: How does your system achieve damping? • explanation of how function is achieved by the structure and behavior ...................................................................... SimMod,Envis Inferred Semiformal

11.3 How is this functional requirement achieved? FATM-228: How do we put the PIN number in? You’ve got... the keypad is here (points to the right of the diagram), so there is a keypad you haven’t drawn yet. • Explanation of how requirement is achieved by mechanisms enabled by structures ............................................... DesDoc,SimMod Inferred Semiformal

11.4 How will this part be maintained? HVAC: How do you get replacement mechanical and electrical equipment into the building when the loading dock is so far from the mechanical and electrical area? General replacement equipment will be carried on trolley conveyed along corridors from load dock. With no chillers or boilers the air handling units are the largest items (and they are located on the roof). A strategy for replacement /addition of plan must be agreed during DD. • description of maintenance procedure ............................................ Envis,Human Lookup

Semiformal

12 Computations on existing model 12.1 Compute a parameter value given other parameters D-30: What is force response with less than 30 degrees phase lag for 10 hertz sine wave input? • a parameter value ....................................................................................... SimMod Inferred Formal

36

12.2 What are the trajectories of parameters D-43: What are the independent effects of the interaction between force, stroke, current and temperature? • graph of behavior trajectories ...................................................................... SimMod Inferred Formal

13 Definitions 13.1 What does a term in the documentation mean? D-10: What do you mean by on-state? D-17: Are you talking about an envelope when you say maximum volume of the damper? • clarification of ambiguity in text or graphic .................................................... Author Lookup • context independent definition of term used formally .................................................................................................................... Glossary Lookup • definition of context-dependent usage of term ............................................ DesDoc Lookup

informal informal informal

14 Other design moves 14.1 Search for information expected to be in documentation, eg, equation or diagram. D-4: Is there a diagram which models the suspension system? D-11: Where is the force equation in terms of the damper and the dashpot? D-53: Where is the metal disk? FATM-240: Have we still got that thing, we have haven’t we? (presses whiteboard button to reveal previous screen.) • Search document index and retrieve requested information, perhaps incompletely

14.2 Change this requirement constraint and update design. HVAC Supply chilled water temperature should be changed to 45 degrees F. All coils need to be recalculated at 45 degrees F. -> Noted. Coils have not yet been sized at this stage of the design. • The detail decisions mentioned by this requirement have not yet been made. Will post the requirement for future and note additional design steps necessary. .................................................................................................. Strategy Inferred Formal

14.3 Have all the arguments for/against this alternative been checked? FATM-299: What other reasons? Any other reason s [to not use this alternative]?... That seems to be about it, doesn’t it? Yes. [we’ve checked them all.] • verify that all arguments for or against the alternative have been addressed

37