INTERNATIONAL CONFERENCE ON ENGINEERING DESIGN ICED 99 MUNICH, AUGUST 24-26, 1999
TOWARD A PRAGMATIC ONTOLOGY FOR PRODUCT DEVELOPMENT PROJECTS IN SMALL TEAMS Özgur Eris1, Poul H.K. Hansen2, Ade Mabogunje1, Larry Leifer1 1 Center for Design Research, Stanford University, USA 2 Department of Production, Aalborg University, Denmark Keywords: Empirical studies, classification, research cumulation,
1
Introduction
For a number of years we have been engaged in empirical studies of product development projects with particular focus on coordination and collaboration. Increasingly, we have been troubled by the apparent lack of cumulative quality in such studies. In addition, we find that results obtained in one setting are rarely transferable to other settings. The encountered difficulties can partly be explained by the complexity of the phenomena, and partly by the high variation in research methodologies and research settings. In this paper we shall discuss the advantage of applying an ontological approach to overcome some of the difficulties, and also, present a tentative ontology. At this initial stage, our focus and findings are limited to qualitative issues.
2
Why an Ontological Approach?
In a recent paper on design research methodology, Dorst [1] states, “Design is a maddeningly difficult subject to capture in empirical research”. We believe it is possible to extend this line of thinking by stating that the difficulties experienced during design research in capturing activity and process lead to a situation in which it is “maddeningly difficult” to cumulate the research findings. Many empirical studies are used solely within the research group which has conducted the study, and often, comparisons with other studies are based on frail foundations. The major limiting factor in generalizing methods and results has been the absence of a unifying set of variables relating one study to another. We postulate that an ontological system for identifying and classifying variables can alleviate these difficulties. An ontology can be defined as the specification of a conceptualization [2]. That is, an ontology is a description (like the formal specification of a program) of the objects, concepts, entities, and relationships that can exist in some area of interest. A conceptualization is an abstract, simplified representation of the area of interest. In our context, an ontology becomes a specification medium accessed by researchers for making ontological commitments. The goal is to communicate consistently in a domain of discourse without necessarily operating on a globally shared theory. Specifically, developing such a system entails; (1) identifying and categorizing variables encountered in design research, (2) using those variables in disclosing similarities and
differences between laboratory and real world design activities, (3) based on those differences, alerting researchers to possible sources of variance that may contaminate or even negate their research findings. Ontologies are often equated with taxonomic hierarchies of classes, however, ontologies need not to be limited to these forms. Within the field of product development, several attempts have been done to apply a taxonomic approach, e.g. Ullman [3] and Duffy [4]. Our initial attempts at composing a framework were built on a similar taxonomical understanding. However, after preliminary analysis of data, we decided that an ontological approach would give us more freedom in achieving our goal due to the looser and pragmatic structure it entails. Our methodology for constructing the ontology consists of: • Discussing the purpose and appropriateness of applying an ontological approach to product development projects in small teams • Conducting literature review of bases, purposes, and methods of identification and classification in other sciences • Formulating tentative ontological frameworks, conducting internal validations, and making the frameworks accessible to researchers • Discussing the frameworks with colleagues from related fields such as engineering design, industrial engineering, civil engineering, mechanical engineering, and artificial intelligence. • Developing criteria and evaluation systems for testing the validity, utility, and reliability of the proposed frameworks In this paper we will discuss the first three steps.
3
Assumptions and Scope
We postulate that the majority of product development efforts are carried out in teams of 3-8 persons, and define that as the product development team size. However, we will regard the teams as open systems with relations to other teams, organizational functions, customers, etc. It is helpful to envision two separate entrances to the ontology in order to distinguish between the observable product development project and the methodological approach of the researcher: (1) Product Development Projects, and (2) Research Methodology. Our primary focus is the product development project. We will deal with the research methodology entry in a separate paper. We identified four categories under the Product Development Project entry: • Project Input and Character • Product Development Process • Project Output and Character • Project Phases The main categories and dimensions of the initial ontological framework as a whole is represented hierarchically in Figure 1.
Figure 1. The main categories and dimensions of the initial ontological framework Product Development In Small Teams
Product Development Projects
Research Methodology
Project Input and Character
Actors
Product Development Process
Activities
Information
Project Output and Character
Physical Artifact
Project Phases
Environment
The first three categories under product development projects can be viewed simply as inputprocess-output categories. We assume that a generic product development project can be divided into two overall project phases: A conceptual design phase and a product realization phase. The distinction between these phases is significant both in academia [5] and in practice, as can be observed in the functional division of most firms. In the following sections, we will discuss the “Product Development Process” category in detail. We regard it to be the key component in the ontology, and also, to be one of the most ambiguous issues dealt by researchers. We have no intentions of coming up with one unifying theory of product development processes, however, we believe it is necessary to present a categorization of the process dimensions in order to introduce a degree of consistency.
4
The Product Development Process category
We applied the methods outlined earlier in section 2.1 in order to construct the initial classification structure of the Product Development Processes. We discussed the appropriateness and potential benefits of our approach, and then constructed the structure based on our own experiences, existing design theories and studies, and the knowledge gained during the application of a computer-based project simulation tool. We kept the initial classification structure as simple as possible so that the identified dimensions would be pragmatic as well as distinct. This lead to the following five overall dimensions: Actors, Activities, Information, Physical Artifact, and Environment. We felt somewhat uncomfortable with basing the structure on our experiences and views as that would be imposing a structure rather than letting the structure “emerge” out of data. However, we decided that that was the most appropriate, and perhaps the only, starting point. Our assumption was that whatever part of the structure we might end up imposing which might clash with data would eventually be refined or phased out during iteration. For the sake of limiting the occurrence of any such clashes, we made the structure relatively general. As for data, we relied on the findings of 19 different empirical studies. Each study makes up a chapter of Analyzing Design Activity [1], the resulting publication of the second Delft Protocols Workshop which was held in 1994. The data prepared for analysis at this workshop consist of audiovisual records of a three-person team consisting of experienced designers and an individual designer separately designing a typical industrial design product in two hours, a
“fastening device” that should allow a given backpack to be fastened to a mountain bike”. The 19 participating research groups were given videotapes and transcribed protocols of the data six months in advance of the workshop. Each research group then conducted analysis independently, and published its methods and findings. We reviewed the contributions in terms of product development process dimensions and tied their terminology to our initial classification. During the review process, we were able to identify 124 variables with a large number of different sub-attributes. Less than 15 of these variables were duplications, the most frequent duplication being “analyzing” with 5 occurrences. We did not encounter any significant problems in mapping the 124 variables into our initial structure. However, given the specific context of the design exercise under analysis, some overall dimensions such as organization were not encountered. We also reviewed and borrowed from the developers of a computer-based project simulation tool, Virtual Design Team [6]. The developers have gone through a comparable classification process to define and test their simulation dimensions. Their experiences were a natural step in constructing our own classification for product development processes. Based on these reviews, we reconsidered and refined our initial classification structure. This process lead us to the following tentative ontology for “Product Development Process”, cf. table 1. Table 1. Initial Ontology for “Product Development Process“ in the context of a product development project. Dimension Actors
Activities
Main Attributes Skills Knowledge Roles Motivation Values Emotions Class Paradigm Cognitive Information processing Problem-solving Decision-making
Social
Information
Physical Artifact
Environment
Function-evolution Problem-solution interaction Mediation
Information sharing Physical Object-engagement Representation Level of detail Level of abstraction Source Interpretation (context) Property Performance Position Organization Physical layout
Sub Attributes/Thesauri structural, discursive, expert experience, common sense, explicit, implicit, episodic chair person, monitor, summarizer, note taker, timekeeper, expert, chief designer, project manager, designer commitment, goals views, preferences tension Action gathering, recognizing, accessing, visualizing, perceiving, reviewing, managing adopting, referring, proposing, structuring, generating examining, analyzing, synthesizing, reasoning, inferring, deducing decomposing, reinforcing evaluating, checking, testing, monitoring controlling, conflict handling, negotiating, persuading, arguing listening, speaking, documenting gesturing, moving, touching, riding wearing verbal, graphic, textual, physical ambiguous abstract, concrete internal, external decision, alternative, criteria, arguments, problem, solution, concept, methods, constraints, intention geometry, material feature, function, structure, behavior, state appearance, feel
It is important to note that the dimensions and the structuring of the main attributes listed in the figure are our own interpretation. The sub-attributes appear explicitly in the published material and constitute the data for this study.
5
Discussion and Implications
Perhaps the most interesting understanding that emerged during our attempt in composing an initial ontological structure was the underlying duality between the information dimension and the rest of the dimensions that are listed on Table 1. For example, the distinction between information and actors was not at all clear. It is plausible to argue that given a certain, yet large, set of information, an actor can be defined (experience, age, personality indicators, gender, knowledge), and conversely, given an actor, a large set of information is defined. A similar duality exists within the relationship between physical artifact and information, activities and information, and, environment and information. Our initial response was to consider removing information as a dimension of the product development process, and to regard it as a sub-attribute for each of the other dimensions. However, through further analysis of the reports, we were able to conclude that the authors did not treat information as being the same as actor, activity, artifact or environment, and that they did attribute information the sub-attributes listed on table 1, treating it as a unique dimension, and differentiating it from others. However, we are still unclear about the meaning of the term information in the design research context. It is difficult to frame and categorize the term because its use in the publications we reviewed is too broad. This line of thinking led us to the following questions: Are design researchers attributing the inherent and poorly understood complexities of the field to the general term information? Can we be more lucid when we label things information? Another important outcome of our study was the successful testing and verification of the feasibility of using an ontological framework as a basis for comparing design research findings. A specific example of the test is the comparison of the contributions from Radcliffe and Ullman et. al. [1] where we find that there is considerable overlap of attributes in the dimensions of activities and artifact, and little overlap in the dimensions of actor and information. Table 2 is a graphic representation of this situation where darkened areas indicate the dimension that has been discussed in detail by the author, and illustrates how the ontology development herein proposed will alleviate some of the problems earlier described. Table 2 – A comparison of two empirical studies based on the ontology. Darkened areas indicate that the dimension has been discussed in detail by the author.
Dimensions
Authors Radcliffe
Ullman et al
Actor Activities Information Artifact Environment
A similar table can be created for all of the categories represented in Figure 1 for each author, resulting in an overall “mapping” of the work onto the ontological structure. Two or more such maps can then be compared and contrasted with each other, providing a basic, relative understanding of the authors’ approaches and positions. In order to increase the resolution of the comparison, categories can be deepened by adding and mapping more subcategories.
In order to assure easy access to the ontological structure, it is pertinent to implement the evolving system as a World Wide Web application. Our assumption here is that if the ontology is made easily accessible, cumulation of methods and findings will take place rapidly through contributions from research groups and individual researchers, and the ontology will evolve over time.
6
Conclusion
In this paper, we argued for the need to build on each others work and to cross-validate our empirical design research results. We proposed an ontological framework to facilitate the sharing and comparison of methods, variables, and results. Furthermore, we illustrated our approach by feeding the findings of published studies into the proposed framework as concrete examples. The process also helped us identify the frequently used term “information” as being somewhat ambiguous, and lead us to question its meaning and relevance to design research. We strongly invite our peers to contribute to the framework as we see crossvalidation and community participation as a crucial step in taking our work beyond its initial phase. References [1]
Cross, N., H. Christiaans & K. Dorst, “Analyzing Design Activity”, John Wiley & Sons, Chichester, 1996
[2]
Gruber, T.R., “A translation approach to portable ontologies”, Knowledge Acquisition, 5(2), 1993, 199-220
[3]
Ullman, D.G., “A Taxonomy for Classifying Engineering Decision Problems and Support Systems”, Artificial Intelligence for Engineering Design, Analysis and Manufacturing, 9, 1995, 427-438
[4]
Duffy, A.H.B., “Ensuring Competitive Advantage with Design Coordination”, Proceedings of the 2nd International Conference on Design to Manufacture in Modern Industry, Bled, Slovenia, 1995, 68-89
[5]
Ulrich, K.T. & S.D. Eppinger, “Product Design and Development“, McGraw-Hill, New York, 1995
[6]
Levitt, R.E.; Cohen, P.G.; Kunz, J.C.; Nass, D.; Christiansen, T.; & Jin, Y., “The Virtual Design Team: Simulating How Organizational Structure and Communication Tools Affect Team Performance” in Carley & Prietula (Eds.), “Computational Organization Theory”, Lawrence Erlbaum Associates, 1994
First authors name: Ozgur Eris Institution/University: Stanford University Department: Center for Design Research Address: 560 Panama Street, Stanford, CA 94305-2232 Country: USA Phone: 650-723-7908 Fax: 650-725-8475 E-mail:
[email protected]
