Integrating Working and Learning: Two Models of Computer Support Tamara Sumner Knowledge Media Institute The Open University Walton Hall, Milton Keynes, MK7 6AA, UK email:
[email protected] http://kmi.open.ac.uk/
Markus Stolze IBM Zurich Research Laboratory Saumerstrasse 4 CH-8803 Rueschlikon/Switzerland email:
[email protected]
ABSTRACT
This paper describes theories and computer systems illustrating two innovative models of computer support for integrating working and learning. The VDDE system illustrates the design critiquing model helping individual professionals in analyzing current work situations, applying existing knowledge to these situations, and articulating new knowledge. The SmartMedia system illustrates the domain construction model helping communities of practice to collaboratively evolve new ways of working. KEYWORDS: Communities of Practice, Critiquing,
Design Environments, Domain Construction, Workplace Learning. NATURE OF WORKPLACE LEARNING
Workplace learning is fundamentally different from traditional school learning (see Table 1). Empirical studies of professional practice, by ourselves and others [12, 13, 17, 18], show that while the focus is primarily on getting the job done, learning is inextricably intertwined with working. Specifically, in order to do their job, professionals must continually learn to apply existing knowledge to routine or innovative situations and to construct new knowledge in response to changing and dynamic workplace situations. Thus, learning is fundamentally embedded in ongoing work activites and these work activities, in turn, give rise to the problems structuring the learning that takes place. Our research described here is concerned with building computational design environments supporting workplace learning; that is, systems integrating both working and learning. The environments being discussed illustrate two innovative models of computer support for integrating
Table 1. School Learning versus Workplace Learning School Emphasis On: Learning “basic” facts and skills Knowledge: Static, Decontextualized, General Goal: “Knowing”
these usually separate activities. One model, the design critiquing model, is targetted at supporting the learning needs of individual designers. The other model, the domain construction model, supports the group learning needs of design communities. The remainder of this paper discusses each of these models by first presenting the theoretical and empirical motivations behind each model and then showing how it has been instantiated in a specific design environment. INDIVIDUAL LEARNING: BECOMING REFLECTIVE PRACTITIONERS
In his seminal book, “The Reflective Practitioner,” Schon [13] describes an action-breakdown-reflection cycle that underlies professional practice. In this cycle, practioners engage in situated action until their expectations are not met and they experience a breakdown [20] in the current work situation. At that moment, practitioners stop and reflect on how to overcome the breakdown before proceeding. These breakdowns in situated action present opportunities for learning and for the construction of new knowledge [3]. For instance, detecting and overcoming breakdowns requires much skill and domain-specific knowledge. People newly hired into a workplace will often lack this necessary knowledge. Even “old-timers” can be challenged. In complex domains, no one person can possess all the knowledge necessary to complete a task [11]. In dynamic domains, what constitutes “necessary” knowledge is continually changing. Thus pracitioners, both old and new, need to continually learn and relearn how to: (1) recognize potential breakdowns, (2) identify knowledge relevant to the breakdown, and (3) apply this knowledge or construct new knowledge to overcome the breakdown.
Workplace Getting the job done, Improving Work Practices Dynamic, Situated, Practice-oriented “Applying” and “Constructing”
Model: Instructionism (knowledge absorption) Constructionism (knowledge construction) Topics / Problems: Given by curriculum
Arise from and embedded in work situation
Scope of Learning: Primarily Individual
Individual, Group, Organization
Figure 1: The Voice Dialog Design Environment.. Designers select building blocks from the gallery (top window) and arrange them in a worksheet (left window) to create a graphic representation of the audio interface design. A critquing component analyzes the design for compliance with interface guidelines and product consistency. Possible problems are signalled in the critic message pane (lower window). The designer can select a critic message and elect to see the rationale behind the rule and can also add more arguments into the hypermedia knowledge-base (right window).
Knowledge-based design environments [2] can help individuals to act as reflective practitioners. Specifically, a design critiquing model [5] embedded in these environments promotes the action-breakdown-reflection cycle. According to this model, computational critics: • analyze representations of work to detect potential problems or opportunities, • signal the designer who then experiences a breakdown and is ready to reflect on its cause, • use this opportunity to deliver relevant information to practitioners to support reflection processes. In this section, we use the Voice Dialog Design Environment to illustrate the basic analyze-signal-deliver design critiquing model. This design environment also extends the basic model by analyzing work representations from multiple and often conflicting perspectives. Voice Dialog Design Environment
The Voice Dialog Design Environment (VDDE ) [8, 9, 16] supports the design of phone-based user interfaces such as voice mail systems and voice information systems. These interfaces consist of a cascading series of voice prompted menus requesting the user to perform certain actions; e.g. “to listen to your messages, press 1.” The user issues commands by pressing touch-tone buttons on the telephone keypad, and the system responds with appropriate voice phrases. Designing in this domain means specifying the interface for a voice dialog application at a detailed level. VDDE (Figure 1) provides a construction kit that allows designers to quickly sketch out the flow of an audio interface by arranging domain-oriented building blocks such as voice menus and prompts into a flow chart-style representation. Designers can hear what their interface
design sounds like by attaching audio recordings to components in the interface and simulating the design. Computational design critics embedded in the system watch designers' actions and comment on potentially problematic aspects of the design under construction. One special challenge facing voice dialog designers is that their design task is influenced by many conflicting design objectives. From the perspective of satisfying new or novice end-users of phone-based products, designers want to create designs that comply with different sets of user interface guidelines (regional, national, and international). However, to satisfy marketing groups and existing experienced end-users, designers also try to create designs that are consistent with related products and existing applications in the installed product base. The problem is that many applications in the installed base predate the interface guidelines and so do not conform to these guidelines. Thus, these design objectives conflict, and the designer must make difficult trade-off decisions between these competing objectives. The computational critics embedded in VDDE support designers to learn more about the different guidelines and the difficult trade-offs between the guidelines and product consistency [1, 6, 8]. The system provides four sets of critic rules that the designer can enable. Three sets of rules correspond to the regional, national, and international phone-based user interface standards. A fourth rule set – the consistency set – compares two designs for inconsistencies. Each rule set contains both generic design knowledge (relevant to most phone-based interfaces) and specific design knowledge (relevant only to certain application types such as voice mail, call answering, voice message delivery, and voice bulletin board applications).
Designers can control the activity of the critiquing system in two ways. First, designers can select what design knowledge should be used to evaluate designs by specifying which rule sets to use and optionally specifying the type of application being developed. Second, designers can control the intervention rate of the critiquing system; i.e., how often critic evaluations are performed. If an active intervention strategy is selected, then the critiquing system automatically analyses the representation for compliance with all enabled rule sets whenever designers place or move design building blocks in the worksheet. If a passive strategy is chosen, the system waits for the designer to order an evaluation of the design under construction, causing all the design’s features to be checked, not just the ones resulting from the last action made by the designer. This passive intervention strategy can be used either instead of, or in conjunction with, the active intervention strategy. Using VDDE , designers create designs by selecting domain-oriented building blocks from the gallery and arranging them in worksheets. As the designer works, the critiquing system analyzes the design being constructed according to the design knowledge and intervention strategy specified by the designer. When the critiquing system detects a possible violation, a brief message signaling the violation is immediately presented in a separate critic message window (referred to as “firing”). Designers can request to see deeper explanations of critiquing messages by selecting individual messages and pressing the “Explain Rule” button (see Figure 1, lower window). Each critic rule is linked to further background knowledge contained in on-line hypermedia versions of the user interface guidelines documents. The “Explain Rule” feature shows designers which building blocks in their particular design caused the selected critic to fire and delivers to the designer the relevant portions of the user interface guidelines. At any time, the designer can extend the hypermedia knowledge-base by adding new information such as the rationale behind difficult tradeoff decisions made when considering critic messages. V DDE was evaluated through both observations of its use in the workplace and through formal studies. In particular, the impact of the critiquing system on the activities of designers was formally studied using protocal analyses of designers using VDDE to solve several design problems over the course of two days (see [1] for a thorough treatment). On the whole, our findings
indicate that the critics in VDDE do support designers in learning about and applying design guidelines and in learning about and making difficult design trade-off decisions; however, the benefits appear to be dependent upon the domain-skill level of the designer. For lesser-experienced designers, i.e., those with skills similar to people who are newly hired, the primary benefits of the critiquing system were to help these designers construct better designs (by detecting possible problems) and to help them learn about and apply design guidelines. None of the lesser-experienced designers in the evaluation had prior exposure to the user interface guidelines. Over the course of several designs, these designers began to anticipate the firing of critic rules they had previously seen during the experiment. Such anticipations indicate designers were learning the guidelines because they not only recalled them, they also remembered the design context in which the guideline applied. However, the critics did not seem to help these lesser-experienced designers in making difficult trade-off decisions. These designers tended to follow the advice of critic rules they perceived as having a higher design priority, even when a preferred solution would have been to break the rule being considered. Though it is difficult to draw conclusions from one study, it appears that confidence in your design skills, as well as overall domain knowledge, is an important factor enabling designers to know when to break the rules. For highly-experienced voice dialog designers, learning new guidelines was not really an issue since these designers were very familiar with the guidelines already. However, the critics benefitted these designers by increasing their awareness of the trade-offs involved when making difficult design decisions. In post-session interviews, the experienced designers reported that knowing the critics were there made them think more deeply about why they were making certain design choices. During the sessions, the impact of the critic system manifested itself by designers anticipating critics before they fired and analyzing deeply the reasons why they were planning to break certain design rules (see Table 2). In a few cases, anticipations led to design modifications before the critic ever fired. In other cases, where designers decided to break anticipated critic rules, they always documented their reasons by extending the hypermedia knowledge-base using the “Explain Rule” feature when the anticipated critic eventually fired.
Table 2. Example excerpt from a highly-experienced designer’s verbalisation showing anticipation of a critic Characterization of Activity
Corresponding part of verbalisation
Anticipation of a critic
"I'm struggling with a critic I know is going to come up. I've got four options here... and that's the maximum number I should have on the menu. But I also need to give them [the end-users] a way to listen to, so I'll need a key for that... So, I've got five options, and that's violating the design guideline."
Analysis of the reasons for "I think the reason I backed myself into this corner is I have picked up on the way breaking the critic rule our current voice messaging vendor has integrated this [service] into their product [...] I knew I was going to get an error because I was violating design guidelines since I have too many options in the menu ..." Attempt to modify the design in "I don't like the way the menu looks. It's a good guideline. So at this point, [...] I order to satisfy the critic rule ... think I would argue for removing the option to print the old documents. [...] I which leads to modifying a would renegotiate with the market unit about the need to get one of them taken design requirement out. And then I'd be able to have everything fit on the menu."
The VDDE system shows how the design critiquing model can support the learning needs of individual designers. Creating systems such as VDDE with an embedded design critiquing model requires design communities to share well-defined practices in the form of standardized design representations and commonly accepted design guidelines. However, for many design communities, particularly those operating in dynamic and innovative domains, such well-defined practices typically do not exist at the outset. In the VDDE project, the construction area significantly evolved over the course of several years in order to keep pace with designers’ changing practices and conceptualizations of the voice dialog domain [10, 16]. The following section describes the next generation of design environments, containing tools and mechanisms supporting design communities to evolve well-defined practices and enrich their design environments with this knowledge. GROUP LEARNING: BECOMING COMMUNITIES OF PRACTICE
Over time, communities of practice evolve customs, conventions and standard practices that help to get the job done more effectively and help newcomers to learn about existing work practices [7, 19]. In our own empirical studies of design communities, we observed how the design vocabularies, representations, and knowledge used by the communities evolved as members continually strove to improve upon existing practices [17, 18]. Typically, these groups began by using illdefined, ambiguous textual representations of their design problem. Over time, they continually evolved their practices until they were using well-defined representations such as specialized flow charts populated with many domain-specific entities. We refer to this evolutionary process as “domain construction” [16]. Thus, for communities of practice, what constitutes their “domain” is not a given, static entity. Instead, “domains” are dynamic entities that reflect a workgroups’ ongoing learning and knowledge construction processes.
Computational tools enabling work groups to become well-functioning communities of practice should embody a domain construction model [4]. According to this model, computational tools must enable practitioners to: • evolve vocabularies reflecting important domain entities (both the name and appearance), • incrementally migrate from ill-defined to welldefined design representations, and • add to and modify existing information in community knowledge bases. The SmartMedia system supports the domain construction model by providing designers with mechanisms for gradually evolving vocabularies and design representations and a language for specifying new critiquing rules. SmartMedia
SmartMedia systems enable users to create “smart media” by adding “depth” in the form of domain-knowledge to their otherwise generic drawing tools [15]. Using SmartMedia, technically skilled designers can incrementally modify basic drawing elements, such as circles and squares, to gradually develop domain-specific, graphical design representations. Domain-specific elements created by designers can be incorporated into the drawing palette for further reuse and refinement. Designers can extend the graphical appearance of design elements, their internal structure and semantics, and their associated behaviours. A typical behaviour might specify how aspects of the element’s appearance change in response to certain property values. Additionally, designers can define interactive controls, to support directly manipulating design element properties. Figure 2 shows a SmartMedia construction window in the domain of city planning.
a b c d e
a'
f
c' f'
Figure 2: The SmartMedia system. Palette items and design objects are from different stages in the domain construction process. Items a and b are generic graphical objects (i.e., rectangles and circles). Items c and d are domain-specific elements representing concepts such as land usage and development timeframes. Items e and f are aggregate domain-specific elements such as city blocks.
Palette items a and b are generic graphical objects – rectangles and ovals – with no domain-specific attributes (see object a'). Special object-editors and a spreadsheetlike language are used to customize graphical objects and palette items to have more domain-specific content. Palette items c and d are domain-specific building blocks that have been derived from the generic graphical objects. For instance, the squares (c) use color to indicate land usage and inner circle shading (d ) to indicate the development timeframe. Items c and d have been grouped into one aggregate “city-block” object (f'). Users can further enrich the domain semantics of objects by defining design rules, such as critics, which are checked during design time.
and empirical evidence arising from studies of design communities and experiences with VDDE.
SmartMedia enables designers to incrementally construct their domain by: • providing an initial palette of generic graphical building blocks, • letting designers gradually and incrementally refine generic building blocks to create their own graphical primitives which can be placed into a palette, • supporting designers to specify domain-concepts, such as land usage, which contribute to the domain semantics of the building blocks, • providing a spreadsheet-like formula language for encoding design knowledge in the form of critiquing rules. These critics employ a visual critiquing approach [14] and signal the designer of potential problems by directly changing visible properties of design elements in the construction area.
ACKNOWLEDGEMENTS
Currently, prototypes of SmartMedia systems have been built in the domains of city planning (shown in Figure 2) and software design. These prototypes are not yet robust enough to be used by real design communities in their day-to-day work. It is only in such naturalistic settings that systems supporting long-term group learning processes such as domain construction can be usefully evaluated.
3. Fischer, G., “Turning Breakdowns into Opportunities for Creativity,” Knowledge-Based Systems Journal, Vol. 7, Iss. 4, pp. 221-232, 1994.
CONCLUSIONS
5. Fischer, G., K. Nakakoji, J. Ostwald, G. Stahl and T. Sumner, “Embedding Computer-Based Critics in the Contexts of Design,” Human Factors in Computing Systems (Interact ‘93 and CHI ‘93), Amsterdam (24-29 April), 1993, pp. 157-164.
We have presented systems illustrating two models of computer support for integrated working and learning. Both the VD D E and SmartMedia systems differ significantly from traditional, passive multimedia learning presentations by their focus on design. It is our belief that systems supporting practitioners to act as designers provide natural opportunities for supporting ongoing situated learning and knowledge construction activities. The VDDE system illustrates how knowledge-based systems can be combined with hypermedia to create active design media supporting the workplace learning needs of individual designers. Evaluations of VD D E indicate that while its design critiquing model was successful in many respects, more work is needed before such systems can better support lesser-experienced designers in evaluating trade-offs decisions concerning conflicting design objectives. The SmartMedia system illustrates incremental formalization mechanisms enabling design communities to progressively evolve well-defined design languages that better support their work practices. While the SmartMedia system is in its early prototype stages, its design is soundly based on theories of group learning
In conclusion, design environments must support the learning needs of both individual designers and design communities. Systems supporting individual learning embody models of well-defined practices, design representations, and design knowledge. In order for such systems to exist, design communities must be supported in gradually evolving their practices to be well-defined and progressively enriching their tools with this knowledge. Thus, the learning needs of individual designers and the learning needs of design communities are mutually supporting and intertwined. Many of the ideas and theories presented here are a product of our “former” community of practice: the Center for LifeLong Learning and Design at the University of Colorado at Boulder. This research was made possible by the generous help of the designers from the Human Factors Group at U S WEST Advanced Technologies. REFERENCES
1. Bonnardel, N. and T. Sumner, “Supporting Evaluation in Design,” ACTA Psychologica (Special Issue on the Usage of Modern Technology by Experts and NonProfessionals), Vol. 91, Iss. 3, pp. 221-244, 1996. 2. Fischer, G., “Domain-Oriented Design Environments,” in Automated Software Engineering, Kluwer Academic Publishers, Boston, MA., 1994, pp. 177-203.
4. Fischer, G., S. Lindstaedt, J. Ostwald, M. Stolze, T. Sumner and B. Zimmermann, “From Domain Modeling to Collaborative Domain Construction,” Symposium on Designing Interactive Systems (DIS ‘95), Ann Arbor, MI (August 23-26), 1995, pp. 75-85.
6. Harstad, B., “New Approaches to Critiquing: Pluralistic Critiquing, Consistency Critiquing, and Multiple Intervention Strategies,” University of Colorado at Boulder, Masters Dissertation, Dept. of Computer Science, 1993. 7. Lave, J., “Situated Learning in Communities of Practice,” in Perspectives on Socially Shared Cognition, L. Resnick, J. Levine and S. Teasley, Ed., American Psychological Association, Washington, DC, 1991, pp. 63-82. 8. Nakakoji, K., T. Sumner and B. Harstad, “PerspectiveBased Critiquing: Helping Designers Cope with Conflicts among Design Intentions,” Artificial Intelligence in Design ‘94, Lausanne, Switzerland (August), 1994, pp. 449-466. 9. Repenning, A. and T. Sumner, “Using Agentsheets to Create a Voice Dialog Design Environment,” Symposium on Applied Computing (SAC ‘92), Kansas City, MO., 1992, pp. 1199-1207.
10. Repenning, A. and T. Sumner, “Agentsheets: A Medium for Creating Domain-Oriented Visual Languages,” IEEE Computer (Special Issue on Visual Programming), Vol. 28, Iss. 3 (March), pp. 17-25, 1995. 11. Rittel, H. and M. M. Webber, “Planning Problems are Wicked Problems,” in Developments in Design Methodology, N. Cross, Ed., John Wiley & Sons, New York, 1984, pp. 135-144. 12. Sachs, P., “Transforming Work: Collaboration, Learning, and Design,” Communications of the ACM (Special Issue on Representations of Work), Vol. 38, Iss. 9, pp. 36-44, 1995. 13. Schoen, D. A., The Reflective Practitioner: How Professionals Think in Action, Basic Books, New York, 1983. 14. Stolze, M., “Visual critiquing in domain oriented design environments: showing the right thing at the right place,” Artificial Intelligence in Design ‘94, Lausanne, Switzerland (August), 1994, pp. 467-482. 15. Stolze, M. and T. Sumner, “SmartMedia Tools: Bridging the Gap Between Generic Applications and Domain-Oriented Systems,” Computer Science, Technical Report CU-CS-792-95, University of Colorado at Boulder, 1996. 16. Sumner, T., “Designers and their tools: Computer Support for Domain Construction,” University of Colorado at Boulder, Ph.D. Dissertation, Dept. of Computer Science, 1995. 17. Sumner, T., “The High-Tech Toolbelt: A Study of Designers in the Workplace,” Human Factors in Computing Systems (CHI ‘95), Denver, CO (May 7-11), 1995, pp. 178-185. 18. Sumner, T. and M. Stolze, “Evolution, Not Revolution: PD in the Toolbelt Era,” Computers in Context: Joining Forces in Design (CIC ‘95), Aarhus, Denmark (August 14-18), 1995, pp. 30-39. 1 9 . Wenger, E., “Toward a theory of cultural transparency: elements of a social discourse of the visible and the invisible,” University of California, Irvine, Ph.D. Dissertation, Dept. of Information and Computer Science, 1990. 20. Winograd, T. and F. Flores, U n d e r s t a n d i n g Computers and Cognition: A New Foundation for Design, Addison-Wesley, Menlo Park, CA, 1986.
