It has been noted that there is a gap between the analysis and products produced by cognitive engineers (CEs) and what is needed by software engineers who ...
PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 49th ANNUAL MEETING—2005
332
CONVEYING WORK-CENTERED DESIGN SPECIFICATIONS TO THE SOFTWARE DESIGNER: A RETROSPECTIVE CASE ANALYSIS Robert G Eggleston Air Force Research Lab WPAFB, OH
Emilie Roth Roth Cognitive Eng. Brookline, MA
Randall Whitaker NGIT Dayton, OH
Ron Scott BBN Cambridge, MA
It has been noted that there is a gap between the analysis and products produced by cognitive engineers (CEs) and what is needed by software engineers who must implement CE-based requirements in the final product medium. In this paper, we report on a retrospective analysis of intermediate design artifacts that were produced from a CE and work-centered design perspective during the course of developing a successful work-centered support system software application. The analysis involved an examination of design analysis and synthesis artifacts produced by the development team that were made available to the software engineers. It concentrated on how work-centered information and requirements were conveyed and made useful to the software engineering staff. Based on this analysis, we suggest that a work-centered specification package needs to contain at least three classes of information: (1) work-centered descriptions of the work context, work activity, and support requirements, (2) principles and guidance specifying “work centeredness” and (3) detailed specifications of the aiding solution with links back to the support requirements. INTRODUCTION A great deal has been written about the issue of bridging the gap from a cognitive engineering analysis to the conceptual design and eventual implementation of a usercentered application. The gap is created by the fact that the development team must go from defining and describing the design problem in user-centered terms to prescribing a usercentered solution that is feasible in the final medium, given cost, schedule, and technology availability constraints. From a cognitive engineering perspective, the question is: how can we best insure the developed product will meet the envisioned work-centered support concept motivated by the use of cognitive systems engineering (CSE) principles and information? The challenge is particularly acute in large systems engineering programs where the cognitive analysis, concept design, and final software implementation may be conducted by different teams of individuals working for different parts of the organization or, in the extreme, different companies that may be geographically separated – with little to no opportunity for direct interaction. Under those conditions it is particularly important that the intermediate design artifacts passed across teams are sufficient to communicate the design intent, rationale, and specification details. In this paper, we present a retrospective analysis of the development of a work-centered support (WCSS) software application that was predicated on general cognitive engineering and specific work centered design principles. We focus on a discussion of the internal design artifacts used by the design team while developing a work-centered design specification and communicating its details, intent and rationale to a wider set of stakeholders (including software developers; program managers; and end-users). Examination of the content of these intermediate design artifacts and the role they played in advancing the design suggests some key
elements that need to be captured and carried along as the design evolves and moves toward implementation specification. Our thesis is that a design specification package for the implementer should include: (1) suitable information about the work context and work being supported, (2) principles and guidance that specify “work centereness,” (3) and a detailed description of the planned aiding solution that encompasses both direct and indirect aiding forms and their integration to achieve a single, coherent work-centered application, with links back to the support requirements by the analysis of the context of work. WCSS DESIGN CASE STUDY This analysis is based on an examination of a development project aimed at producing a work-centered support software application to aid weather management work in an airlift services organization. The project was initiated in 2001 and resulted in a fully functional, interactive prototype application in about 18 months. The project was guided by a stated need by the customer organization to better utilize weather forecasting to aid airlift mission planning and replanning, pre and post vehicle launch. The project was organized as an integrated research and product development team (IPT). Our research goal was to explore the concepts and principles of an emerging humanoriented technology known as work-centered support systems (WCSS) (Eggleston et al., 2000). The development goal was to produce an interactive WCSS prototype that would address cognitively demanding problems involving weather management in airlift flight operations. This involved better integrating weather data and tools and flight planning data and tools to enable improved weather-sensitive mission decisions in forming mission plans. It also involved enabling the weather specialist to better anticipate the impact of emergent
PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 49th ANNUAL MEETING—2005
weather phenomena on missions being executed so that they could be replanned in a manner that was best for airlift operations as a whole. Here we will describe only selected aspects of the project as they relate to the issue of developing and communicating a design specification to the software designers. A more detailed description of the project and the work-centered support design solution is presented elsewhere (Scott et al., 2005; cf. Wampler, et al., 2005). Because this was an R&D effort, we were able to put together a highly experienced, Ph.D-level educated, interdisciplinary team, composed of a senior computer scientist who served as lead software engineer, a cognitive engineer experienced in knowledge capture and cognitive analysis, and a talented and experienced user interface designer. These individuals were able to work collaboratively from project inception to delivery of a running prototype that was implemented in the operations center and is currently in daily use. In spite of the small, cohesive nature of the design team, they needed to produce intermediate design artifacts to communicate design suggestions and rationale to each other, to stake-holders within the user community, as well as to others within their own organizations (e.g., junior software engineers who were responsible for the detailed software development; program managers). The intermediate artifacts the team produced over the life of the project were examined to understand how they contributed to the specification of the ultimate design and it’s rationale. Gaps From CSE Analysis To Software Design At least two gaps can be identified in the process of going from an analysis of the work context and work to be supported to the development of a work-centered aid. First, there is the gap between a cognitive work analysis, however performed, and a proposed design solution expressed in some representational medium. Second, there is a gap in going from this specification to the implementation or build specification for fabrication in the final medium of the product. We will focus on the second gap, in part because the individuals who conducted the cognitive engineering analysis also produced the work-centered design specification. In a traditional systems design approach, the system implementer may only have access to the compiled results of intermediate analyses and not the staffers who produced the analysis or the detailed factors that led to the final design specification. As a result, in situations where the system cannot be built as specified, he or she will have little guidance in determining what compromises are acceptable and which changes would violate critical support requirements. A fundamental question is: what does the implementation designer need to receive from earlier analyses so as to be able to implement a system that meets the work-centered design intent? We approach this issue by addressing three questions: •
What background information does the designer need (e.g. about cognitive systems engineering concepts, the design goal, work context, work activities, etc.)?
333
•
What work aiding concept description is needed (e.g. aiding principles, design guidance, general concept specification)? • What detailed design specification information is needed, given implementation constraints? We examined the different intermediate artifacts produced by the design team to assess which of these issues they addressed and what form they took. Grounding The Implementation Designer We used two main techniques or design artifacts that were produced as a way to ground the implementation designer in the specific design problem from a user-aiding point of view. One technique involved the use of a comprehensive and detailed set of knowledge capture (KC) notes. The KC notes are broader than just a recording of information gleaned from subject matter experts. They include an analysis of cognitive work demands, suggestions of opportunities for work aiding, and other factors that reflect the use of cognitive engineering principles (e.g. analyze the work context, work activities, worker strategies, cognitive demands, etc.). The KC notes serve to recast and add detail to the design problem from that provided by the customer. This provides the implementer the vision for the technical solution from a work-centered aiding point of view. Details about the work context indicate constraints that must be taken into consideration. For example, the KC notes reveal that on-site reporting of weather may differ between civilian and military pilots because of different concerns. Thus, it is important for the implementation design to preserve this distinction because it can impact mission replanning decisions. The KC notes also describe hard cases (work problems) and illustrative incidents that serve to highlight work situation details and cognitive demands that are not revealed by the canonical steps in the basic work procedures. For example, if the forecast during initial flight planning projected a stiff headwind and during execution the headwind is lower, the aircraft may be too heavy to land at the scheduled airfield. This suggests a need to alert users to the situation.
Table 1 Knowledge Capture Notes: Topical Coverage 1. Basic work description/ general facts 2. Process description of target work 3. Cognitive demands, problem solving, decision making view of work 4. Details of work context 5. Communication/interaction patterns among groups contributing to the organization’s operational work flow 6. example ‘hard cases’ and illustrative incidents 7. Opportunities for work support 8. Possible scenarios to highlight off normal situations 9. Potential future organizational changes 10. External ‘market’ forces/influences
PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 49th ANNUAL MEETING—2005
Examples like these help to inform the designer about issues that need to be considered when addressing implementation constraints in achieving the design specification. In short, each facet of the KC notes provides important detailed information or insights valuable to the implementation design from the worker point of view. The KC notes evolved over time based on cycles of observation, interview, and design team analysis, and the final edition contained a well-organized body of work-centered knowledge of value to a software designer. The subject heading used in the KC notes provides an indication of the types of information they made available to the design team. The implementation designer must also be grounded in the general design framework or concepts that are reflected in the work-centered design specification provided to him or her. These concepts indicate high level work-centered design principles and guidelines that imply that the implemented design must have certain properties. These general properties are problem and context independent and so it is useful to understand these properties to aid him or her in recognizing them in the contextualized work-centered design specification. Some of these properties are distributed throughout the design and may not be easily described by an annotated graphical specification or compactly in words. We employed the Work Centered Design framework (Eggleston, 2003, Eggleston and Whitaker, 2002; and Eggleston et al., 2000; Young et al., 2000) in this development project. This framework includes several principles and concepts for the design of workcentered aiding technology. These include the principle of embedding automation-based (direct) aiding within presentation frames that provide indirect aiding via interactive visual ( representational) frames. The representations make use of the worker’s commonly used terms and ontology, and aiding that spans all facets of work from a worker’s point of view. This type of grounding in work-centered principles, aiding product properties, and the basic work-centered design framework was provided in the form of a tutorial presentation to implementation designers involved in this project. A copy of presentation materials, which were made available throughout the duration of the project, may be regarded as a design artifact used to bridge the gap from analysis and the work-centered design concept to the implementation design rendered by the software engineering element of the IPT. These work-centered concepts are embedded in the KC notes in the context of the subject matter and problem solving required of the weather staffer. As a result, the software engineer may gain a deeper understanding of design characteristics that reflect work-centeredness. The designer may be able to use this increased awareness and workcentered knowledge when confronting detail design issues that are not adequately addressed by the provided implementation specification.
334
worker in the context; (2) to provide the basis for defining and focusing the design intervention—reframing and adding detail to the stakeholders’ objective; and (3) to stimulate insight about possible design solution concepts. The development of a work aiding design solution specification marks the completion of the transition from an emphasis on analytic description to an emphasis on design prescription in a representational medium. This is the heart of the design team’s output to the software engineers responsible for implementing the design in the final medium. The software engineer may be regarded as an implementation designer who recasts the design to meet physical, software architecture, and data constraints imposed by the available technologies that can be used for the actual application. We used several design artifacts in this project to carry the design specification to the implementing staff. These artifacts can be clustered into three broad areas: (1) Description of work from a worker’s first person perspective, (2) an over view description of the support/aiding concepts, and (3) detailed design specifications for proposed GUIs. Summary work description. The work description is a compact summary of work from the actor’s perspective. It is a characterization in terms of a work position/role (here the weather specialist and the flight manager specialist) consisting of the following items: • Shared characteristics across roles • Unique characteristics by role • Structure of the principal subject matter • Referential context of work in the subject matter • Focal referent for problem solving • Specific work support needs • Critical information elements It recasts key information derived from the KC notes in a summary form for rapid communication. In this project, the summary work description was expressed in a briefing format. This form is suitable for use internally with design team members as well as externally for communicating with other stakeholders (e.g., customers, sponsors).
Work-Centered Aiding Design Specification In general terms, work analysis has three aims: (1) to describe the work context and activities in general terms and in terms of procedural and cognitive demands placed on the
Figure 1. An example of a compact work description from a first person perspective.
PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 49th ANNUAL MEETING—2005
335
• An example of a partial description of the structure of weather management work is shown in Figure 1. The example is organized from the point of view of specific weather cell positions (here Back Office and Front Office positions). It highlights the fields of analysis and the focal referent (e.g. weather phenomenon, set of missions, and individual mission) to which they are applied. It also includes the interaction between weather staffers and flight managers, (indicated by IMT entries). As Figure 1 shows, this brief work field description provides important information structures involved in problem solving in this domain. Overview description of the support concept. The overview of the support concept part of the specification package largely expresses the visual form of the informationaction space(s) of a direct manipulation graphical user interface application. This overview begins (see Figure 2) by addressing main issues like the fusion of weather information with airlift mission information (through the use of information layers) and work-centered criteria that it is intended to achieve. The complete overview series of charts also addresses the question of how many and in what form the user interfaces need to be in order to handle the work structure from the worker’s point of view. In other words, the charts illustrate the concept and justify it in terms of work-centered requirements.
Major work space properties o Functional operations of the space o Information and control layout & operations • Individual feature details o Information fields o Work-centered labeling o Justifications and connections to the structural work descriptions Again, we have found it to be convenient to present the detailed specification in briefing chart form. An illustration of a detailed specification is shown in Figure 3. This example addresses the work stream of missions for which the weather specialist must produce a forecast. It provides a coupling to the mission tracks shown on the geospatial display (see Figure 2). Further, it serves to connect the work of the weather specialist with that of the flight manager who is responsible for the mission plan and replanning work. Hence, it supports the coordination between these two workers when weather issues are addressed relative to a mission.
Figure 3. An illustration of a detailed design feature specification. Additional charts may be needed to adequately present the specification.
Figure 2. An example of an overview specification of a WCSS design solution. A complete overview may require several charts. Detailed WCSS design specification. The detailed specification fleshes out the basic representational form or structure of the WCSS. We have presented the specification in the form of a heavily annotated set of detailed visual illustrations of the support aid. These illustrations and supporting textual descriptions address items such as: • General spatial layout of the user interface (UI) front surfaces o Central information visualization o Peripheral information elements o I/O control fields
Multiple charts were used to complete the detailed specification. As always, work context information is included along with the technical details of the specification to help the software engineer to stay grounded in the workcentered mindset as it pertains to the specific design problem. As a complement to the work-centered design specification material, we also include an illustrative scenario of work from the first person perspective of the work role. The scenario is based on a work thread that highlights the complexities and cognitive demands of work. This serves to provide the implementation designer with a better understanding of how the tool is used and why various workcentered features are needed. FINDINGS AND DISCUSSION The implemented design substantially conformed to the work-centered design specification and, most importantly,
PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 49th ANNUAL MEETING—2005
preserved the essential properties of a work-centered support system. Some compromises were made to accommodate cost and schedule considerations, such as the control mechanization for personalizing the ordering of map overlay layers on the main application panel. And some features were delayed for a possible future upgrade, such as a more extensive depiction of the relation of weather information and vertical (altitude) dimension. A limited empirical evaluation (Eggleston et al., 2003) and anecdotal data confirms that this work-centered aiding application improved proactive weather management performance in support of airlift operations. We have briefly described the set of intermediate design artifacts that were developed in the process of design and implementation of a work-centered support application. Three primary components were identified that covered conceptual properties of a WCSS, general and context specific guidance, and a detailed specification of the organizational structure and specific aiding features of the design. We contend that these intermediate design artifacts constitute the core elements of a work-centered design specification. Typically design specifications are more narrowly circumscribed, only containing a ‘blueprint’ of the design to be implemented. We believe that, at least for the design of work support aiding applications, a broader view of a design specification is needed. Why does the software designer need all of the extra information we provided? First, because the conceptual design specification may not address all the detailed design questions that are likely to emerge during the design implementation phase. Second, it has been our experience that during the design implementation phase, new issues can emerge that make particular aspects of the conceptual design unworkable or that reveal design trade-offs (e.g., due to hardware/software constraints or data properties). Often it is not practical to consult the work-centered designer for every issue that emerges. In those cases the software designer needs a good understanding of the basic principles and properties of work-centered design to drop back to in working through the design problem. Similarly, the software designer needs to understand the context of work and the factors that contribute to cognitive complexity of work for a target worker (role, person or group) that do not flow directly from the basic procedural requirements for that work position. We provide this information in the KC notes, reinforce it with the role-based work summary and further highlight and reinforce the workcentered requirements during the course of describing the specification for a coherent work-centered aiding solution. In short, the specification needs to be as much about education and guidance about user work and work-centered aiding as it is about product specifications. Both are needed to help the software designer make well-informed design decisions for the actual product architecture. This retrospective analysis examined design artifacts that we produced and felt would be useful for conveying the design to the software implementer, as well as other stakeholders in the development. We do not regard them as a complete or necessarily well-formed set of items to include in a specification package. We recognize some obvious short comings with the specification information described here.
336
For example, knowledge capture notes may not be the most efficient way to provide important general and context specific information about the design concept. It is a resource the software designer should have ready access to, but perhaps a less verbose description would be adequate much of the time. The summary descriptions used in this project may have been too cryptic in some areas to be adequate. For example the specification package did not explicitly explain the coordinated and other details of each visualization. Bridging the gaps from work analysis to design specifications is difficult. This retrospective analysis of one design project indicates how we attempted to bridge the gap. We were fortunate that in this project the specification package did not have to carry the full load of representing the work centered design to the software implementers. The senior software engineer participated fully in all work analysis activities, work centered design activities, as well as software development activities. Informal knowledge gained from this high level of involvement no doubt compensated for any short falls in the specification artifacts. We hope to improve on this situation. The issue of design specification is an important topic on the work centered design research agenda that we are actively pursuing. ACKNOWLEDGEMENT We sincerely thank Laura Militello and Laurie Quill for sensitivizing us to the range of internal design artifacts that we generated in this development product. REFERENCES Eggleston, R.G. (2003). Work-Centered Design: A Cognitive Systems Engineering Approach To System Design. Proceedings of the Human Factors and Ergonomics Society 47th Annual Meeting, Denver, CO, Oct 13-18, 2003.(pp 263-267) Eggleston, R.G. and Whitaker, R.D. (2002). Work Centered Support System Design: Using Organizing Frames To Reduce Work Complexity. Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting, Baltimore, ME, Sep 30-Oct 4, 2002 Eggleston, R.G., Young, M.J., and Whitaker, R.D. (2000) Work-Centered Support System Technology: A New Interface Client Technology for the Battlespace Infosphere. Proceedings of NEACON 2000, Dayton Oh, 10-12 October, 2000. Eggleston, R. G., Roth, E. M. and Scott, R. (2003). A framework for workcentered product evaluation. In Proceedings of the Human Factors and Ergonomics Society 47th Annual Meeting (pp. 503 – 507). Santa Monica, CA: Human Factors and Ergonomics Society. Scott, R., Roth, E. M., Deutsch, S. E., Malchiodi, E., Kazmierczak, T., Eggleston, R. and Kuper, S. R., Whitaker, R. (2005). Work-Centered Support Systems: A Human-Centered Approach to Intelligent System Design. IEEE Intelligent Systems, vol. 20, issue 2, pp. 73-81. Wampler, J., Whitaker, R., Roth, E., Scott, R., Stilson, M. and ThomasMeyers, G. (2005). Cognitive Work Aids for C2 Planning: Actionable Information to Support Operational Decision Making. In Proceedings of the 10th International Command and Control Research and Technology Symposium (June, 2005). Young, M.J., Eggleston, R.G., and Whitaker, R.D. (2000). Direct manipulation interface techniques for interaction with software agents. Paper presented at the NATO/RTO symposium on Usability of Information in Battle Management Operations, Oslo, Norway, April 2000.
This work is not subject to U.S. copyright restrictions.
