For example, Kaptelinin (1996) notes that task-based representations, such as. GOMS (Card, Moran, & Newell, 1983), use a similar language and hierarchical.
CONCERNS AT WORK: DESIGNING USEFUL PROCEDURES
John C. McCarthy1 , Peter C. Wright2 , Andrew F. Monk2 , and Leon A. Watts2
1
Department of Applied Psychology, University College Cork, Cork, Ireland
2
Departments of Psychology and Computer Science, University Of York, York YO1 5DD, England
Running Head: Concerns at work Author Details:
John McCarthy is a psychologist with interests in work
practices and organisation; he is a lecturer in the Department of Applied Psychology, University College Cork, Ireland. Peter Wright is a cognitive scientist with an interest in technologies, work pratice and design methods; he is a lecturer in the HCI group of the department of computer science of University of York, UK. Andrew Monk is an HCI researcher with an interest in design methods and electronic communication; he is reader in psychology at the University of York, UK. Leon Watts is an applied psychologist with an interest in interactive behaviour and collaborative work; he is a visiting researcher in the Human-Computer Interaction Group, Institute of Applied Mathematics, Grenoble, and previously worked as a researcher in the Department of Psychology, University of York, UK.
ABSTRACT The conceptual basis for designing procedures is confused by the problematics of characterising a relationship between procedures and work practices. As they emerge from scientific management theory, procedures connote a means of rationalising and controlling work. However, interpretations of the use of procedures reveal differences in emphasis on the work required to relate procedures to practice, from comprehending to evaluating appropriateness or reasonableness. These evaluations point to a moral character in this work which we characterise in terms of workers’ concerns. Moreover, as conceptual differences in emphasis such as these can prove intractable, we argue that a more productive approach to resolving the problematics would be to evaluate the usefulness of a sensitivity to concerns in designing procedures. Three brief case studies of the use of procedures in safety critical settings point to workers making judgements when relating procedures to their practice, including judgements of the value of the procedures they were using. These cases also demonstrated the complexity of concerns which were multiple and interacting and which had spatial and temporal characteristics. A review of approaches to work which inform HCI design suggests that activity-based approaches, which contextualise goals and actions in terms of both origins and personal investment, provide the minimum meaningful context required to accommodate concerns. Finally, we present an analysis of the implementation of medical guidelines in Britain which exemplifies the transformation in thinking required to design practically useful procedures: from models of work which emphasise control to
those which emphasise commitment and from conceptualisations of procedures as rationalising and controlling to conceptualisations of procedures as educational. This analysis features the sensitivity to concerns in this particular case and draws some suggestive lines from what this case reveals about concerns to the kind of contributions a sensitivity to concerns would make to a contextual design process.
CONTENTS 1.
PRACTICALLY USEFUL PROCEDURES
2. CONCERNS AND PROCEDURES AT WORK 2.1. Customising the Pilot’s Quick Reference Handbook 2.2. Ambulance Despatch 2.3. The Missing O-Ring 2.4. Summary 3. DO DESIGN FRAMEWORKS INCLUDE CONCERNS 3.1. Task-Based Approaches 3.2. Activity-based Approaches 3.3. Summary
4. MAKING PROCEDURES WORK: COMMITTING TO CLINICAL GUIDELINES 4.1. Guidelines Rather Than Stipulations 4.2. Insights from Clinical Guidelines 5. CONCLUSIONS
1. PRACTICALLY USEFUL PROCEDURES This paper is concerned with the design of practically useful procedures. We define procedures broadly to include any artefact intended to guide action, though our particular interest is in their use to organise work. In this context, procedures can be placed on a series of continua: from artefacts designed to stipulate exactly how work should be done to those designed as templates to be used judiciously; from pencil and paper checklists to computer-based checklists; from a bare description of steps to be followed to a well documented guideline with recommendations and explanations. Identifying a space for consideration of the form of specific procedures barely points to the complexity of relations between procedures and work practices. Orr (1996) reminds us that procedures come out of the scientific management tradition which tries to rationalise practice. This can be seen in aviation where it is argued that procedures should be used to minimise the consequences of human
limitations (Hawkins, 1987). It can also be seen in the directive documentation provided to photocopier technicians to prescribe their behaviour, in the belief that technicians will solve problems more quickly by following prescriptions than by reasoning from their experience and understanding (Orr, 1996). Berg (1997) features a number of similar justifications in his account of the introduction of decision support tools into medicine. Advocates argue that, in the face of people’s limited capacity for remembering important but infrequently used information, support tools can even out differences between workers who would otherwise perform quite differently, standardise behaviour across situations, and bring stateof-the-art technical understanding to bear on performance. In short, procedures are provided to rationalise performance which might otherwise depend on ‘unreliable’ individual experience. However, there is evidence to suggest that procedures don’t rationalise practice. Orr and Berg point to social practices that make procedures work. Orr draws attention to the talk of technicians through which knowledge is shared and repairs accomplished. Berg refers to processes of negotiation between disciplining practice to the formal demands of the artefact and localising the artefact to a specific site, which shape both artefact and practices. The argument for rationalising work practices by deploying procedures assumes that procedures control activity. Orr, Berg, and others have demonstrated that a causal relationship between procedures and activity does not strictly speaking exist. For a start, procedures can be badly designed. Orr’s technicians talk about some of their procedures being circular and others being easy to misread. Even technicians who like to use procedures warn that, without understanding the intent
behind a procedure, one can easily get lost while trying to follow it. However, even when procedures are well designed, their relationship with practice can be problematic. In her analysis of instruction following, Suchman (1987, 1993) argues that procedures are inevitably incomplete. They underspecify the ‘world’ being described as they contain descriptions of objects and actions which cannot be the objects and actions in situ. If we accept that procedures are inevitably indexical, then the problematics of relating procedures to practice are worked out by the person using them in a particular situation. Hutchins (1995) looking at the use of written procedures in navigation puts it this way: “In order to use a written procedure as a guide to action, the task performer must coordinate with both the written procedure and the environment in which the actions are to be taken.” (p.295) According to Hutchins, transforming the written procedure into activity requires a series of smaller transformations including: reading and comprehending the words, understanding what they mean, and turning that understanding into some notion of appropriate action. We don’t need to agree with the details of Suchman’s or Hutchins’ analyses to recognise the central point made by both of them, that is, that there is always work to be done to carry out any procedure. Therefore, those who introduce procedures to control or rationalise work are still faced with people having to interpret and act from their own experiences and concerns and, as a consequence, with whatever individual differences that presents. Schmidt and Simone (Schmidt & Simone, 1996; Schmidt, 1997), developing an account of coordination mechanisms as a
foundation for CSCW design, challenge the strength of this conclusion. They argue, on the basis of their analysis of the use of Kanban, that procedures can determine action. Kanban is a just-in-time production control system where cards are used to carry information about the current state of production of specific parts and as production orders instructing receivers to initiate specified production activities. In a field study of Kanban use, workers produced batches specified by Kanban cards unless they had strong reasons not to do so. For example, when there was a rush order, workers would reconfigure the system by taking the Kanban card out of circulation. As soon as the rush order was dealt with, they would switch back to the Kanban configuration by putting the card back into circulation. Schmidt and Simone conclude that workers only deviate from stipulations when they have compelling reasons to do so. Otherwise, the stipulations in the protocol or procedure determine action. An alternative reading of the Kanban case study would emphasise the work done to interpret and evaluate what is demanded by the Kanban protocol. There is the work of interpreting what is written on the cards, for example, part numbers and the work of relating what is written on the card to an appropriate course of action. Schmidt and Simone play down interpretation and evaluation in favour of an account of the worker, relatively unproblematically, producing “the batch as specified by the card, in accordance with the general rules of the protocol” (p.172). Yet, in their analysis, Kanban workers deviate from the protocol when it makes sense to do so. Schmidt and Simone refer to these deviations as local, temporary modifications to suit the requirements of the situation. Whether producing the batch
as specified or deviating from the specification, this implies that the work features everyday judgements of reasonableness or appropriateness, moral judgements which we characterise in terms of workers’ concerns. At bottom, then, the apparent opposition becomes a matter of emphasis. Whereas Suchman emphasises the work done to carry out procedures, Schmidt and Simone play down this work and emphasise instead correlations between their reading of the procedures and the actions of workers. Whereas Suchman emphasises workers prospectively and retrospectively making sense of their practical reasoning and activities, Schmidt and Simone treat this kind of reflection as exceptional. As a conceptual issue, this difference in emphasis presents irreconcilable perspectives on the problematics of relating procedures and practice. As a design issue, it presents us with a dilemma to be resolved in the practice of constructing procedures. Instead of trying to establish whether procedures determine practice or not, the question becomes one of whether attention to workers’ concerns is useful in the design of procedures. A focus on ‘concerns’ draws attention to the concrete, volitional, and the moral in human activity. ‘Concern’ is related to Bourdieu’s (1998) use of notions such as investment, interest, and preoccupation and carries with it some of the same connotations: that is, it points to aspects of an activity mattering to the actor. But, with respect to procedures, it goes a little further. Following Bakhtin (1993), we use it to emphasise what Hicks (1996) called “the moment-to-moment ‘oughtness’ in which particular agents were responsive to particular situations of being” (p.107). According to Bakhtin, this moment-to-moment oughtness invests each human act with an emotional-volitional tone and answerability which is particular to
the person, object, and situation and which is more like faithfulness than adherence to a set of norms. In this, we detect resonances of what is present in accounts which emphasise the work done by people to relate various expressions of ‘oughtness’ (including those contained in procedures) to their practices, and absent in those which ignore this quality. Our choice of ‘concerns’ to express this salient quality of relations between person, object, and situation is rhetorical. It carries with it a sense of working at and caring about what happens, without being so concretely defined as to run the risk of creating another abstraction. We use the term specifically to draw attention to the work of relating representations, which carry obligations, to work practices and to play up the volitional, value-laden aspects of that work. But we also use it vaguely in the sense that we would like it to take its specific meaning from the particular contexts in which it is used. In this regard, our use of ‘concerns’ as a noun is tentative. Our preference is to use it to point to a quality of action but the implications of a turn to concerned action as a generative metaphor in HCI are as yet underplayed and our hope is that pointing to examples of the foci of concerns may help elucidate implications for design. One way to explore the problematics of procedure and practice as a design question is to use ‘concerns’ to critique designs, specifically to evaluate procedures with respect to their sensitivity to concerns. In the next section, we pursue this theme with respect to the use of three procedures deployed in safety critical settings. Later, we evaluate the extent to which concerns are already part of the theories of work that inform design in HCI and describe, as an example of how concerns
might be accommodated in design, a particular case in which workers’ concerns were featured in developing procedures.
2. CONCERNS AND PROCEDURES AT WORK The work we are attempting to do with ‘concerns’ fits with approaches in HCI design which emphasise the context of computer use (e.g. Bannon & Bodker, 1991; Beyer & Holzblatt, 1998; Bodker, 1997; McCarthy et al., 1997; Nardi, 1996). Concerns, as a construct, can be seen as emerging from some of the theoretical approaches to context and, if it is to be a useful construct in design, should bear on practical interventions such as Contextual Design (Beyer & Holzblatt, 1998). For example, concerns can be seen as relevant to Beyer and Holzblatt’s five work models in two ways: as a construct for relating the work models to each other and as a means of conceptually elaborating the cultural model. Insofar as they speak to the relationship between worker, object and situation, concerns span the five models and may be used in integrating observations across the models. More specifically, a focus on concerns suggests a need for a more conceptually elaborated cultural model than the Beyer and Holzblatt model which reduces culture to influences. Beyer and Holzblatt also make work redesign an explicit step in the contextual design process. Our focus on concerns in design is not only consistent with this turn in HCI from decontextualised computer applications to work tools and activities (Bannon & Bodker, 1991; Bodker, 1997) but with the related challenge to instrumental conceptualisations of work (Sachs, 1995).
Procedures, as artefacts to guide or control work, provide an interesting challenge for contextual design informed by the rubric that design must fit work practice. Moreover, when deployed in a safety critical settings, procedures are used to at least give the impression of practice disciplined by rationality. Their salience is emphasised to workers in training and in allocating responsibility for failure and their safety value is emphasised in public presentations of these industries. If procedures were to be used as clear stipulations in any work area, it would surely be here. As such, analysis of the use of procedures in safety critical work, provides the acid test for establishing whether sensitivity to concerns would be useful in the design of procedures. For this reason, the three cases which follow evaluate the use of procedures in safety critical settings. The first case refers to the use of Standard Operating Procedures (SOPs) in aviation as a way of standardising practice in cockpits. 2.1. Customising the Pilot’s Quick Reference Handbook On all flight decks, SOPs are available in a Quick Reference Handbook (QRH) which contains descriptions of procedures for all relevant flight tasks that the designers can anticipate, including routine tasks such as take-off and landing and tasks related to dealing with emergencies. The procedures are typically defined by aircraft manufacturers or operating companies and often take the form of a checklist printed on a card. The detailed characteristics of the checklists can vary from situation to situation and from aircraft to aircraft but, for purposes of illustration, figure 1 shows an extract from a B-757 Normal Checklist.
--------------------------FIGURE 1 HERE --------------------------Wright, Pocock and Fields (1998) report a study of pilots’ use of such procedures. Following observation, interviews, and analysis of pilot documentation, they draw attention to a number of issues which are relevant to relating procedures to practice. For example, it seems that pilots using SOPs to deal with an emergency would be likely to have to use a number of different SOPs concurrently. Safely interleaving between SOPs requires an understanding of the physical systems with which they are interacting to make judgements about where in a procedure it is safe to delay and move to another procedure. Pilots often have to prioritise or order the actions associated with the different SOPs. This is acknowledged by pilots to be a situated activity depending on contingencies such as phase of flight, height above ground, and weight of aircraft. In their interviews with pilots, Wright et al. were informed that pilots often carry a personal copy of the QRH which they use in preference to the flight deck copy. The personal copy is often annotated. Pilots seemed to think of this as customising the QRH to fit the requirements for flying an aircraft which they experience in line operations and simulator sessions. For example, one pilot’s QRH which contained 118 different procedures had no less than 331 annotations made by the pilot himself. A summary of these annotations is presented in figure 2. ----------------------FIGURE 2 HERE
----------------------The pilot’s annotations reflect his concerns about the effectiveness of the procedures, understanding how and when to use them, and why the procedures are the way they are. They can be seen as an attempt to bridge the gap between work as represented in the procedures and work as responding to moment-to-moment oughtness, irreversibly and consequentially. In bridging the gap, the pilot renders the procedures meaningful to himself. There is also a sense in these annotations of concerns stretched over time. The annotations are made in anticipation of encountering the kinds of events that would require the use of these procedures, not as or after those events are encountered. The very possibility of engaging in some of the activities represented in the procedures matters sufficiently to the pilot that he works out and records his options and justifications in advance.
Would a sensitivity to concerns help in the design of SOPs? Minimally, specification of the pilot’s particular concerns could be used to redesign the SOPs, for example, by providing rationale for actions as some of the more recent electronic SOPs do. Although this might be a better place for a pilot to start working with SOPs, it misses a key design point which is that the annotations reflect an individual’s particular experiences, interpretations, and concerns. Beyond design fixes, a sensitivity to concerns might help us to understand why pilots end up with underspecified SOPs, but it would require sensitivity to a variety of concerns, not just the pilot’s. For example, if annotations point to pilots’ concerns, the SOPs themselves may point to the operating company’s concerns. A viable hypothesis is that, as well as safe operation, procedure writing is informed by concerns relating to litigation and explicating responsibility if litigation arises. This may lead to a form of defensive procedure writing, which inhibits the author from dealing with contingency for fear of muddying waters about responsibility. A final design point is that concern with organisational issues such as standardisation and sensitivity to litigation is likely to result in separation of concern for good procedures and concern for good practice, a tacit acceptance that work gets done in the space between procedures. As we will see with the next example, this can lead to a procedure being ignored in practice. 2.2. Ambulance Despatch The organisation of ambulance control has been described in a number of studies (e.g. Martin, Bowers, & Wastell, 1997) and, in this section, we develop an example bearing on the use of procedures in ambulance control based on one such
study (McCarthy et al., 1997). This example centres on decisions made about which ambulance to despatch. On receiving an emergency call, the ambulance controller must select one of the available ambulances and direct it to the scene. She works out which ambulances are available from convergent information from two main sources. The first is an electronic display screen that records the history of communications with each ambulance, including calls to indicate return to station. The second is a notepad divided into boxes for each station, on which the controller has recorded ambulances in station, stood down, despatched and the time at which it arrived in station, stood down, or despatched. Because the electronic display does not hold enough information on one screen and it does not explicitly represent order of return to station, the controller mainly relies on her notepad in deciding which ambulance to despatch. The controller selects the ambulance, which has spent the longest time in station, from those available at the station nearest to the scene. This First In First Out rotation of ambulances is common practice among controllers and is generally felt by ambulance crew and controllers to be the fairest way to organise the work. Taking account of the station is seen as a preference for using local knowledge and expertise in achieving an efficient service. First In First Out rotation of ambulances runs counter to management policy and procedures. The main procedure in question was issued as an instruction to workers always to despatch the ambulance nearest the scene of the emergency. It was supported by the design of the electronic display screen, mentioned above, which does not explicitly represent order of return to
station. It was further reinforced by strongly discouraging the use of private notepads. The procedure for the task of despatching is to select from those ambulances that are available and near the scene of the emergency, regardless of station or rotation. This reflects management concern for efficient and economic deployment of resources across the whole region. The controllers’ and crews’ preference for a station based rotation system of despatch reflects concerns for fair distribution of work, general welfare of colleagues, and a different view of efficiency based on local knowledge. Both sets of concerns reflect moral sensitivities. The problem for the operation of the procedure is that they reflect divergent moral sensitivities: an ‘oughtness’ of global efficiency and an ‘oughtness’ of local efficiency and welfare of colleagues. In this case, concerns are stretched both across time and social space, to reflect an enduring faithfulness to personal and professional commitments. It is hard to see any design solution to the problem posed by the contradictory concerns of management and ambulance controllers. This makes it an interesting case for design because we cannot be distracted from the main issue by claims that the immediate ‘problems’ could be put right by redesign. It cuts to what can often be the heart of the matter for designing procedures, which is the potential for incommensurable concerns derived from qualitatively different views on efficiency and models of work. This is where Beyer and Holzblatt’s commendation to make the design fit current work practices rubs against the technical-rational view of procedures as stipulations. As a consequence, the ambulance control example raises interesting questions about whether contextual design is a process of transformation of representations or a critical dialogue. This is also the case with our final example.
2.3. The Missing O-Ring Woods et al. (1994) describe an accident scenario in which an aircraft lost power on all three of its engines (National Transportation Safety Board [NTSB], 1984) having lost oil pressure to all three in mid-flight. Two of the engines stopped in mid-air and the third stopped at about the time the pilots safely landed the aircraft. The primary cause of the failure was that O rings normally attached to an engine part in each of the three engines were missing. These rings seal the part making it impossible for oil to leak out through the fitting. The engine part in question was a master chip detector used to detect engine wear. These are replaced on all three engines at scheduled intervals by a maintenance crew. The mechanics' work card had a checklist which required them to fit the O ring to the chip before insertion and there was a space adjacent to this item on the checklist for them to sign when the task was completed. However, the day-to-day work practice of the two mechanics for the flight in question was such that they always obtained replacement chips from the foreman's cabinet. These chips were always ready to insert with the O rings already fitted by their supervisor. The night before the accident, the mechanics found that there were no chips in the foreman's cabinet and they thus had to get the parts from the stockroom. These chips did not have O rings fitted and in the prevailing low light conditions the mechanics did not notice their absence. The foreman’s actions were ‘in violation’ of procedures since fitting the O rings was a job to be carried by the mechanics not by himself. But there had been a history of O ring problems attributable to mechanical error. The foreman’s
workaround ensured that the O rings were correctly fitted to the chip, indicating his concern that they be fitted on every occasion and the inadequacy of the original procedure in this respect. In turn, this speaks to concerns about personal and professional accountability, controlling what is done in his area, and perhaps concern for the pressures on mechanics. The foreman’s actions speak to the dilemma of relating the ‘oughtness’ represented in procedures to the ‘oughtness’ experienced in contingent practice. The procedures are written as if all else were equal but that is never the case. The supervisor knows that all else is not equal and acts with sensitivity to cultural and historical contingency. The implications of being sensitive to concerns in this case relate more to the nature of design than to particular fixes. It points to the merits of design as critical evolution of representations, which may be problematic for Beyer and Holzblatt’s separation of work redesign from the system design process. Of course, they are aware of the problems and separate it off for emphasis, not with the intention that it be separated in practice. The O-ring case points to the need to place the design of procedures squarely in the context of the design of work. 2.4. Summary The three examples feature workers in safety critical settings exercising their judgement when relating procedures to work practices, not because they thought little of the procedures, though that may well have been the case in ambulance despatch at least, but because doing their jobs was a moral issue for them. They found themselves making judgements about how to do their work informed by a sense of faithfulness redolent of concern.
These examples also point to the complexity of concerns: multiple, temporal, and spatial. Concerns are not just about aspects of a situation contemporaneous with activity in that situation. They can reflect anticipation of actions, their consequences and accountability. Moreover, concerns are not centred solely on individual workers but reflect histories and cultures of oughtness in a social space, sometimes conflicting histories and cultures. The work practices we have observed, which reveal faithfulness to the oughtness of concerns, problematise technicalrational constructions and design of procedures. The usefulness of a sensitivity to concerns extends beyond design solutions to a critique of design itself. Some of the critique has already been aired elsewhere (e.g. Berg, 1997) but it is worth rehearsing again. Concerns fit best with a design process that is critical and not just transformational and which accommodates dialogue between the concerns of different actors rather than imposing a monological technicism. It also requires contextualisation of procedures in work, where many procedures may be at play at one time. Having established the usefulness of a sensitivity to concerns in design, we now turn to the question of the sensitivity of current design frameworks to concerns.
3. DO DESIGN FRAMEWORKS INCLUDE CONCERNS? As we have seen in the previous section, sensitivity to concerns places the question of relationships between procedure and work practice in a broad social and organisational context. It features aspects of work which speak to experiences of commitment and faithfulness which stretch over time and across social and cultural space. The argument is that the design of procedures would profit from
attending to this rich experiential canvas. The question is whether existing design frameworks have the capacity to do that. For the purpose of exploring this issue, we consider approaches to work in interactive system design in terms of whether they are task-based or activity-based. 3.1. Task-Based Approaches Task-based representations of work construe work as goal directed behaviour which can be decomposed in meaningful ways. Some contextualise goals and others don’t, and this turns out to be a crucial difference in terms of their capacity to accommodate concerns. For example, Kaptelinin (1996) notes that task-based representations, such as GOMS (Card, Moran, & Newell, 1983), use a similar language and hierarchical structuring of concepts to those used in activity theory (Kuutti, 1996). Both address ‘goal’ directed behaviour and their respective constituent elements, ‘operators’ and ‘methods’ in GOMS and ‘operations’ and ‘actions’ in activity theory, correspond. A key difference, however, is that while activity theory puts operators and actions into the context of activities, thereby accommodating reflection on the origins of goals and workers’ investment in them, “GOMS does not deal at all with the origins of goals” (op.cit. p.61). According to Kaptelinin, task-based rendering of work is guided by a view of goals as system goals to be achieved by a closed loop information processing system. This is echoed in definitions of task analysis: “Task analysis involves the study of what an operator (or team of operators) is required to do to achieve a system goal” (Kirwan & Ainsworth, 1992, p.15).
Sensitivity to concerns gives work goals a different status to the goals in task analysis which are ‘given from nowhere’ and which are not amenable to consideration in terms of constructs such as interest, investment, faithfulness, oughtness and concern. This is not an accident: in task analysis, the information processing loop is intentionally closed. Task analysis is geared towards identifying invariants in the individual, theoretical, cognitive mechanisms that allow people to solve problems and complete tasks (Simon, 1990). Some of the limitations of the task-based approach in explaining and predicting behaviour (Landauer, 1991) and in helping with making design decisions (Carroll, 1990) have already been well rehearsed. Carroll argued that a deeper understanding of the contextually rich work actually undertaken by people is required in design. Hutchins (1995) offers a richer approach to context and to the role of artefacts in his Distributed Cognition, without abandoning the functionalist task-based account of interactive behaviour. His analysis of ‘cognition in the wild’ has led to developing an approach to cognition which is distributed between people, artefacts, institutional supports, and so on. This requires a consideration of tasks not as the sole property of a single agent but shared among human and non-human agents. The information processing loop is opened up to the environment of artefacts and people. This change in focus from individual task performance to tasks as properties of distributed functional systems allows for a more social and interactional approach to cognition but the interaction is still inherently instrumental, a means to an end, and the process is only social to the extent that information is processed by more than one person. For example, the goal, while no longer ‘given from nowhere’, is ‘just there’. It is part of the ‘taken for granted’ of the distributed
cognitive analysis. The ‘taken for granted’ in decontextualised task analysis, the origins of goals and individual investment in goals, is precisely where the focus needs to be to accommodate concerns. 3.2. Activity-based Approaches In activity theory, actions and operators are embedded into the context of activity, that is, “historically conditioned systems of relations among individuals and their proximal culturally organised environments” (Cole & Engestrom, 1993). Although both task analysis and activity theory have a strong functional aspect, contextualising actions in activity takes behaviour from the depersonalised or mechanically goal-directed arena of task to the motivated arena of activity, “the minimal meaningful context for individual actions” (Kuutti, 1996), an arena which may accommodate concern more easily. In an activity system, the concerned, interested worker makes choices and acts on priorities and pressures felt in a milieu of meanings and possibilities. The system is not a closed information processing system but a cultural-historical activity system, motivated by transforming an object into an outcome. In activity system approaches to work, the object can be seen as an expression of concerns in that work setting. For example, the house plan as object expresses functional, cultural, and aesthetic concerns about houses to be transformed into an outcome in the form of a newly built house. In another example, Engestrom (1993), examining the work activity of general practitioners in a health centre, posits alternative objects. He contrasts the object ‘patients with ambiguous problems as quantity’ with the object ‘patients with ambiguous problems as life
systems’. This contrast is an expression of the kind of choices doctors have to make in practice in response to a number of pressures experienced daily: for example, bureaucratic demands for more output and adherence to regulations, patients demanding more time and better care, nurses in the centre pushing for a holistic view of patient care, and hospital specialists demanding effective screening of patients for referral. The two objects reflect a difference of emphasis in activity mediated by the weighting of different concerns in practice. The alternatives proposed by Engestrom embody different concerns, not unlike the difference between the management and the ambulance controllers in section 2.2 who respectively model the ambulance crew as resources or as colleagues. This is not a detached modelling but one which embodies concerns which really do matter to the people engaged in the activity. Ambulance controllers risk at least the dissatisfaction of management in acting as they do. They make ‘ambulance crew as colleagues’ an expression of their concerns in practice by operating the FIFO rotation against procedure. And managers risk the discontent of workers by making ‘ambulance crew as resources’ an expression of their concerns. These discriminations between different concerns creating different objects are made by workers in practice, giving work its emotional-volitional tone. Our discussion of the relationship between operating procedures and practice in Section 2 demonstrated that procedure is not constitutive of work primarily because the concept of procedure has no sense of the rich interactional context of interest, engagement, and concern. Phenomenological approaches to situated action offer a radical alternative. For example, Suchman (1987) argues that situation is constitutive of action. In contrast with task-based frameworks where the situation is
characterised as an aspect of the means to achieve ends or part of the conditions for accomplishing a goal, situated action suggests that situations and actions are intimately linked: “.. the detail of intent and action must be contingent on the circumstantial and interactional particulars of actual situations” (Suchman, p.186). Lave (Lave, 1988, 1993; Lave & Wenger, 1991) also offers an explicitly relational account of socially situated practice insisting that people acting and the social world of activity cannot be separated. “Theories of situated activity do not separate action, thought, feeling, and value and their collective cultural historical forms of located, interested, conflictual, meaningful activity” (Lave 1993, p.7). Moreover, Lave proposes that the character of situated practice is heterogeneous and multifocal. She points to the ways in which people who constitute ‘a situation’ together, know different things and speak with different interests and experience. A radically situated representation of work would leave as much to be desired with respect to addressing concerns as a radically task-based or cognitive representation. What gives life to any of them is meaningful interaction. To quote Bohler (cited in Joas, 1996): “Situational orientation and goal orientation are interlinked with one another from the outset. For if we did not have certain dispositions towards goals, no matter how vague, which are given ante actu in the form of needs, interests and norms, an event would not occur for us as a situation we are in, but would remain devoid of meaning and mute” (p.161). Both Suchman and Lave offer a relational position. Suchman argues that the term situated action “underscores the view that every course of action depends in
essential ways upon its material and social circumstances. Rather than attempting to abstract action away from its circumstances and represent it as a rational plan, the approach is to study how people use their circumstances to achieve intelligent action” (1987, p.50). Lave talks explicitly about activity as dialectical relationship between person and setting. While activity theory and situated action share a cultural historical orientation, activity theory would emphasise division of labour and community norms built up over time in analysis of the ambulance control scenario, situated practice would emphasise the social construction of precisely this kind of situated practice “to give meaning to figural concerns” (Lave 1993, p.19). Both would agree that concerns can only be identified from the inside, from the experiences of those engaged in the meaningful activities. 3.3. Summary Task-based approaches to modelling work employ a construction of goal as system goal, the origin of which is outside their scope. They are therefore silent on the question of concerns. In contrast, activity-based approaches offer a richer set of relationships. The pivotal construct, object, in activity theory can be seen to express something of the way in which concerns shape the activity system. Analytically, sensitivity to concerns could help the analyst characterise the object of an activity. Experientially, concerns are at the heart of activity, and relational approaches to situated practice offer insight into how practice is shaped to promote figural concerns. Concerns are not ‘in the situation’, no more than they are ‘in the task’ or ‘in the object’. Rather they are characteristic of the transactional relationship
between person-in-activity and the setting of that activity. They are part of what gives shape or colour to a situation, the experience that makes a situation a setting for meaningful action. When exemplifying activity-based approaches, we have concentrated on Activity Theory and Situated Practice because of their currency in HCI. We could equally well have drawn on the cultural-historical approach to activity advocated by Cole, Scribner and others (e.g. Cole, 1996; Tobach et al., 1997), the Soft Systems Approach developed by Checkland (Checkland & Scholes, 1990), Strauss’ work on articulation of complex work (Strauss, 1988), or the older tradition in sociotechnical systems (The Open Systems Group, 1972). Our main aim in this section has not been comprehensiveness, rather to indicate which frameworks address concerns. We have argued in this section that activity-based frameworks accommodate concerns. These are frameworks which contextualise the technical accomplishment of procedure following with respect to the “open-endedness and hazard of action” (Dunne, 1993, p.367), not as means-to-ends but as an interplay of means-and-ends permeated with oughtness. This reframing of work enables us to reframe procedures in terms of commitments rather than control as a means of exploring how to design useful procedures.
4. MAKING PROCEDURES WORK: COMMITTING TO CLINICAL GUIDELINES The history of procedures in the British National Health Services exemplifies the transformation in thinking required to design and implement useful procedures. It reveals a transformation from a rationalist means-ends conceptualisation of procedures and work to an activity-based conceptualisation which emphasises investment by practitioners in a way of working and which transforms procedures from control mechanisms to foci for commitment and resources for education-inaction. In terms of designing for concerns, the experience of the British National Health Service can be seen as a pointer to a Contextual Design of procedures leavened by a sensitivity to concerns. Early attempts to prescribe procedures for diagnosis and treatment in the British National Health Service met with considerable opposition from medical professionals concerned with their professional independence and afraid of the potential for insensitivity to their local circumstances. The early procedures may also have been poorly framed making them hard to apply anyway. The opposition forced some significant changes to the ways in which medical procedures were specified. Widely accepted and empirically justified guidelines on how to design and implement clinical guidelines are now emerging (e.g. Grimshaw and Russell 1993a, 1993b; Effective Health Care, 1994), some of the attributes of which are listed in figure 3.
-------------------FIGURE 3 HERE -------------------Alongside the expected scientific basis for the guidelines (validity, reproducibility, reliability, clinical applicability) and economic viability (cost effectiveness), we find 'representative development' and 'clinical flexibility'. These record the need to incorporate the concerns of the many parties involved but also to allow practitioners to adapt the guidance contained in the formal documentation to meet the practical constraints under which they work, including their need to be sensitive to the concerns of others in their daily work. Two characteristics of this new approach seem noteworthy in the context of this paper: (i) the evidential basis of each recommendation is spelled out by a team of professionals and the procedures are expressed as ‘guidelines’ with the explicitly stated implication that they are not obligatory; (ii) efforts are made to encourage a sense of ownership for the guidelines as implemented and to make sure they are usable in practical circumstances. Together these characteristics reflect an implementation strategy which depends on commitment and faithfulness rather than control. 4.1. Guidelines Rather Than Stipulations The stated purpose of clinical guidelines is to change clinical practice for the general benefit of the Health Service. It may be tempting for administrators to try to enforce them, in much the same way that airline operators and regulators try to enforce the use of SOPs. However, the structure of the health service and the large
degree of autonomy enjoyed by GPs and consultants within this structure makes this very hard to do. There are no legal or formal managerial devices for enforcing procedures. The users of the guidelines are encouraged to use their judgement, based on their appraisal of the situation in conjunction with the problem descriptions contained in the guideline specifications. This points not so much to designing procedures to fit practice as designing procedures as resources for those most immediately involved in the practice to change the practice themselves. This can be seen clearly in the case where the authors of procedures feel that there is an overwhelming case that some procedure should always be followed. The negotiation of change means that the author has to make that case in full. This has the interesting side effect of making the designers or authors spell out their concerns in developing the guideline. Will it save money for the health service? Will it result in a better quality of life for the patient? They even have to spell out an estimate of the probability that following the guideline will result in the expected cost savings or improved care. Eccles et al. (1996) suggest that each part of a guideline should be graded as to its evidential basis and indeed the Department of Health issues guidelines with a three-tier system of recommendation confidence. Large effects demonstrated in formal meta-analyses of several quantitative studies would indicate strong reasons for following that part. Other parts may be based on less thorough reviews or even a body of expert opinion, in which case it will be much more up to the individual doctor's judgement whether to adhere to the guideline. Spelling out the concern underlying a guideline, as well as the evidence that following the guideline will meet that concern, acknowledges the inescapable moment-to-moment oughtness
and answerability of the doctor’s work. Guidelines are thus presented as a tool at the disposal of the clinician "to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances" (Institute of Medicine, 1992). The purpose of guidelines, in this context, is to improve clinical practice based on the accumulation and interpretation of medical knowledge, that is, it is largely an educational activity. Research has shown that guidelines are most effective in changing the behaviour of the people who have had a hand in developing them, that is, the process of developing a clinical guideline has a large educational effect on those involved (Grimshaw & Russell, 1994).Those who are less involved and who are simply given the guidelines may not change their practices at all. Eccles et al. suggest various devices that may be used to encourage a feeling of investment in the guidelines: national guidelines should be customised by local committees; local circumstances, such as the technology available, may require modifications; local variations in features such as the names given to personnel and facilities should be permitted. The design of these guidelines and the context of adaptability with concern for ownership into which they are deployed seems like a recognition that procedures are best viewed as resources for action and education rather than as control mechanisms. 4.2. Insights from Clinical Guidelines The process of designing and implementing the clinical guidelines seems quite different to our understanding of the process of designing procedures for fitting O Rings, ambulance despatch, and Kanban. A comparison of aspects of these
approaches, outlined in figure 4, might profit other attempts to design useful and usable procedures. ---------------------FIGURE 4 HERE --------------------Are there aspects of the design process and/or implementation that can be taken from the clinical guidelines case and applied elsewhere? In domains such as manufacturing, aviation, process control, and ambulance control, it is the operator who is obliged to follow the guidelines and it is the operator who is generally held responsible for the consequences of following them or not. Thus although the power relationships between a pilot and an operating company, or between and ambulance controller and manager are quite different to those between a consultant and a healthcare manager, the fact remains that it is the pilot and the ambulance controller and not the operating company or manager that is ultimately responsible for the actions taken. For this reason, the person using a specified procedure must feel, and actually be, in control of the actions taken. At the very least, in meansends terms, this requires making explicit the ends for which the means of the procedure has been designed. The user faced with competing concerns can then make an informed judgement when procedures appear to be incompatible with some concern or responsibility that was unforeseen by the designer of the procedure. It further requires that procedures allow for at least some flexibility in the implementation and organisation, to allow the user room to perform the
pragmatically inevitable manoeuvring essential for satisficing the pressures of different concerns. The other lesson arising from this work is the well rehearsed principle, within HCI, of participatory design. Usability and ownership have been achieved by involving the people who will eventually have to apply the guidelines. Although not sufficient in isolation of other constraints, gaining an appreciation of local concerns is certainly a necessary precursor to the success of any design enterprise.
5. CONCLUSIONS In this paper, we have argued that the design of procedures could benefit from a sensitivity to the practical concerns of workers. We introduced concerns as a way of developing the debate on the problematics of relating procedures to practice already joined by Suchman, Schmidt, Hutchins, and others. Our contribution, in this regard, is to highlight the moral dimension of conceptualisations of moment-tomoment transactions between workers and work settings, including procedures. This leads to a reframing of work as open-ended activity involving workers making everyday judgements of value, and procedures as one expression of oughtness in that activity. Workers are seen as moral agents taking responsibility for their actions and working out relations between procedures and work practice with respect to their personal experience of moment-to-moment oughtness. In this context, we use concerns as a shorthand for referring to the foci of moral action, the things that matter to actors engaged in the action.
Several specific characteristics of concerns were identified in evaluating the use of procedures in safety critical settings. Pilots were concerned with the likely effectiveness of and rationale for their Standard Operating Procedures (SOPs), and their concerns seemed to stretch over time. Their annotations of SOPs anticipated the consequences and contingencies of future actions and events. The concerns of ambulance controllers were for local efficiency of practice, which included specific concerns for fairness in the distribution of work and for the welfare of their colleagues. This contrasted with the concerns of ambulance control managers whose concerns, such as efficient resource deployment, were global. Finally the concerns of the supervisors in the O-ring case reflected histories and cultures of oughtness distributed across a social and organisational space. In each case, workers adapted procedures to value distinctions reflective of a history and culture of work experience. One of the practical contributions of a sensitivity to concern is to provide a perspective on work which offers a language for critical dialogue not otherwise easily available. It could be used to enhance Contextual Design (Beyer & Holzblatt, 1998) by adding a critical register to its work modelling. For example, the culture of ambulance control is characterised by concern for fairness and commitment to colleague welfare, values which are not easily captured in a model of culture reduced to influence. It is also characterised by a clash between faithfulness to local and global concerns, a conflict which requires polyphony in work modelling, and which problematises Beyer and Holzblatt’s separation of work and system design. Contextual Design is rich in techniques for collecting the data which we would require to develop an account of concerns in a work setting but a sensitivity to
concerns, and the reframings of work and procedures entailed, would enhance the critical capacity of Contextual Design. The practical focus of this paper was the design of useful procedures. Aside from pointing to activity-based frameworks as appropriate for conceptualising procedure design, we developed some insights into the design process from a case study of the implementation of clinical guidelines. We are open to the criticism that we cast the net too wide when we treat guidelines as procedures. But, of course, that is the point of all the reframing initiated by a sensitivity to concerns, that procedures should be guidelines and that guideline design should be dialogical. Designing practically useful procedures is a matter of designing for education, reflection on the interplay between means and ends, not for control. ______________________________________ Acknowledgments: We thank Ian Russell, Health Economics, and Stephen Pocock, Computer Science, University of York, for their contributions, the HCI review editor Lucy Suchman for her careful reading of our work, and three anonymous reviewers whose comments helped us to present our ideas more clearly. Support: Forbairt-British Council Grant No. BC/96/037 and UK ESRC Cognitive Engineering Programme, Grant L127251024. Authors’ Addresses: John C. McCarthy, Department of Applied Psychology, University College Cork, Ireland. Email: mccarthyj@ucc.ie. Peter Wright, Department of Computer ScienceUniversity of York, Heslington, York, YO1 5DD, UK. Email: peter.wright@cs.york.ac.uk. Andrew Monk, Department of
Psychology, University of York, York YO1 5DD, UK. Email: AM1@york.ac.uk. Leon Watts, Equipe IIHM, Laboratoire CLIPS-IMAG, BP 53, 38041 Grenoble Cedex 9, France. Email: l.watts@psych.york.ac.uk _________________________________
REFERENCES Bakhtin, M.M. (1993). Towards a Philosophy of the Act. Austin, Texas: University of Texas Press. Bannon, L. & Bodker, S. (1991). Beyond the interface: Encountering artefacts. In J. Carroll (Ed.), Designing Interaction: Psychology at the Human Computer Interface (pp. 227-253). Cambridge: Cambridge University Press. Berg, M. (1997). Rationalizing medical work: decision support techniques and medical practices. Cambridge Mass.: MIT Press. Beyer, H. & Holtzblatt, K. (1998). Contextual design: defining customer-centred systems. San Francisco: Morgan Kaufmann. Bodker, S. (1997) Creating conditions for participation: Conflicts and resources in system development. Human-Computer Interaction, 11, 215-236. Bourdieu, P. (1998). Practical reason: on the theory of action. Cambridge: Polity Press. Card, S.K., Moran, T.P. & Newell, A. (1983). The psychology of human computer interaction. Hillsdale, NJ: Lawrence Erlbaum Associates.
Carroll, J.M. (1990). Infinite detail and emulation in an ontologically minimized HCI. Proceedings of the CHI’90 Conference on Human Factors in Computer Systems, 321-328. New York: ACM. Checkland, P. & Scholes, J. (1990). Soft Systems Methodology in Action. Chichester: Wiley. Cole, M. (1996). Cultural Psychology. Cambridge Mass.: Belknap Press of Harvard University Press. Cole, M. & Engestrom, Y. (1993). A cultural-historical approach to distributed cognition. In G. Salomon (Ed.), Distributed Cognition: Psychological and Educational Considerations (pp. 1-46). Cambridge: Cambridge University Press. Dunne, J. (1993). Back to the rough ground: ‘phronesis’ and ‘techne’ in modern philosophy and Aristotle. Notre Dame: University of Notre Dame Press. Eccles, M., Clapp, Z., Grimshaw, J., Adams, P.C., Higgins, B., Purves, I. & Russell, I. (1996). North of England evidence based guidelines development project: methods of guideline development, BMJ , 312, 760-2. Effective Health Care (1994). Implementing clinical practice guidelines: Can guidelines be used to improve clinical practice? Bulletin No.8. Leeds: University of Leeds. Engestrom, Y. (1993). Developmental studies of work as a testbench of activity theory: The case of primary care medical practice. In S. Chaiklin and J. Lave
(Eds.), Understanding Practice: Perspectives on Activity and Context (pp. 64103). Cambridge: Cambridge University Press.. Grimshaw, J.M. & Russell, I.T. (1993a). Achieving health gain through clinical guidelines, I: developing scientifically valid guidelines. Qual Health Care , 2, 243-8. Grimshaw, J.M. & Russell, I.T. (1993b). Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet, 342, 1317-21. Grimshaw, J.M. & Russell, I.T. (1994). Achieving health gain through clinical guidelines, II: ensuring guidelines change medical practice. Qual Health Care , 3, 45-52. Hawkins, F.H. (1987). Human Factors in Flight. (2nd ed.). Aldershot, UK: Avebury Press. Hicks, D. (1996). Learning as a prosaic act. Mind, Culture, and Activity, 3, 102118. Hutchins, E. (1995). Cognition in the Wild. Cambridge Mass: MIT Press. Institute of Medicine (1992). Guidelines for clinical practice: From development to use. Washington, DC: National Academy Press. Joas, H. (1996). The Creativity of Action.. Cambridge: Polity Press. Kaptelinin, V. (1996). Computer-mediated activity: Functional organs in social and developmental contexts. In B.A. Nardi (Ed.) Context and Consciousness: Activity Theory and Human-Computer Interaction (pp. 45-68). Cambridge Mass: MIT Press.
Kirwan, B. & Ainsworth, (1992). A Guide to Task Analysis. London: Taylor & Francis. Kuutti, K. (1996). Activity theory as a potential framework for human-computer interaction research. In B.A. Nardi (Ed.), Context and Consciousness: Activity Theory and Human-Computer Interaction (pp.17-44). Cambridge Mass: MIT Press. Landauer, T.K. (1991). Let’s get real: A position paper on the role of cognitive psychology in the design of humanly useful and Useful systems. In J. Carroll (Ed.), Designing Interaction: Psychology at the Human Computer Interface (pp. 60-73). Cambridge: Cambridge University Press. Lave J. (1988). Cognition in Practice: Mind, Mathematics and Culture in Everyday Life. Cambridge: Cambridge University Press. Lave, J. (1993). The practice of learning. In S. Chaiklin & J. Lave (Eds.), Understanding Practice: Perspectives on Activity and Context (pp. 3-32). Cambridge: Cambridge University Press. Lave, J. & Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation. Cambridge: Cambridge University Press. Martin, D. Bowers, J. & Wastell, D. (1997). The interactional affordances of technology: An ethnography of human-computer interaction in an ambulance control centre. In H. Thimbleby, B. O’Conaill, & P. Thomas (Eds.), People and Computers XII: Proceedings of HCI 1997 (pp. 263-282). London: Springer.
McCarthy, J., Wright, P., Healey, P., Dearden, A. & Harrison, M.D. (1997). Locating the scene: The particular and the general in contexts for ambulance control. Proceedings of Group'97, the International Conference on Supporting Group Work, 101-110. New York: ACM. McCarthy, J, Wright, P., Healey, P. & Harrison, M. (1997). Human error in context: Accountability of work activity in high-consequence work systems. International Journal of Human-Computer Studies, 47, 735-766. NTSB (1984). Eastern Air Lines Lockhead L-1011, N334EA Miami International Airport, FL, May 5 1983. National Transportation Safety Board Report No. AAR 84/04. Washington DC: NTSB. Open Systems Group (1972). Systems Behaviour. London: Open University / Chapman Publishing. Orr, J.E. (1996). Talking about machines: an ethnography of a modern job. Ithaca, N.Y.: Cornell University Press. Sachs, P. (1995). Transforming work: Collaboration, learning, and design. Communications of the ACM, 38, 37-44. Schmidt, K. & Simone, C. (1996). Coordination mechanisms: Towards a conceptual foundation of CSCW system design. Computer Supported Cooperative Work: The Journal of Collaborative Computing, 5, 155-200. Schmidt, K. (1997) Of maps and scripts: The status of formal constructs in cooperative work. Proceedings of Group'97, the International Conference on Supporting Group Work, 138-147. New York: ACM.
Simon, H. A. (1990). Invariants of human behaviour. The Annual Review of Psychology, 41, 1-19. Strauss, A. (1988). The articulation of project work: An organisational process. The Sociological Quarterly, 26, 1-19. Suchman, L.A.(1987). Plans and situated actions: The problem of human computer interaction. Cambridge: Cambridge University Press. Suchman, L. (1993). Response to Vera and Simon’s situated action: A symbolic interpretation. Cognitive Science, 17, 71-75. Tobach, E., Falmagne R.J., Brown Parlee, M., Martin, L.M.W., & Scribner Kapelman, A., (1997). Mind and Social Practice: Selected Writings of Sylvia Scribner. Cambridge: Cambridge University Press. Woods, D.D. Johannesen, L.J., Cook, R.I. & Sarter, N.B. (1994) Behind human error: Cognitive systems, computers, complexity and hindsight. CSERIAC State of the art report. CSERIAC 94-01. Wright, P.C. Pocock, S. & Fields, B (1998). The prescription and practice of work on the flight deck. Proceedings of ECCE9: Ninth European Conference on Cognitive Ergonomics, 37-42. EACE: INRIA France.
FIGURE CAPTIONS Figure 1: Extract from a typical B-757 normal checklist highlighting pre-flight phase items Figure 2: Analysis of annotations to a pilot’s quick reference handbook Figure 3: Desirable attributes of clinical guidelines, from 'Effective Health Care' Number 8, December 1994. Figure 4: Comparisons of approaches to designing and implementing procedures and guidelines
Figure 1
757 NORMAL CHECKLIST BEFORE START GEAR
PINS
....................................
OXYGEN
ON
BOARD
...........................................CHECKED
PASSENGER FLIGHT
SIGNS
...........................AUTO
INSTRUMENTS
PARKING FUEL
3
BRAKE
CONTROL
ALTIMETERS TAKEOFF
&
...............................
...................................
...
QNH
BRIEFING
SET
SET
...................................
CUT
ON
ON OFF
.................SET-X-CHECKED
........................
COMPLETED
CLEARED FOR START FUEL
............................................
AIRSPEED CDU TRIMS FLIGHT
BUGS
CONFIRMED
........................................
......................................................... ..........
ZERO-ZERO
CONTROLS
&
..................
...........................
SET SET UNITS
CHECKED
Figure 2
Type of annotation
Frequency
Percentage
of annotation cause of warning or condition
21
6.5
consequence of warning or condition
20
6.2
consequence of taking an action
32
9.9
addition of extra actions or
61
18.9
clarifying the reason for procedure
63
19.5
alternative procedure/ uncertainty
38
11.8
26
8.0
14
4.3
48
14.9
procedure
about effectiveness of current procedure delimiting the scope of a condition or procedure highlighting importance of particular steps additional relevant background systems information
Total
323
100
Figure 3 Attribute
Meaning
Validity
Correctly interpreting available evidence so that when followed valid guidelines lead to improvements in health
Cost effectiveness
Guidelines lead to improvements in health at acceptable costs.
Reproducibility
Given the same evidence another guideline group produce similar recommendations.
Reliability
Given the same clinical circumstances another health professional applies them similarly.
Representative development
All key disciplines and interests (including patients) contribute to guideline developments.
Clinical
Target population is defined in accordance with scientific evidence.
applicability Clinical flexibility
Guidelines identify exceptions and indicate how patient preferences are to be incorporated in decision-making.
Clarity
Guidelines use precise definitions, unambiguous language, and user-friendly formats.
Meticulous documentation
Guidelines record participants, assumptions and methods; and link recommendations to the available evidence.
Scheduled review Guidelines state when and how they are to be reviewed. Utilisation
Guidelines indicate ways in which adherence to recommendations
review
can be sensibly monitored.
Figure 4 Sample Characteristics
Clinical guidelines
Kanban
O Rings
Worker autonomy
Locally customised rather than
By bending procedure to deal with
By violating By violating procedure e.g. procedure e.g. foreman fitting FIFO
stipulated
circumstances
O Ring
Attends to patient concerns
No worker Ignored: fixed involvement in procedure design which does
Range of
Appears to be
Local context / Local knowledge
Ambulance control
despatch Excluded by prohibiting despatch of
contributions to flexible and development adaptive
not adapt to events or concerns
ambulances to areas they know well
Sense of investment
In design and implementation
Open local adaptation
Covert local adaptations
Covert local adaptations
Usable in practice
Patient symptom led
Coordination led
Violations to Violations to complete work complete work
Explicit about concerns
Patient welfare, In a limited clinical way with judgement, etc. autonomy issues
Formal accountability
Manager’s concerns for resource utilisation only
