AEB & TRGG
Design & re-design of representation
The design and re-design of a simultaneous representation of conceptual and device models Ann Blandford
&
Thomas Green
School of Computing Science Middlesex University Bounds Green Road London N11 2NQ
[email protected]
Computer Based Learning Unit University of Leeds
[email protected]
Abstract This paper reports on the design and re-design of an Ontological Sketch Model. OSM is a representation and method of usability assessment that is based on descriptions of the device model and the user’s conceptual model of that device. The ultimate aim of this work is to develop an approach that focuses attention on device and conceptual models, and that is usable, useful and widely available for use. We present the rationale for the design of the OSM. The initial design for the OSM was based on descriptions of the entities in the domain of application, actions that the user can perform, and relationships (such as “constrains” and “affects”) between entities. A formative evaluation of the usefulness and usability of this design showed that the approach was promising, but also highlighted limitations of the representation. In particular, novice analysts had difficulty writing descriptions at an appropriate level of detail and understanding the meanings of some relationships. This led to a re-design of the notation. The re-design is still based on descriptions of entities, actions and relationships, but provides stronger constraints on what can be expressed, reducing the possibility of novices producing inappropriate descriptions and also providing a better basis for formalising usability properties. We have also placed greater emphasis on formalising the fit or misfit between device and conceptual models.
Introduction Usability is a central concern in the design of interactive systems. Within Human-Computer Interaction, there are two main ways of incorporating user concerns in the design lifecycle: approaches that inform the design of a system, such as task analysis or the use of guidelines, and approaches that help in the evaluation of a design, such as heuristic evaluation or cognitive modelling techniques (Blandford, forthcoming). We are concerned with usability evaluation — with providing tools that designers can easily use to evaluate proposed designs. Lightweight approaches such as heuristic evaluation (Nielsen, 1994) are informal and highly reliant on craft skill. Such methods can be very effective in the hands of a skilled analyst, but they are dependent on the skill of the individual. In principle, theory-based methods utilising established results from cognitive psychology are less dependent on craft skill, can yield a deeper understanding of usability problems and appropriate solutions and can be applied earlier in the design process when fewer commitments have been made. However, while there is significant work on theory-based methods of usability assessment within the research community, there has been little take-up of these methods by industry (Bellotti, 1989). There are many reasons for this, including the difficulty of technology transfer between the academic developers of such methods and the industrial users (Buckingham Shum & Hammond, 1994). The long-term aim of the work reported here is to deliver an approach to usability evaluation that focuses on representations of the system (or device) model and the user’s conceptual model, and that can be applied by members of a design team at comparatively low cost, without depending too heavily on craft skill or expertise in cognitive science. We refer to our approach as Ontological Sketch Modelling — ‘Ontological’ because it is centred on entities, rather than user tasks, and ‘Sketch Modelling’ because it avoids exhaustive analysis, as described below. The user’s conceptual model is central to the approach, so merits some explanation. It describes the user’s model of the domain of application. It can be difficult to create an accurate user’s conceptual model, but sources of information might include a requirements specification (identifying the domain tasks the device is intended to support, and the domain entities involved),
1
AEB & TRGG
Design & re-design of representation
more formal domain modelling, re-use of models describing other systems for the same domain, the analyst’s own experience of working in the domain and empirical results. The aim of this paper is to present the first iteration in the design of this approach, including the background work that has most influenced the approach, the design aims and the results of a formative evaluation of the first design. A small note on the scope of our approach is needed. For practical reasons, we have restricted the research reported in this paper to the simple case of one user working with one computer-based application. But there is no such limitation in principle. Analyses not reported here include everyday domestic devices such as digital and analog watches, and future work will extend to cooperative work systems.
Background Cognitive task analysis in one shape or another has dominated most techniques of HCI usability evaluation for many years. Some notable successes have been achieved which should not be belittled, but we believe that it is time to develop alternative approaches. One very well-known form of task analysis is the GOMS/KLM family of models, recently reviewed by John & Kieras (1996), in which the modeller takes a list of tasks that the user needs to perform, and develops a method for each task using the device in question. Reasonably accurate predictions of time-toperform can then be made using observed times for elementary actions such as making a keystroke. An alternative, focusing more directly on the device and less on the user (e.g. ignoring mental operations), is the User Action Notation (Hartson, Siochi & Hix, 1992); this too is taskbased, but because more detail is recorded about how the device responds to each user action, it is more informative about useful feedback from device to user. Yet a third form is Task-Action Grammar (Payne and Green, 1986), in which tasks are differentiated by elementary features, and by mapping task features onto high-level generative action rules it is possible to examine the internal consistency or regularity of the device’s action language. Despite their great differences, these approaches have much in common. They demand very detailed analysis, which is as unsuitable for the quick pace of exploratory design as it is infeasible for the non-specialist modeller; their closed dictionary of tasks makes no concession to how real users find novel solutions to novel problems; and they bypass completely the possible differences between the task as perceived by the user, and the task as allowed by the device. To illustrate the last point, consider the editing of a document containing numbered sections. A GOMS analysis would predict the time taken to create a new numbered section heading for a certain wordprocessor, a UAN analysis would note that the word processor gave visual feedback when a word was selected, and TAG analysis would show whether inserting a new section number was similar to inserting a new figure number, say. None of these would, in any explicit way, alert the analyst to the need to update the subsequent section numbers, nor to the problem that those numbers may have to be updated one by one instead of all at once – examples respectively of knock-on viscosity and repetition viscosity (Green, 1990), discussed in more detail below. In activity theory terms there is a breakdown in the activity. An OSM should support reasoning about the relationships between assigned section numbers and their sequential position, and generate suitable alerts to warn of a misfit between the domain-level action (‘insert new section’) and the device level-actions required. To predict viscosity and allied problems, at least some degree of knowledge analysis is needed, so that mismatches can be revealed. OSM draws on Programmable User Modelling (PUM; Young, Green & Simon, 1989; Blandford & Young 1996). In the knowledge analysis stage of PUM, the knowledge the user needs (in terms of conceptual objects they work with and preconditions and effects of operations) is laid out, and an account is given of where that knowledge might come from. Blandford and Young (1996) give an account of how mismatches between the user’s knowledge and the device model can lead to usability difficulties. In full PUM analysis that knowledge is then expressed formally in an ‘Instruction Language’ and cognitive modelling techniques are used to reason about likely user behaviours when interacting with the device. OSM demands much less rigour than a full PUM knowledge analysis, and allows a wider range of concerns to be expressed within the model. OSM does not attempt cognitive modelling, which is an extremely difficult task, but instead reports likely problems arising from the relationship between the device and the user’s conceptualisation of it. Such a misfit analysis, as we might term it, has been attempted at least twice in the HCI literature, but has not become a strong tradition. Moran’s (1983) ETIT analysis mapped the ‘external’ task of the domain onto the ‘internal’ task of the device, from which an efficiency metric could be
2
AEB & TRGG
Design & re-design of representation
computed, essentially the number of device actions required to achieve one domain-level goal. Payne (1993) drew on ETIT for his ‘Task-Entity Analysis’ by which he explained the low usage of early electronic diaries and calendars. This approach started from the conceptual entities described by calendar users, supplemented by entities inferred from psychological research, and posed the question whether users of electronic diaries could use the conceptual entities in the required ways. Payne found that intentions to do things formed the main class of things-to-beremembered, and that such intentions can be nested and also have a dependency structure. Intentions have relative importance: there are ones that it would be catastrophic to forget, and others that are just maybes. Finally, intentions may be ‘pulses’, to be enacted at a particular time, or ‘steps’, which have a tolerant window of opportunity (Ellis, 1988); and moreover, a ‘point’ in time might be related to a personal or a conventional time line. Payne next looked at how a number of diary users indicated these attributes of intentions, finding that users of paper diaries could make use of variations in writing size, scribbles and arrows, and vaguely-specified times to convey all those different attributes of intentions. Users of electronic diaries did not, at that time, have a rich enough medium to represent what diaries need to contain, so that electronic diaries were unsatisfactory. Payne’s approach was entirely informal and could not be attempted without knowledge of HCI and cognitive psychology; also, it was not powerful enough to yield alerts for viscosity, etc. Nevertheless, it demonstrates what can be achieved by considering misfits between the system model and the user’s conceptual model. To understand the relationships that generate misfit, it is essential to have a method to describe information structures. Here we draw on the tradition of domain modelling, and in particular on ERMIA, or entity-relationship modelling for information artefacts (Green and Benyon, 1996). Entity-relationship modelling is a well-known tradition in information science, with a moderate degree of formality and a well-understood (if slightly problematic) semantics. Green and Benyon treated devices as tools to manipulate information structures, and information displays and representations as static representations of structures. This allowed a certain amount of scope for exhibiting misfits between users’ conceptual models and the structures imposed by the device. We draw on that achievement, especially on the choice of the ‘entity’ as the main object of analysis rather than the task, but in several ways ERMIA is unsuited to the aim of finding a useful, usable approach to usability evaluation. ER analysis (in its pure form) allows entities to be related in only one way, a simple assertion that A is related to B, and the power of the technique comes from analysing the cardinality and optionality of relationships. There is no provision for modelling the consequences of actions, for example. Also, ER modelling, and therefore ERMIA modelling, requires a detailed and rigorous analysis of the device, going well beyond the sketchiness that we believe is a prerequisite of a tool that will be usable in a design context. The last tradition we draw on is a broad-brush framework developed by Green (1991), the ‘cognitive dimensions of notations’, which was intended specifically to raise the level of discourse by providing ‘discussion tools’. A discussion tool is a concept that is easily understood, sufficiently general but not too bland, and captures an important quality. By providing a small vocabulary of such concepts, together with examples and some analysis of how they are interrelated, Green argued that it would be possible to give non-HCI specialists a more sophisticated mode of analysis than merely counting features. By its nature, this is an approach that is easy to learn. In addition, the ideas have been taken up by others (e.g. Lavery, 1996; Roast & Siddiqi, 1997). Many of the dimensions defined by Green can in retrospect be seen to depend on the internal relationships of the information structures, such as ‘hidden dependencies’ between one entity and another; others are examples of misfits —for example, the viscosity examples mentioned above, or ‘premature commitment’, described below. The cognitive dimensions have, to date, been presented entirely informally, and the method of analysis was no more than asking a designer to think carefully about each dimension in turn (e.g. Green & Petre, 1996). With OSM, we aim to evaluate as many as possible of the dimensions algorithmically, once a description of the artefact has been constructed.
Design aims In the previous section, we have outlined some of the approaches that have informed the design of OSM, whether directly (such as ERMIA and PUM) or indirectly. In addition, the primary aims in designing the representation and method is that it must be both useful and usable. A modelling technique is useful if it helps the analyst gain insights into usability issues that might well have been missed otherwise. No modelling technique can guarantee to identify problems, as it is a tool that depends on the analyst for effective use, but it should help the analyst to identify particular usability concerns. Also, any modelling technique is of limited scope; what is needed is
3
AEB & TRGG
Design & re-design of representation
not a universal tool, but one whose usefulness is clearly scoped. Particular issues that we aim to address include identification of Green’s Cognitive Dimensions (Green & Petre, 1996) and of mismatches between device and user’s conceptual models (e.g. Blandford & Young, 1996). A modelling technique is usable if it is easy to learn and to understand. For any technique to be usable, the notation needs to have a clear syntax and semantics, and an accompanying methodology. To achieve an appropriate balance between usefulness (covering a broad range of concerns) and usability (not requiring too much initial input) the approach is designed to be incremental, in the sense that people can achieve some useful results quite quickly, but can also get more powerful results as their expertise develops. As an analyst adds more information to a model, they may gain more insights into its essential structure through the process of constructing and refining the model, and they can apply more reasoning (manual or automated) to it to assess more aspects of the design. The ontology of the OSM is determined by the properties that the model should allow us to reason about, such as viscosity and premature commitment. That is: if we wish to reason about these aspects of a design, we need to describe that design is appropriate terms. Motivations for the particular approach we are taking include: •
we are drawing extensively on existing research, as summarised above, to guide the development.
•
however, we are avoiding representations that have users’ tasks or plans as a central component, partly because such representations have been comparatively well investigated (e.g. in work on GOMS, Cognitive Walkthrough (Wharton et al, 1994), and other approaches to cognitive task analysis), partly because such approaches are poorly suited for many classes of interactions (e.g. with graphics programs, games software, etc.) and partly because a taskoriented approach does not support reasoning about many important usability properties.
•
there are many features of a device that make it more or less well suited for different domains of application. We want to allow analysts to be able to assess the usability of a device in relation to a particular domain (for example, in terms of the closeness of the mapping between user’s conceptual model and device model) by being able to talk explicitly about both domain and device entities, and various properties of both.
•
while we are avoiding a task-oriented approach, basing our representation instead on entities and relationships, we see a representation of actions as being essential for reasoning about changes to the device state.
•
in aiming for a ‘sketch’ approach, we are trying to achieve an appropriate balance between precision and completeness on the one hand, and informal description on the other. We view the notation of the modelling as providing affordances that support the analyst’s reasoning (for which — in the tradition of formal methods — the process of producing the model using a constrained language can help provide insights); also, precision and consistency are needed for automated (computer-based) reasoning. However, the need to produce a formally complete and consistent model should not become a straight-jacket for the analyst.
•
it is frequently possible to produce descriptions at different levels of detail, or focusing on different aspects; each of these can serve a value in addressing particular usability issues, and we do not believe that there is a single “correct” model of any system, even using the same modelling language. We wish to support an incremental approach to analysis that allows the designer to supply only as much detail as is needed for a particular analysis.
•
while usability alerts can be generated by eye for small models, for larger models we expect the inferences to be made by automated analysis; the notation needs to support this.
Design is never simply a matter of stating requirements and expecting the perfect solution to emerge; it is an iterative process of proposing possible solutions, testing and refining them. This is as true of a usability evaluation method as of system design. We now present the initial design of an OSM and a formative evaluation of that design that guided design modifications.
Initial design of OSM The initial design was based on three separate, loosely inter-related, tables that allowed the analyst to describe entities, actions and relationships, with a checklist of points to be alert to when assessing usability. Our presentation of this design is based on the tutorial material that was used
4
AEB & TRGG
Design & re-design of representation
in the formative evaluation (described below). This tutorial material included brief examples from MS Word (version 5 for the Macintosh) to exemplify the modelling approach. Entities
The analyst lists the ‘things’ that the user has to know about, such as ‘character’, ‘word’, ‘paragraph’, ‘column-width’. For each of these entities, the analyst notes down any of the following that are significant: attributes
what attributes an entity has. For example, a character in a document has a font, a size, etc. accessibility
whether the entity is user-private – an idea in the head of the user, such as the plot of a novel; device-private – something the user can’t change easily, maybe not even see, but has to know about, such as the order in which drawing object are laid down, or shared, such as a scrollbar. User-private entities are typically components of a domain model that are not explicitly represented within the device; they may be things that users will want to manipulate, but which they cannot work with easily using this device. relevance
whether the entity is relevant to the domain of application, or just to the device. For example, a word is relevant to the domain of writing and typesetting (domain-relevant), but a scroll-bar is only relevant to the word-processor device (device-relevant). A user who is familiar with the domain of application will have to learn about the entities that are only device-relevant to be able to use this particular system effectively. persistent visibility
whether an entity can be seen by the user. For example, a word is persistent (always visible), whereas the offspring of a style in the style hierarchy are invisible to the user. The symbols for newline, tab etc. that can be revealed in MS Word (if you know how) are transient. disguise
whether an entity has a meaningful name, or a meaningful symbol, or an icon that would let you guess its meaning if you didn’t already know; or whether it is disguised. In MS Word, the tool for changing column widths in tables is disguised, and new users have a lot of trouble finding it sometimes. (The icon originates in the markings on typewriters, now unfamiliar to today’s users.) change column width
Actions
The analyst lists the types of action users can do, such as selecting something (a paragraph, a word, etc.), and what the actions can be applied to. It is not necessary to list all the different actions, but to focus on the interesting actions that require new ideas, or that require users to know about some special tool or some other entity (such as the column width tool illustrated above). An action description consists of: action
what the user does entity
what they do it to effect
what effect that action has context
anything particular that has to be true about the state of the device for that action to have that effect Relationships
Relationships connect two or more entities. We recognise three particular kinds of relationships: consists-of, affects, and constrains. In Word, a paragraph consists of sentences, sentences consist of words, words consist of characters, etc. Changing the style attribute of one of those entities will affect everything it consists of (by inheritance).
5
AEB & TRGG
Design & re-design of representation
An affects relationship connects entities A and B when changing an attribute of A causes (or may cause) a change to an attribute of B, but they are not connected by consists-of. For example, the pagination algorithm in Word is such that adding to the amount of text preceding a table in a document may cause a new page to be started within the table (number of characters affects position of page-breaks). A constraint relationship simply asserts that certain attributes of entities are in some way constrained; for instance, typographical convention requires all the lines of a table to lie on the same page if possible. For each relationship, the analyst should record type
consists-of, affects, constrains, or another kind entities
the two (or more) entities that are related. Reasoning with the representation
The purpose of writing out the OSM is to help the analyst to spot potential usability difficulties. While writing the OSM, or afterwards, the analyst can check for potential difficulties. These include: •
Is there a simple mapping between the underlying model and the surface presentation? If not, is this likely to cause the user difficulties?
•
Are there actions that users can perform that cause irreversible changes to the system?
•
How easy is it for the user to modify the things that matter to the user? [The analyst should focus particularly on how easy it is to change attributes or relations concerning user-private entities that have no direct device representation.] This may indicate “repetition viscosity” (Green 1990), in which changing an attribute that has a simple meaning to the user involves repetitive device actions.
•
Is the real-world meaning of entities / attributes in the device representation always clear?
•
Are there non-obvious dependencies between the context and the effect of a user action? (Does the same action have substantially different effects depending on the context? Will this always be obvious to the user?)
•
Are there non-obvious relationships between the entities the user manipulates and the effects they have on other entities? (E.g. in a word-processor adding or deleting text can affect where page-breaks fall). This may indicate “knock-on viscosity” (Green 1990), meaning that changing one attribute may lead to the need to adjust other things to restore the values of others.
•
Are there actions / entities that the user is unlikely to discover for any reason?
•
Are there actions whose effects are unpredictable to the user for any reason?
An Illustration: J-Sketch
To illustrate the use of the first OSM notation, we apply it to a prototype sketching program, J-Sketch (Rieman, 1996). In brief, J-Sketch allows the user to produce sketchy drawings based on Bezier curves (see figure 1). The user has a choice of two methods of creating curves; in “ c l i c k ” mode, the user lays down individual points, and the most recent 4 points are used by the program to construct each line in the curve; in “drag” mode, the user drags the mouse to form the curve, and points are laid down automatically at 4 per second. The user has a choice of three colours (black, grey and white) and 3 pen-widths. There are 5 drawing layers; the layers can be reordered by clicking on tabs at the edge of each layer, and each layer can be made visible or invisible by clicking on the grey tab-edges.
6
AEB & TRGG
Design & re-design of representation
Figure 1: A J-Sketch screen. Layer 3 is currently active; any marks on layers 0 & 1 will be visible, while those on layers 2 and 4 will not.
J-Sketch was one of the programs used for the formative evaluation presented in the next section. As described in more detail below, part of the evaluation involved comparing OSM’s usability predictions against observed user behaviour. The model and analysis presented here were those used for evaluation. This OSM description was constructed largely from the page of user notes distributed with J-Sketch.
7
AEB & TRGG
Design & re-design of representation
J-Sketch Entities
The main entities and their properties are listed in the following table: Entity
Attribute
Accessibility R e l e v a n c e P e r s i s t e n c e Disguise
domain-object shape user sketchy-line shape, sketchiness, shared colour and width layer current/other; shared visible/invisible
domain domain
persistent persistent
device
persistent
mode point pen undo icon
device device domain device
transient invisible persistent persistent
discrete / continuous shared position device width and colour shared shared
tabs not very salient; tab edges disguised
disguised
The main points to note here are that the domain-objects (the “things” the user is sketching a picture of — e.g. the whale, eye, tail, fin, mouth, etc. in figure 1) have no explicit device representation; conversely, the individual points are not accessible to the user. J-Sketch Actions
The main actions available to the user are listed in the following table: Action click
Effect lay down a point for a sketchy-line lay down shape for sketchy-line
Context discrete mode
Notes
continuous mode
press onscreenbackspace / select menu-undo / press keyboarddelete hide-layer
undoes the most recent point
Works repeatedly through current curve (but cannot undo to a previous curve).
speed of dragging affects sketchiness 3 alternative actions all achieve the same effect
clear-layer
deletes all drawing from layer
select-pen
sets width of lines for next curve; starts a new curve sets colour of lines for next curve; starts a new curve finish-drag-curve finish-point-curve
drag
select-ink lift mouse button press space-bar, or click on pen or ink
Object drawing area drawing area
hides content of layer
layer re-appears if user starts drawing in it semantically confusable with “hide layer”.
drawing in drag mode drawing in click mode
no visible cue
The principal usability concerns in relation to actions are noted in the final column of this table. One additional point to note is that there is no facility to “redo”, nor to reverse the effect of clearing a layer.
8
AEB & TRGG
Design & re-design of representation
J-Sketch Relationships
The final stage of constructing the OSM is to list important relations. In this case, the program is very simple, so we have only identified a few relations: Type consists-of
Entity sketchy-line
Entity points
consists-of
domain-object
sketchy-lines
is on
curve
layer
Notes (at least 4 to be visible)
can be difficult for users to find out this relationship if they forget it.
As discussed below, this set of relationships omits one important one — that a sketchy-line can consist of (or denote) several domain objects, as well as the converse being true. The omission does not imply any weakness in OSM’s expressiveness; the modeller simply overlooked it. Usability Notes
Overall, the process of constructing the OSM description indicated that in most respects the system is likely to be easy to use. In noting usability concerns of this prototype system, some issues arise from the process of generating the description; others arise from reasoning about aspects of it, such as the consequences of relationships holding. The main usability concerns identified through constructing this OSM description were as follows: •
We originally included a point as an entity when we tried to describe the effect of the undo action (which is that it undoes the last point). Undoing is meaningful (if limited) in click mode, but not at all in drag mode. This highlights the inappropriateness of “undo” in relation to the entities the user is working with (particularly in drag mode).
•
There are three ways to undo, including pressing an on-screen button and selecting an item from a pull-down menu. This might cause confusion, since there is only one way to achieve most other effects.
•
Since 4 points are needed to compute a line, the first three clicks in click mode have no visible effect. This might be disconcerting for the novice user.
•
Curves are on layers. This relationship is always known to the device, but it has to be remembered by the user. The contents of a layer can only be found out by hiding it and then revealing it, to see what disappears and reappears.
•
The user has to be aware of what mode the system is in; clicking in drag mode just makes lots of invisible points, while dragging in click mode only lays down the point at which the user presses the mouse button down. This system behaviour is likely to be confusing if the user ever changes mode.
•
The drag action has property “speed” which affects the sketchiness of the curve (by affecting the spacing between points). Users need to understand this relationship if they wish to control the sketchiness of the curve.
•
The tabs for making layers visible / invisible are not obvious to a novice user.
This OSM description and set of usability concerns was made just on the basis of reading the notes supplied with the program, and using the program sufficiently (usually interleaved with writing the description) to be able to produce an adequate description. (That is: the requirements of writing the description in OSM terms raised questions about how the program works, which could only be answered by running it and finding the answers. In a design context this would entail asking focused questions of the designer or another expert, exactly like any other form of documentation.) Below, we compare the results of that sketch-modelling with the results of an empirical study of the use of J-Sketch.
A formative evaluation of the initial design of OSM As outlined above, our aim in this work is to produce a technique that is both useful and usable. To test this version of the OSM against these criteria, we conducted an informal study, focusing on curve drawing, in which we assessed the usefulness and usability of OSMs. Curve-drawing programs (J-Sketch and the curve-drawing facilities within ClarisWorks) were selected for this study for several reasons. Such programs are widely used and are currently a
9
AEB & TRGG
Design & re-design of representation
very active focus of development in the graphics world, but comparatively few usability analyses of them have been conducted. Curve-drawing programs are also inherently more fun and more interesting to investigate than word-processors or spreadsheets; our potential subjects had little previous exposure to them; and they are simple enough for some progress to be made within the necessarily limited time available, yet complex enough to defy full mastery within that time. The study, described below, was conducted with a group of 20 final-year undergraduate students who were enrolled on a module on HCI and Graphics. The results presented here are preliminary, and are being used to help highlight design modifications needed to the OSM approach. Assessing usefulness
To assess the usefulness of the OSM approach, we did an OSM-based usability analysis of each of the software packages we were using, and compared our results against empirical data. The analysis of J-Sketch is reported above; the analysis of ClarisWorks was substantially longer, and is omitted from this paper for the sake of brevity. Students were put into pairs, matched as closely as possible for prior experience. Each pair was allocated to one of the two drawing programs, and while partner A worked, thinking aloud, partner B noted what A did and what difficulties were encountered. Audio and video recordings were taken of two pairs (one using each program). Students were asked to draw a whale (pictures were provided as a guide), then to modify it by moving its tail; the session lasted for 1.5 hours. The following week, the same procedure was repeated, with each pair of students using the other program. However, this session was preceded by a seminar in which students were asked to describe the program they had used to a pair of student who had used the other program, so that each pair approached the second program with more prior knowledge. Every statement in the students’ notes was categorised in terms of difficulties encountered and things they found easily. From this, an overall pattern of difficulties was established, to be compared against the OSM-based prediction.
Figure 2: whale produced by student using J-Sketch
As predicted, most students learned to use the basic facilities of J-Sketch readily. This is illustrated graphically by the pair who were videoed using J-Sketch in the first week. Figure 2 shows the whale produced after 20 minutes of activity by a student who had never used this software before. In addition, the OSM explicitly captured nearly all the usability problems. The exceptions are as follows: 1) Although “the domain object” was identified as a user-private entity, the analysis did not extend to consider ways the user might manipulate the entity — for example, by filling it (to make the whale grey). Subjects actually found filling tedious and annoying (since filling with a sketchy line causes the line to go outside boundary, like a small child’s colouring in.) 2) The fact that there were 3 ways to undo and only 1 to clear was expected to cause confusion. This does not appear to have been the case. Most students understood the use of the “ < = “ on-screen undo button; few discovered the use of the delete key for undoing. The results for ClarisWorks are less clear-cut. Of the nine specific difficulties encountered by subjects, 4 had been predicted, but 5 had not. Additional predictions were made about aspects of the program that subjects never got as far as exploring. Some of the unpredicted difficulties were ones that an OSM analysis would not be expected to highlight; for example, students commented that they had difficulty getting the shape acceptable — a point that might emerge from several aspects of the system being difficult to work with, but not one that would emerge directly from an OSM analysis. Other difficulties were ones that could in principle have been detected, but were not. One example is that some users tried to select part of a device object (e.g. the tail or the mouth, which have domain significance) and manipulate just that part. They could not do this;
10
AEB & TRGG
Design & re-design of representation
this difficulty did not emerge from the OSM analysis because, as mentioned above, the analyst had not noted that a device object could represent several domain objects. Another example is that many students were confused about the effect of dragging the cursor in the drawing space (rather than clicking); the effect of dragging was not documented in the user manual, so it was not included in the OSM. Both of these difficulties are illustrated in the whale produced by the pair videoed using ClarisWorks in the first week (Figure 3); the very long snout is the result of them dragging the mouse in the drawing space, while the misshapen tail is a consequence of the difficulty they encountered in manipulating the tail.
Figure 3: whale produced by student using ClarisWorks
We have shown some examples of failure to use OSM to predict usability problems that “ i n principle” it could have been used for. These serve to emphasise the value of expertise — both in the use of the system being analysed (in this case, ClarisWorks) and in the use of OSMs. Nevertheless, the degree of success in this first usefulness evaluation indicates that the approach of basing a notation on entities, actions and relationships is worth pursuing. Assessing usability
To assess the usability of OSMs in a broad sense, we asked the same 20 subjects to produce their own OSM descriptions of the same systems, after the “usefulness” study had been completed, according to the following timetable: session 1
duration 1 hour
2 3 (1 week later) 4 in their own time
1.5 hours 0.5 hour 2 hours
activity training session outlining the OSM approach and working through a brief example (based on MS Word), produce OSM descriptions of J-Sketch short de-briefing on their J-Sketch analyses produce OSM descriptions of ClarisWorks produce a short usability report on the OSM methodology
The fact that these analyses were done after subjects had used each system for just one session meant that their OSM descriptions were based on a limited understanding of the systems they were describing. However, the focus for these sessions was less on the completeness of their descriptions and more on how accurately subjects understood the notation and how well they used their descriptions to support usability reasoning. Preliminary analysis indicates that: •
when describing J-Sketch all students except one described appropriate things as entities. (One student included some action descriptions, possibly because they appear as labels within pull-down menus.) However, most subjects only described shared entities, omitting either user-private or device-private ones. When describing ClarisWorks (after de-briefing on their J-Sketch analyses), some students did include user-private entities. All ClarisWorks descriptions were of appropriate entities, though some were at too fine a grain of detail to be really useful.
11
AEB & TRGG
Design & re-design of representation
•
similarly, all students (with one exception for J-Sketch) described actions in appropriate ways. For ClarisWorks, many of these were again at too fine a level of detail.
•
the descriptions of relationships were highly variable, and many were inappropriate. In particular, many were reiterations of action information (“doing X affects entity Y”).
•
the subjects’ usability notes did not clearly derive from their OSM descriptions, but from their experience. For example they included statements like “filling is tedious”.
Their subsequent usability reports on OSM confirm the evidence from the data — that entities and actions were easily comprehended and described, but that relationships presented more difficulties, and that few of the subjects really understood how the modelling was meant to be used to derive usability assessments of the system. While these initial results are promising, since they represent modelling activity after a very short period of training and practice, their greatest value lies in the way they have been used to inform re-design of the OSM. It should also be noted that the applications that we studied, being characteristic of their class, did not contain certain types of potential user problem
Revised design of OSM In the revised design, we address three particular aspects of the modelling that were either not explicit, or were inadequately addressed, in the first version. 1. the role of the modelling in supporting usability assessment
We can list explicitly the usability properties that we wish OSM to support. That is: the notation must allow one to generate clear definitions of particular usability properties, and these properties can be used to assess the appropriateness of a candidate notation. These precise definitions can be used to support automated reasoning in future. The target list of properties we are addressing is as follows: •
faithfulness of the mappings between the underlying device model, the surface representation and the user’s conceptual model
•
viscosity
•
premature commitment
•
order errors
•
irreversible changes
•
context dependency
•
discoverability
•
predictability
•
effect observability
•
discriminability of entities and actions
•
need to comprehend non-domain-relevant abstractions
This list may be modified in future, as the result of further design iterations. Below, we present the formalisation of viscosity and premature commitment to illustrate the relationship between a model and particular usability properties. 2. Producing clearer guidelines on what can appropriately be described as an entity or an action.
As noted above, subjects in the study sometimes included entities and actions at too fine a grain of detail, or even described individual menu labels as entities. This leads us to generate a more precise definition of an action: An action causes the creation or deletion of an entity, or changes an attribute of an entity. This definition, in turn, constrains what can be described as an entity or an attribute: An entity is something that can be created or deleted, or that has attributes that can be changed.
12
AEB & TRGG
Design & re-design of representation
An attribute is something whose value can be changed by an action, but that cannot be created or deleted. Based on these more precise definitions of entities, actions and attributes, we re-structure the tables presented for version 1 of the OSM to centre on entities and how they can be changed, with a subsidiary table to expand on the device-specific detail of actions. This is illustrated in the JSketch example below. Entities
The analyst lists the ‘things’ that the user has to know about, such as ‘character’, ‘word’, ‘paragraph’, ‘column-width’. For each of these entities, the analyst notes down any of the following that are significant: relevance
whether the entity is relevant to the domain of application, or just to the device, as before. accessibility
whether the entity is user-private, device-private or shared, as before. actions to add/delete instances
specifying the actions by which instances of the entity can be added or deleted. attributes
what attributes an entity has. For example, a character in a document has a font, a size, etc. actions to set/change attributes
specifying the actions by which attribute values can be set initially, or changed later. Additional information is provided either as annotations to the table (in line with the intentional “sketchiness” of the approach) or in an additional table that provides more information about actions. Actions and other information
If the analyst chooses to add more detailed information about actions, it can be done as follows: conceptual action
relating the action to its effect, as in the “Entities” table physical action
providing mode detail of the device-specific physical action through which the conceptual action achieves its effect entity
any device object (which is probably not included in the “Entities” table) the action is performed on disguised?
whether that device object is disguised in any way (e.g. has meaningful symbol or name), or whether it is easily confused with another device object persistent visibility
whether that device object is persistent, invisible or transient, as in version 1. context
anything particular that has to be true about the state of the device for that action to have that effect (including preconditions) 3. Clarifying the semantics of relationships
Without a very clear definition of each relationship, it was not possible to convey the use of relationships to subjects in the study. We now give these more precise definitions: Consists-of (1:m) relationships hold between entities: For example, a book consists of many chapters. Affects relationships hold between attributes. Changing the value of one attribute may change the value of another. Domain-constrains relationships hold between attributes of entities, such that one is a domain-relevant entity and the other is a device-relevant one. A domain-constrains relationship describes a domain requirement
13
AEB & TRGG
Design & re-design of representation
that the user has to achieve through the device. For example, a wordprocessing task may include the constraint that the document be less than a specified number of pages long. Device-constrains relationships hold between attributes of device-relevant entities. A device-constrains relationship is one that the user has to work within, not one the user has to strive to satisfy. E.g. a floppy disk has the attribute fullness which can range between empty and full-capacity. It was in developing these more precise definitions that it was recognised that there were two distinct types of constraint, which affect usability is different ways, and are therefore now represented differently. Informal reasoning with the representation
As with version 1, the analyst can use the representation to reason about usability issues, including: •
Is there a simple mapping between the device model and the surface presentation?
•
Is the user likely to make order errors?
•
Are there actions that users can perform that cause irreversible changes to the system?
•
Are there non-obvious dependencies between the context and the effect of a user action?
•
Are there actions / entities that the user is unlikely to discover for any reason?
•
Are there actions whose effects are unpredictable to the user for any reason?
•
Is the real-world meaning of entities / attributes in the device representation always clear?
•
Can the user see the effects of actions?
•
Are there device objects or actions that the user might confuse with each other?
In particular, the representation provides a means of reasoning about the relationship between the user’s conceptual model and the device model. Whether or not the representation can be used for particular usability reasoning depends on how much information the analyst provides. In addition, we are moving towards a point where the representation can be used for automated reasoning about usability properties. Formalising usability properties for automated reasoning
To illustrate how we formalise properties, we take two properties, viscosity and premature commitment, as examples. A proof-of-concept implementation has been produced (in Prolog) to demonstrate the feasibility of such automated reasoning. In what follows, we take E to be an entity of interest with an attribute A, and P to be some other entity with an attribute Q. viscosity
Informally, “viscosity” captures the idea that a system is difficult to change in some way. When we formalise this, we find that there are two sets of circumstances in which systems are typically viscous: 1) Repetition viscosity: changing an attribute that has a simple meaning to the user involves repetitive device actions. Changing attribute A of entity E, E(A), needs many actions if:
E(A) is not directly modifiable P(Q) affects E(A) P(Q) is modifiable E consists-of P For example, we get repetition viscosity on figure numbers in a document because whenever a figure is added, deleted or moved, a range of figures need to be re-numbered one by one. Similarly, we get repetition viscosity for changing figure numbers from Arabic to Roman numerals (“number-type”). Stated more formally:
set-of-figures(number-type) is not directly modifiable figure(number-type) affects set-of-figures(number-type) 14
AEB & TRGG
Design & re-design of representation
figure(number-type) is modifiable set-of-figures consists-of figure 2) Knock-on viscosity: changing one attribute may lead to the need to adjust other things to restore the values of others. Changing E(A) has possible knock-on if:
E(A) is modifiable modifying E(A) affects P(Q) there is a domain constraint on P(Q) The first example of repetition viscosity also suffers from knock-on viscosity because of the domain constraint that the figure numbers should be in numerical order through the document. Similarly, there is knock-on viscosity for inserting a new section in a document, but a powerful word-processor will typically include a feature to automatically re-number other sections, so in this case repetition viscosity is avoided.
premature commitment While the two types of viscosity had been recognised prior to the re-design of OSM, the various kinds of premature commitment had not. Informally, premature commitment occurs when the user has to provide information to the system earlier than they would choose to do. We have several sets of conditions that alert to possible premature commitment. 1) Non-modifiability premature commitment: An attribute cannot be changed after it has been set:
E(A) is settable E(A) is not modifiable There are examples of this type of premature commitment in J-Sketch — for example, the width and colour of a line cannot be changed; neither can the layer the line is on. 2) Abstraction-based premature commitment: In order to make something (which is not directly modifiable) easy to change later, conditions have to be set initially so that it can be easily changed via an affects relationship. This is achieved by creating an abstraction — i.e. establishing a grouping of entities in some way for which an attribute can be changed simultaneously.
E(A) is not directly modifiable P(Q) affects E(A) if is settable P(Q) is modifiable Examples of this kind of premature commitment include applying styles to paragraphs so that the appearance of all those paragraphs can be changed together (here, the is that the paragraph has a particular style), and linking lines and boxes together, in some drawing programs, so that the line moves with the box (here, the is that the two drawingobject points are linked together). This type of premature commitment can be characterised as creating an abstraction (a named style, or a link) in order to avoid subsequent repetition viscosity. 3) Device-constraint premature commitment: The third type of premature commitment involves setting an attribute of one entity that constrains the way new instances of another entity can be created:
E(A) is settable P(Q) is not settable There is a device constraint between E(A) and P(Q) It is possible to add more P’s One example of this is drawing a map on the back of an envelope: once the user has started drawing the map, each new instruction should be drawn in the appropriate direction on the envelope, but the envelope may not be big enough in that directions. Another example is that of setting the field width in a data structure when the size of all items to be entered in that field is not known:
15
AEB & TRGG
Design & re-design of representation
field(width) is settable item(width) is not settable field(width)>=item(width) more items can be added Example: J-Sketch revisited
To illustrate the revised OSM design, we apply it to J-Sketch. Now, the table appears as follows: entity line
relevance (dom/dev) dom
access (u/s/d) s
add by
delete by
attributes
press & drag 1 [drag mode]
draw over with white / undo repeatedly [current line 2 only] / clear layer containing line
width colour sketchiness
OR click repeatedly [at least 4 points] [click mode]
shape location layer [not inspectable] shown/hidden
domain
dom
u
drawing lines
deleting lines
shape
object
outline-type fill-type location layer shown/hidden
point
dev
d
layer
dev
s
pointer from action (which relates action to its effect) [1] press & drag [2] undo repeatedly
[3] select pen-width [4] click on tab edge change mode
wait with mouse button depressed [drag mode] / click [click mode] X
physical action
object
click
initial value defined by select pen-width 3 select inkwell drag speed / click locations
change value by X X X
mouse movement / click locations mouse position current top layer
X
always shown initially inherited from line inherited from line inherited from line inherited from line inherited from line initially set to shown click location
inherited from layer X
click on 4 tab edge [or start drawing on layer] click on tab
undo [most recent]
location
X
shown/hidden
initially set to shown
top / other
initially ordered
disguised object?
transient object?
context
n/a undo icon a bit
n/a n
press delete key
-
n/a
n/a
in drag mode limited to current line limited to current line
click click select-from-menu
nib icon tab edge mode
n y n
n n y
16
X X
X X X X inherited from line X
AEB & TRGG
Design & re-design of representation
The consists-of and affects relationships that hold in this case are represented graphically by the grey arrows on the table above. To summarise: Consists-of (1:m) relationships hold between entities: line consists-of point line consists-of domain-object (one line can represent several domain objects) domain-object consists-of line (one domain object can be composed of several lines) Affects relationships hold between attributes: point location affects line location point location affects line sketchiness
point location affects line shape layer shown/hidden affects line shown/hidden if line is on layer line shown/hidden affects domain-object shown/hidden This description is clearly incomplete, but is sufficient to permit the usability reasoning presented earlier. In addition, it allows us to reason about premature commitment: 1) The X’s down the right hand column of the entities table indicate non-modifiability premature commitment. 2) We also have abstraction-based premature commitment: the user has to plan which layer to put a line on if lines form a show/hide object. (In the formative study reported above, some students “animated” the whale’s tail by putting different tails on different layers, then showing and hiding the layers.) In this case, exploiting transitivity of “affects” relationships:
domain-object(shown/hidden) is not directly modifiable layer(shown/hidden) affects domain-object(shown/hidden) if domain-object is on layer domain-object(layer) is settable layer(shown/hidden) is modifiable As these examples illustrate, achieving an appropriate balance between sketchiness and precision is not straightforward; further work is needed to refine this aspect of modelling.
Conclusions At the time of writing it would be idle to pretend that the OSM approach is mature, but we believe that from the work reported in this paper – the initial design, the formative evaluation, and the redesign – some useful lessons can be drawn, as well as some clear indications of how to proceed further. We shall summarise our thoughts into sections at different levels of generality: the goals of HCI, criteria for successful modelling approaches in HCI, some comparisons among different types of model, support for modellers, and the particular class of application we studied, viz. curve-drawing. The Goals of HCI
Researchers and practitioners in HCI have many different goals, of course. Some HCI workers are design innovators, some are educators, some are developers. Some develop means of evaluating and assessing the efforts of designers, and we number ourselves within that group. In all these communities it is easy to forget the existence of the other communities, and to speak as though HCI were only ‘about’ one’s own community. As a result, it is quite possible to develop ideas that may be novel, but are unusable to non-specialists, or that address issues that regrettably are irrelevant in practice. The techniques developed to assess usability may, themselves, be the object of consideration in the same terms. Are they usable, or do they present difficulties of interpretation or application? Green (1995) discussed these and other questions under the heading of ‘the HCI of HCI’. Our experience of trying to make our goals explicit in the design and redesign of OSMs has led us to realise more strongly than ever that HCI needs to be reflective. Usability assessment techniques must themselves be usable. Moreover, as Bellotti (1989) points out, the techniques must deliver something useful.
17
AEB & TRGG
Design & re-design of representation
A challenge, then, to developers of HCI methodologies is to state who should be expected to use them and why, just as designers of applications or authors of texts identify the potential users. In the case of OSMs, our intention is to develop an approach that is open to designers and other evaluators who are conversant with the artefact to be evaluated and its domain of use, but not specialists in HCI; and to make that approach cheap enough and useful enough to entice designers to use it from choice, as one of their tools. Good intentions are not enough. A further challenge is to find ways to demonstrate effectiveness. Research into comparative evaluation of evaluation techniques has gathered pace. Jeffries et al. (1991) compared four techniques; Lavery et al. (1997) not only compared techniques, but illustrated the difficulty of scoring, which requires matching or grading problems identified by different techniques. Evidently this will not be easy, and by these criteria our formative evaluation study was only a beginning. Yet, just as it is now increasingly common for reports of novel HCI tools to include some kind of usability evaluation, our experience of evaluation and redesign leads us to suggest that other developers of HCI methodologies and tools intended for nonspecialist use should report usability assessments. Conditions for usable modelling approaches in HCI
In the absence of empirical data, it is still possible to make usability predictions for modelling techniques using existing HCI approaches, although the validity is a matter of faith or else an issue for empirical investigation. One possible approach is to apply ‘cognitive dimensions’ themselves. As an informal, broadbrush approach, it can be applied to any situation in which an information structure is represented, created or manipulated. HCI models are one class of information structure, and their properties can be assessed in the same terms as any other. If we assume that the modeller is creating a model in much the same exploratory, opportunistic way that many other designers work, then a condition for usability would be that the representation should not be too viscous, nor should it contain many hidden dependencies. Exploratory design by definition includes changes of mind and reconstructions. A model like GOMS (Card et al, 1983) or TAG (Payne and Green, 1986) is unsuitable for exploratory design; both are viscous, and in addition TAG contains many hidden dependencies. OSM was deliberately designed to encourage ‘sketch modelling’; we have deliberately avoided exhaustive analysis and its ‘death by detail’ in favour of a quick and easy approach. The main data structure, a simple table of entities, attributes, and relationships, is far easier to manipulate and to read than most notations used in HCI modelling. Another approach to predicting the usability of modelling techniques might be inspired by recent work on display-based modelling, e.g. Kitajima and Polson (1992). Empirical observations, common-sense and cognitive models agree in predicting that working with an interface is easier when the perceived display guides the user at each step. Since a substantial component of our OSM approach is the description of visible entities, it is at least partly display-driven and should therefore be relatively easy to perform. Task-based models, on the other hand, by their very nature are driven by tasks, which have no immediate visual or perceptual representation. The difference is rather like describing a teapot by listing the parts you can see (body, handle, lid, spout, hot water, tea leaves) versus listing what you can do with it (warm pot, add tea, add hot water, ...). Thus we expect that, for non-specialists, OSMs should be much easier than task-based analyses. Comparisons among different types of model
Three distinctions among user models seem to be relevant in placing OSMs in the landscape. First, OSMs are declarative descriptions of structure, rather than procedural descriptions of tasks. Unlike almost all previous HCI methods of usability evaluation, Ontological Sketch Modelling is centred on entities rather than user tasks (hence ‘Ontological’). There has been increasing interest in domain modelling, some of which has focused on structural aspects (e.g. Benyon, 1995) or on the use of explicit ontologies (e.g. Guarino, 1997), but these have not to our knowledge been applied to usability evaluation. This seems to be a very promising approach, although OSM in no way supersedes the procedural class of model, which has its own role and makes a different type of prediction. Second, because OSMs are intended to identify misfits, they model both the user’s conceptual view and the ‘device view’. It would be possible to extend procedural models in the same direction, by comparing the user’s view of tasks with the task structure imposed by the device, and the results of such an analysis would be very interesting.
18
AEB & TRGG
Design & re-design of representation
Thirdly, OSMs take a particular stance on the issue of formality and detail. We regard this issue as one of the determinants of ‘market success’; models that create a wealth of detail are likely to overload non-specialist users, and models or other approaches that are too light on formality and detail are unable to deliver effective predictions. Support for modellers
Level of detail has already been put forward as one determinant of market success. Another determinant is likely to be the accessibility of the model and of worked examples, and the provision of tools to assist modelling. Our formative evaluation brought home to us the need for exemplars, without which it is not necessarily clear what the benefits of a modelling approach might be, nor where to start. Even simple tools can be very effective. Curry and Monk’s (1995) production system approach to dialog modelling was made more comprehensible by Monk’s simple spreadsheet-based tool (Monk, no date) with which dialogs can be executed in simplified form, and because that tool resides on the web it can be accessed very readily. This, we believe, is a clear pointer to future modellers. A web-accessible package containing support tools and examples could make an HCI approach available to a far greater section of the design community. In support of this goal, we aim to make the approach widely accessible, by providing a web-based set of examples to demonstrate what can and cannot be done with OSMs, to scope the OSM approach, and to provide a resource for re-use of components. We intend making available a tool to support analysis, to help analysts to structure their understanding of the domain and device appropriately, and to do the automated reasoning. At present we have implemented a demonstrator prototype that illustrates how reasoning can be done for some usability concerns (notably viscosity), but we have not addressed the issue of usability of the tool itself yet, or of making it available over the web. The domain of application: curve-drawing
Finally, some remarks on drawing curves. We chose that domain to test our approach because it was important, fun, easy to understand, non-trivial – and virgin territory, unmarked by previous usability studies. Anecdotal evidence told us that bézier-curve drawing programs were difficult to get the hang of. This was certainly supported by the evidence from our study, even though ClarisWorks, the application we used, seemed to us to be the most understandable of the three or four that we considered. There is much room, therefore, for further studies of usability in this domain, aiming to make bézier-based applications easier to learn and to use. There is also room for developing alternatives to the bézier model, to see whether they are easier to learn and use. Curve-drawing also raises some challenges for other, task-based, approaches to modelling, since these aproaches at present only include entities with shared access. It must be remembered that task-based approaches such as GOMS were created in the days when word-processing was the application of greatest interest. A task-based analysis of a word-processor produces a reasonably rich set of tasks, because the methods for creating and manipulating text have to take into account quite a large hierarchy of domain entities (character, word, paragraph, section, etc) which have shared access. Learning how to do these tasks was a large part of learning how to use the wordprocessor, and deploying them as methods made up a large part of the skilled user’s activity. GOMS provided a reasonable description of the methods and its descendant CCT (Kieras and Polson, 1985) provided a reasonable description of the learning process, so task-based analysis can be quite successful at describing activities with a rich set of shared entities. In contrast, task-based analysis of curve drawing has a much poorer set of shared entities to work with – possibly no more than curve, line, and point. Other entities, such as fin or tail, are strictly user-private, and therefore cannot be taken into account in any approach that only models shared entities. A task-based analysis would therefore only generate a rather small set of tasks, which may not adequately describe the true richness of the user’s mental life. There is another challenge for task-based models that have a single-threaded control structure like GOMS. The nature of text is that there are many internal dependencies which restrict the order of performing tasks; a GOMS-like model is therefore adequate. Such is not the case with a diagram, where any line or point may be altered independently of all others, making it possible to proceed in a much more opportunistic fashion than in writing text. Users can work on a variety of tasks concurrently, possibly dealing with all the changes that have to be made in one part of the diagram and then turning to another part.
19
AEB & TRGG
Design & re-design of representation
Entity-based analysis, as we have seen, can be applied very successfully to curve-drawing. It would be very instructive to see our analysis complemented by a task-based analysis. We believe that such an enterprise would give further insights into the ease and difficulty of drawing curves, and might also help to cast light on the scope and effectiveness of task-based models as a class.
Acknowledgements This work has been supported by a grant from the Faculty of Technology at Middlesex University. We are grateful to Juliet Congreve and the students on her HCI and Graphics course for their co-operation and participation in the empirical study, and to John Rieman for making J-Sketch available to us. Claris and ClarisWorks are trademarks of Claris Corporation.
References Bellotti, V. (1989) ‘Implications of current design practice for the use of HCI’ --in D. Jones & R. Winder (Eds.) People and Computers IV, Proceedings of HCI’89, 13-34. Cambridge University Press Benyon, D. (1995) A data-centered framework for user-centred design. In Nordby, K., Helmersen, P. H., Gilmore, D. J, and Arnesen, S. A. (Eds.) Human-Computer Interaction: INTERACT-95. London: Chapman and Hall. Blandford, A. E. & Young, R. M. (1996) Specifying user knowledge for the design of interactive systems. Software Engineering Journal. 11.6, 323-333. Blandford, A. (forthcoming) ‘Human Factors in Computing Systems’. To appear in: D. Hemmendinger, A. Ralston & E. Reilly (Eds.) The Encyclopedia of Computer Science, 4th ed., International Thomson Computer Press Buckingham Shum, S. & Hammond, N. (1994) ‘Transferring HCI modelling and design techniques to practitioners: A framework and empirical work’ in G. Cockton, S.W. Draper & G. Weir (Eds.) People and Computers IX, Proceedings of HCI’94, 21-36. Cambridge University Press Card, S.K., Moran, T.P. & Newell, A. (1983). The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates: Hillsdale, NJ. Curry, M. B. and Monk, A .F. (1995) Dialogue modelling of graphical user interfaces with a production system. Behaviour and Information Technology14(1) 41-55 Ellis, J. (1988) Memory for future intentions: Investigating pulses and steps. In M. M. Gruneberg, P. E. Morris & R. N. Sykes (Eds.) Practical aspects of memory, Vol. 1. Current research and issues. 371-376. Chichester: Wiley. Green, T. R. G. (1990) The cognitive dimension of viscosity: a sticky problem for HCI. In D. Diaper, D. Gilmore, G. Cockton and B. Shackel (Eds.) Human-Computer Interaction – INTERACT ’90. Elsevier. Green, T. R. G. (1991) Describing information artifacts with cognitive dimensions and structure maps. In D. Diaper and N. V. Hammond (Eds.) People and Computers VI . Cambridge University Press. Green, T. R. G. (1995) Looking through HCI. Invited paper at 10th AnnualWorkshop of BCS HCI Group. In Kirby, M. A. R., Dix, A. J. and Finlay, J. E. (Eds.) People and Computers X. CUP Green, T. R. G. & Benyon, D. (1996) The skull beneath the skin: entity-relationship models of information artifacts. International Journal of Human-Computer Studies, 44(6) 801-828 Green, T. R. G. & Petre, M. (1996) Usability analysis of visual programming environments: a 'cognitive dimensions' framework. J. Visual Languages and Computing, 7, 131-174. Guarino, N. (1997) Understanding, building and using ontologies. International Journal of Human-Computer Studies 46, 293-310. Hartson, H.R., Siochi, A.C. & Hix, D. (1992). The UAN: A user-oriented representation for direct manipulation interface designs. ACM Transactions on Office Information Systems, 8, 181-203. Jeffries, R., Miller, J. R., Wharton, C. and Uyeda, K. (1991) User interface evaluation in the real world: a comparison of four techniques. In S. P. Robertson, G. M. Olson and J. R. Olson (Eds.), Proceedings of ACM CHI’91 Conference on Human Factors in Computing Systems (ACM Press).
20
AEB & TRGG
Design & re-design of representation
John, B. & Kieras, D. E. (1996). Using GOMS for user interface design and evaluation: which technique? ACM ToCHI.1-30 Kieras, D. & Polson, P. G. An approach to the formal analysis of user complexity. Int. J. Man-Machine Studies, 22 (1985), 356-394. Kitajima, M. and Polson, P. G. (1992) A computational model of skilled of a graphical user interface. In P. Bauersfeld, J. Bennett and G. Lynch (Eds.) Proceedings of ACM CHI’92 Conference on Human Factors in Computing Systems (ACM Press). Lavery, D. (1996) Specialising design principles and cognitive walkthroughs for software visualisations, In A. Blandford and H. Thimbleby (Eds.) HCI 96 Adjunct Proceedings. Lavery, D., Cockton, G. and Atkinson, M. (1997) Comparison of evaluation methods using structured usability problem reports. Behaviour and Information Technology, 16 (4/5), 246-266. Monk, A. F. (no date): The Action Simulator package. Available by ftp via URL http://www.york.ac.uk/~am1/ftpable.html Moran, T. P. (1983) Getting into a system: external-internal task mapping analysis. Proc. CHI 83 ACM Conf. on Human Factors in Computing Systems, pp 45-49. New York: ACM. Nielsen, J. (1994) Heuristic Evaluation. In J. Nielsen & R. Mack (Eds.), Usability Inspection Methods (pp. 2562). New York: John Wiley. Payne, S. J. and Green, T. R. G. (1986) Task-Action Grammars: a model of the mental representation of task languages. Human-Computer Interaction, 2, 93-133. Payne, S. J. (1993) Understanding calendar use. Human-Computer Interaction, 8, 83-100. Rieman, J. (1996) J-Sketch: A drawing tool for informal graphics, In A. Blandford and H. Thimbleby (Eds.) HCI 96 Adjunct Proceedings. Roast, C.R. & Siddiqi, J. (1997) Formally assessing software modifiability. In C.R. Roast & J. Siddiqi (Eds.) Formal Aspects of the Human Computer Interface. London: Springer. Wharton, C., Rieman, J., Lewis, C., & Polson, P. (1994). The cognitive walkthrough method: A practitioner's guide. In J. Nielsen & R. Mack (Eds.), Usability inspection methods (pp. 105-140). New York: John Wiley. Young, R. M., Green, T. R. G., & Simon, T. (1989) Programmable user models for predictive evaluation of interface designs. In Proceedings of CHI ‘89. ACM, New York.
21
