PUMA
PROGRAMMABLE USER MODELLING APPLICATIONS
Demonstrating the cognitive plausibility of interactive system specifications Richard Butterworth Ann Blandford David Duke February 2000 WP25
Principal Investigator Prof Ann Blandford Middlesex University Research Fellow Dr Richard Butterworth Middlesex University Academic Collaborator Dr David Duke University of York Research Fellow Jason Good Middlesex Unversity Industrial Collaborator Sue Milner Praxis Critical Systems Ltd Academic Collaborator Prof Richard Young University of Hertfordshire http://www.cs.mdx.ac.uk/puma/ Contact: Prof Ann Blandford School of Computing Science Middlesex University Bounds Green Road London. N11 2NQ. UK. tel : +44 (0)181 362 6163 fax : +44 (0)181 362 6411 email :
[email protected]
Project funded by EPSRC grant number GR/L00391
Demonstrating the cognitive plausibility of interactive system specifications Richard Butterworth
Ann Blandford
David Duke
Abstract Much of the behaviour of an interactive system is determined by its user population. This paper describes how assumptions about the user can be brought into system models in order to reason about their behaviour. There is a tension between the inspectability of user assumptions and the tractablity of models; inspectable models tend to not be very tractable and vice versa. We describe how we can get round this tension, by deriving tractable models from explicit user assumptions. The resulting models may not of themselves be very inspectable to human-factors workers, but the process by which they are derived is inspectable. Hence we claim that we can have both tractability and inspectability. We exemplify our claims using a simple cognitive model and ‘Meeting Maker’, an interactive electronic diary system.
1
Introduction
Interactive systems rely extensively on their user population to determine their behaviour. A design process for such systems should therefore take users into account. One way of doing this is to make inspectable assumptions about users and include them in a design process with an equivalent level of concern to the other components in the system. Such assumptions would then guide the design of the automated parts of the system, which we refer to as the ‘device’1 . The goal of an interactive system design process is therefore to implement a device which when conjoined with its users will fulfil given requirements. Ideally if the assumptions about the user population could be expressed as a logical expression user and the requirements and device specification req and device respectively then the job of the designer is to implement a device such that. . . req user ∧ device Such a situation is highly idealised, however, and many problems and issues are raised by this approach. This paper explores and describes several of these issues, concluding that the classical refinement calculus [13, 26] is a sufficient reasoning tool to deal with the issues that arise.
1.1
A problem
Butterworth and Blandford [7] describe several trade-offs that need to be considered when integrating models of human cognition into design models of interactive systems. Of primary concern is the level of abstraction that is necessary to rigorously reason about a model. Existing cognitive models (e.g. ACT [1] or Soar [28]) are detailed, operational 1
Note our terminology; we use ‘system’ to refer to a system in its entirety, including its users, whereas we use ‘device’ to refer to the automated part of an interactive system.
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
2
affairs, originally constructed to simulate human behaviour, whereas reasoning about abstract interactive system models requires assumptions about user behaviour to be expressed abstractly. We therefore have a tension: between the operational nature of cognitive models, originally intended to be used for simulation, and the use to which we wish to put them, namely reasoning rigorously about interactive behaviour. However the benefits of abstracting across a cognitive model must be balanced against the drawback that an abstracted cognitive model has reduced validity. (An example of this tension can be seen by comparing the interactive system model presented by Blandford, Butterworth and Good [3], where cognitive assumptions are explicitly included and the model is analysed by simulation, with the interactive system model presented by Butterworth, Blandford and Duke [8] where the user assumptions are very implicit in the model but the model is abstract enough to be reasoned about rigorously.) Furthermore its is probable that a system model that is abstract enough to reason about will have very implicit user assumptions and device components. i.e. the system model will not be simply of the form user ∧ device. Not until later in the refinement cycle will the different components of the system be explicitly separated off and ‘implemented’2 . The ability to inspect and validate the assumptions made about the user is crucial. Despite moves in the right direction by applied psychologists and theoretically based HCI workers there as yet exists a scant body of psychological theory that can be directly applied in the design of interactive systems. The communication between psychologists and interactive system designers therefore needs to be two-way: psychology needs to be able inform system design about user behaviour and system design needs to be able to question psychology about user behaviour. The inspectability of user assumptions is essential to this communication. We therefore seem to have a sticky problem: models of interactive systems need to be reasoned about and inspected for cognitive validity, and it would appear that these two requirements are antithetical to one another. Squaring this circle is the major issue that this paper will address.
1.2
A solution?
Given an interactive system model sys there are (at least) two questions we may wish to ask of it: firstly; is it functionally correct? and secondly; are the assumptions it makes about its users valid? The first question is a case of establishing whether sys is correct with respect to a statement of functional requirements req and the refinement calculus gives us the tools to do this. We need to show that req sys. If we could split sys such that sys = user ∧device then we could have a simple answer to the second question too, but we argued above that for sys to be abstract enough to reason with this is unlikely to be the case; in a simple system model the user assumptions may have become very implicit and therefore it is not easy to split off user in a neat fashion. Therefore we propose the following solution; even though sys itself cannot be inspected for cognitive validity, the process by which sys is arrived at can be inspected. 2 Of course the users cannot be implemented; they already exist and their properties need to be taken into account by the components that can be implemented.
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
3
req
Functionally correct?
baseModel
✲
Cognitively plausible?
❄
sys
Figure 1: Functional correctness and cognitive plausibility In a process entirely analogous to stepwise refinement we claim that we can express a general model of human cognition that effectively acts like a schema, and then instantiate this schema with details relevant to the specific interactive system being modelled. The starting point to this process — the ‘base model’ — is explicit and inspectable as are all the steps which instantiate it to a specific system model; therefore claims can be made about the cognitive validity of the resulting model. This process uses no other mathematical tools than the refinement calculus. We show this argument graphically in figure 1. The process of answering the two questions ‘is sys functionally correct?’ and ‘is sys cognitively plausible?’ is shown horizontally for cognitive plausibility and vertically for functional correctness. Each process is a case of deriving the model sys by mathematical transformations, in the case of functional correctness from req and in the case of cognitive plausibility from the base model baseModel . We illustrate this solution by first describing an example interactive system and modelling two different strategies by which the user may interact with the device. One of these strategies is functionally correct and one is not, a fact that we demonstrate using refinement. In subsequent sections we then trace the cognitive validity of these models, showing that they are indeed both cognitively valid. We furthermore present an argument that a functionally correct version of the latter strategy is cognitively invalid, and therefore that the latter strategy is rather problematic. The issues raised by our process are discussed in the conclusion.
1.3
The background to our work
There is now a considerable body of work advocating the use of formal models for reasoning about interactive systems (e.g. [19, 14, 30, 20, 24]) which look at interactive systems for several differing perspectives. Typically those approaches attempt to ‘model an interactive system from the user’s point of view’. This modelling ranges from very abstract models in the style of Dix’s PIE model [14] through to tools for modelling and reasoning about interfaces and their widgets [15]. A common feature of most of these approaches is that they model and reason about interactions rather than interfaces. It is a common misconception
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
4
that HCI stands for ‘human-computer interface’, whereas most HCI workers will tell you that HCI is about interaction, and for a system to be genuinely usable requires more of the developer than simply producing an attractive front-end. Modelling an interactive system ‘from a user’s point of view’ typically does not involve making explicit assumptions about the users. Much work that aims to describe HCI issues formally [14, 29, 5] sets about defining mathematically expressed properties of interactive systems that are claimed to have relevance to why (or why not) a user finds a system usable. However these properties typically make many implicit assumptions about the user that are difficult to validate by cognitive scientists. The work reported here is in the Programmable User Modelling (PUM) tradition [33] where explicit assumptions about the user are built into system models to evaluate their predicted usability. Since its inception, theoretical PUM work has grown steadily more abstract, concentrating less on the mini-planning user model that was used to simulate user behaviour and moving more into the realms of abstract mathematical modelling, where behaviour is reasoned about rather than explicitly generated. Although our approach, which includes explicit user assumptions drawn from cognitive science, is unusual, it is not unique. We look at two other approaches which use cognitive science to reason formally about interactive systems in section 3.
2
Meeting Maker
We illustrate our approach with an interactive system called ‘Meeting Maker’. Meeting Maker is an electronic diary system such that several users can maintain their own personal diaries on-line. Meeting maker can automatically suggest suitable meeting times for a collection of users based on their diary entries. Meeting Maker presents an interesting case study; it is a feature rich package of interactive components which at first looks daunting to model. However we show that abstracting across those features can produce analyses which are tractable and useful when considering redesigns to the system. A more thorough and detailed introduction to the example we use here is given by Butterworth and Blandford [7]. Note that although we claim that our approach is valuable in the predictive analysis of interactive system, here we analyse an existing system in a post hoc fashion. This is because we wish to show our arguments using an example that is (to us) well thought out and ‘stable’, rather than attempting to make a clear argument with the sort of more fluid ‘hazy’ examples typical of ongoing design processes. How a derivative of the approach described here was used predictively in the design of a real, safety critical system is described by Good and Blandford [17]. We proceed by informally describing some of the functionality offered by Meeting Maker, a scenario of use and two strategies that the user undertakes within that scenario. We then formalise each of these descriptions. Each formalisation is an abstraction over the informal description and we justify that abstraction.
WP25
2.1
Cognitive plausibility
5
Butterworth, Blandford & Duke
The functionality offered by Meeting Maker
The scenario we consider involves how users enter data into their personal Meeting Maker diary, so we only describe the functionality involved with this process. For each user Meeting Maker stores a representation of their diary. A diary is a collection of ‘events’ that are scheduled to occur. An event contains information about its whereabouts, start time, duration and so on. Meeting Maker’s state space is an electronic diary denoted ed which is a set of events, an event being an undefined type. Event = ˆ ... ed : P(Event)
(1) (2)
This simple representation is sufficient for our purposes. More complicated representations which place the diary on a time line (i.e. as a mapping from time to event) or model the diary as a sequence of events are not necessary. In fact the set representation captures what goes on in the Meeting Maker implementation surprisingly accurately: very busy users are allowed to schedule events that overlap or even several events to happen at the same time! The user is offered tools to edit the events stored in their diary, most of which are those that one would expect: the user can add an event or delete an existing event. Each of these tools is represented by a specification fragment in Morgan’s [26] notation. Firstly addEvent(e) adds an event e to ed . addEvent(e) = ˆ ed : [ed = ed0 ∪ {e}]
(3)
deleteEvent(e) removes event e from ed . Its precondition asserts that e is in ed .
deleteEvent(e) = ˆ ed :
e ∈ ed , ed = ed0 \ {e}
(4)
Furthermore Meeting Maker also offers a facility to add a series of events that have similar characteristics. For example, a user may wish to add to their diary a series of lectures that they are giving every Tuesday at 11am in the main lecture theatre for the next three months. Instead of entering those events one by one into the diary the user can enter the characteristics of the series of events (its starting date and end date, its frequency — in this case weekly — where each of the events occur and at what time) and Meeting Maker will enter the series of events that are described by that characterisation. addSeries(es) adds a set of events es to ed . addSeries(es) = ˆ ed : [ed = ed0 ∪ es]
(5)
addSeries is quite an abstraction over the tool we have described above: it simply describes the addition of an arbitrary set of events to ed . In reality this set is not arbitrary; it is defined by the user, but how the user does this is complicated3 . We will be investigating how the user corrects what addSeries has placed in ed . If we can argue that a user will 3
See Butterworth and Blandford [7] to see just how complicated it is.
WP25
Cognitive plausibility
Day Weds Weds Mon Weds Weds Weds Weds Thurs Weds Weds Weds
Date 20th Jan 27th Jan 8th Feb 10th Feb 17th Feb 24th Feb 3rd Mar 4th Mar 10th Mar 24th Mar 31st Mar
Butterworth, Blandford & Duke
Time 3pm 3pm 1pm 3pm 3pm 3pm 3pm 1pm 2pm 3pm 3pm
Duration 1 hour 1 hour 2 hours 1 hour 1 hour 1 hour 1 hour 2 hours 1 hour 1 hour 1 hour
6
Title Room Seminar 2E27 Seminar 2E27 Management meeting 2C15 Seminar 2E27 Seminar 2E27 Seminar 2E27 Seminar 2E27 Management meeting 2C15 Seminar 2E27 Seminar 2E27 Seminar 2E27
Figure 2: A paper diary to be entered into Meeting Maker successfully correct an arbitrary set of events, then the user will be able to correct a less than arbitrary set. This technique is widespread in these sorts of abstract analyses; the analyst considers the worst possible situation that the device may inflict on the user and then analyses the implications of that. If the user can deal successfully with such worst cases then they will be able to deal with whatever the device can give them.
2.2
A scenario of use
We analyse the following scenario: in a certain establishment a secretary circulates a paper copy of upcoming regular departmental events such as seminars and management meetings. An example of such a paper diary is shown in figure 2. It is up to each individual user to enter the events on this paper diary into their Meeting Maker diaries. An important feature to note about the paper diary is that though the events on it are regular they are not precisely regular. There are two management meetings that occur on different days and there are no seminars on 17th March or 3rd February. addSeries is useful in adding this paper diary to the electronic diary, but it is not sufficient to perform the task correctly; the user will subsequently have to delete events that addSeries adds and add events it does not. Formally the paper diary (denoted pd ) is modelled equivalently to the electronic diary ed ; it is simply a set of events. pd : P(Event)
(6)
The scenario requires that the user add the contents of the paper diary to the electronic diary and we can model this scenario as a specification fragment. We denote it req as it is what is required of the system (recall that we consider the user to be part of that system). req = ˆ ed : [ed = ed0 ∪ pd ]
(7)
req plays at least two roles in our analysis. It is a statement of what is wanted of the system without considering how that is achieved; it is therefore a statement of system
WP25
Cognitive plausibility
oneByOne = ˆ
Butterworth, Blandford & Duke
toAdd : [toAdd = pd ] ; do toAdd = ∅ → var e • e : [e ∈ toAdd ] ; addEvent(e); toAdd : [toAdd = toAdd0 \ {e}] od
7
(8)
Figure 3: The one by one strategy requirements in the classic sense. More subtly, however, it may be interpreted as a statement of what the system designer wants the user to want to achieve. This is a labyrinthine comment but important; an interactive system as we model it is reliant on the user for motivation. The user is motivated to achieve a certain goal and will interact with a device to achieve that goal; the device itself has no motivation, so if you take away the user an interactive system will exhibit null behaviour. Hence if the designer wants req to be satisfied then it is a good idea to ensure that the user wants req to be satisfied too. However, as discussed later, this may not be the case.
2.3
Two strategies
We define and analyse two ‘strategies’ (where a strategy is like a computer program, except that some or all of the responsibility for it is placed with the user) for entering an arbitrary collection of events into the Meeting Maker electronic diary. 2.3.1
The one by one strategy
The first strategy we call the ‘one-by-one’ strategy. It describes the user selecting individual events from the paper diary and entering them one by one into the electronic diary. The user will enter an individual event from the paper diary into Meeting Maker and then mark that event off the paper diary (possibly mentally by remembering which events have been entered, or physically by crossing each event of the paper diary) and then repeat this process until no events remain to be entered. A formal model of the one by one strategy is given in figure 3. A new variable toAdd is introduced which represents the events left on the paper diary which have yet to be added to the electronic diary. Whether toAdd represents some mental store of what still needs to be added or is the events on the paper diary not yet crossed off is not an issue. oneByOne describes the user setting toAdd equal to the paper diary and then looping through a process of selecting an event from toAdd , adding it to pd and removing it from toAdd while there are still events to add. 2.3.2
The en-masse strategy
The second strategy we call the ‘en-masse’ strategy and describes the user entering a series of events and then subsequently correcting that series in order that it precisely matches
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
8
ˆ enMasse1 = addSeries(es); toAdd : [toAdd = {e | e ∈ es ∧ e ∈ pd }] ; do toAdd = ∅ → var e • e : [e ∈ toAdd ] ; addEvent(e); toAdd : [toAdd = toAdd0 \ {e}] od
(9)
Figure 4: The en-masse strategy the events on the paper diary. So the user will use the tool that enters a series of events in such a way that as close a match as possible to the series on the paper diary is produced. For example a user attempting to enter the events shown in figure 2 would most likely characterise the series as starting on 20th January and ending on 31st March, being weekly seminars of one hour duration and taking place in room 2E27. The user will then have to repeatedly add and delete events to this series so that it precisely matches the paper diary. A formal model of the en-masse strategy (denoted enMasse1 ; we will suggest a variation later denoted enMasse2 ) describes the user adding an arbitrary set of events es to the electronic diary using setSeries and then working out which events still need to be added to ed and doing so. A formalisation of this is shown in figure 4. toAdd is the set of all events e that are on the paper diary, but have not been added by addSeries. The subsequent loop then adds those events to the electronic diary as necessary, until toAdd is empty.
2.4
Reasoning about the strategies
The purpose of these two strategies being expressed mathematically is so that we can reason with them. The question we ask is; are they functionally correct? Will a user following these strategies succeed in achieving the goal that the designer wishes them to? We demonstrate these claims by proving the following two assertions. . . req oneByOne
(10)
req enMasse1
(11)
. . . i.e. the one by one strategy is functionally correct with respect to req, but the en masse strategy is not. Detailed proofs are shown in the appendices (sections A.1 and A.2 respectively) but the proofs are sketched here. For the one by one proof we simply have to produce a loop invariant that asserts that ed expands only by elements drawn from toAdd (and therefore originally from pd ) and a loop variant that asserts that the size of toAdd decreases towards the empty set. We then need to show that the loop body is implied by this invariant and variant. We show this to be the case, hence the assertion is proved.
WP25
Cognitive plausibility
9
Butterworth, Blandford & Duke
ˆ enMasse2 = addSeries(es); toAdd : [toAdd = {e | e ∈ es ∧ e ∈ pd }] ; toDelete : [toDelete = {e | e ∈ es ∧ e ∈ pd }] ; do toAdd = ∅ → var e • e : [e ∈ toAdd ] ; addEvent(e); toAdd : [toAdd = toAdd0 \ {e}] [] toDelete = ∅ → var e • e : [e ∈ toDelete] ; deleteEvent(e); toDelete : [toDelete = toDelete0 \ {e}] od
(12)
Figure 5: The improved en-masse strategy We disprove req enMasse1 by counter example, showing that the size of ed after the strategy need not be equal to the size of its initial value added to the size of pd (|ed | = |ed0 | + |pd |). addSeries adds an arbitrary set of events to ed , therefore it is possible that before the loop is entered into the size of ed may be greater than the size of |ed0 |+|pd |. So to rectify this we would need to argue that the loop can reduce the size of ed , which it cannot. All it can do is addEvent which increases the size of ed . Hence because the size of ed may finish up greater than |ed0 | + |pd | we must conclude that ed need not finish up equal to ed0 ∪ pd . A proof (particularly a negative proof) gives insight into what is necessary to redesign a system such that it becomes functionally correct [8]. In the case of the en masse strategy the proof makes the problem clear; the user does not delete events that have been incorrectly added by addSeries. Hence we can suggest a new strategy enMasse2 such that the user decides what events need deleting and does so. This new strategy is shown in figure 5. Now we can prove that enMasse2 is correct with respect to req; a proof of which is shown in appendix A.3. So far this analysis has not said anything that should be surprising to a formal methods audience; apart from our use of standard programming constructs to represent user as well as device actions, we have simply proposed requirements and specifications and demonstrated (the lack of) functional correctness between them. However the main point of this paper is to show how we come to the following argument; although enMasse2 is functionally correct, enMasse2 is not cognitively plausible. Further-
Functional correctness Cognitive plausibility
oneByOne Yes Yes
enMasse1 No Yes
enMasse2 Yes No
Figure 6: A comparison of the one by one and en-masse strategies
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
10
PUM Cognitive architecture
→
Behaviour
→
Behavioural properties
GOMS Figure 7: The position of GOMS based and PUM based approaches in the analysis of interactive systems more, although enMasse1 is functionally incorrect it is cognitively plausible. oneByOne is both functionally correct and cognitively plausible. All this is summarised in table 6. By ‘cognitively plausible’ we mean that there is an argument that the model in question can be derived from an inspectable ‘base’ cognitive model, by a sequence of inspectable transformations that preserve the cognitive validity of the base model. In the next section we discuss cognitive models in general and in particular a base model of rational goal driven behaviour. Then in section 4 we show that oneByOne and enMasse1 can be derived mathematically from this base model, but that enMasse2 cannot.
3
The base model of user behaviour
We now describe the cognitive science that we will use in section 4 to justify the claims of cognitive plausibility we made in the previous section. Before doing so we discuss other similar work in this field and the relation of our approach to it, then we discuss cognitive architectures in general. We then propose a model of rational user behaviour and present a formalisation of that model.
3.1
Other cognitive theories and approaches used in analysing interactive systems
Our approach is not novel in that it binds findings from cognitive science into the design of interactive systems. However our approach is innovative in the way it uses the cognitive science; our approach aims to allow an analyst to argue with justification that a strategy4 is cognitively plausible. Other approaches, notably GOMS [11, 22, 21] invite the analyst to propose a strategy and then analyse that strategy to provide measures of predicted performance time and cognitive load. Those predictions are based on a cognitive model (the ‘Model Human Processor’) and empirical observations. However the PUM approach stems from artificial engines that do not analyse behaviour; they generate it. Hence our derivative approach reflects this distinction; whereas GOMS analyses the properties of behaviour, PUM based approaches form arguments about where that behaviour comes from, specifically what 4
Note that we use ‘strategy’ and ‘task’ interchangeably.
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
11
knowledge generates it. (Of course a PUM based approach can also analyse properties of behaviour.) A schematic diagram of this argument is shown in figure 7. Syndetics [16] is another approach based on the integration of a cognitive model called ‘Interacting cognitive subsystems’ (ICS) [2] into interactive system models. Like our approach, syndetics treats a system as a composition of user and device and treats the two with equivalent levels of concern. However the nature of ICS is radically different to the nature of the cognitive model we use, hence the type of analysis performed is quite different. Whereas our approach looks at system behaviour, syndetics is concerned with the configuration of information flows within the system and whether those flows are stable or problematic. Syndetics therefore analyses interactive systems at ‘snapshots’ in time whereas our approach considers behaviour in its more traditional sense.
3.2
Cognitive architectures, knowledge and rationality
Before describing our general model of user behaviour it is useful to describe the ‘cognitive architecture’ on which it is based, and what a cognitive architecture is. A cognitive architecture is considered to be a determinant of behaviour that is general to all users. Essentially we can characterise a cognitive architecture as a function that takes knowledge to user behaviour. The user will have and receive different knowledge during an interaction and that knowledge will determine their behaviour. The architecture dictates what behaviour given knowledge will generate. Architectures in the PUM tradition describe how the user deals with knowledge by assuming that the user is ‘rational’ and characterising rational behaviours. ‘Rationality’ is based on Newell’s definition [27]; namely that an agent (the user) will select operations that take the system state nearer to the user’s goal state. Butterworth and Blandford [6] discuss how Newell’s ideas about rationality can be used to analyse interactive systems. We capture rationality in the model in this paper as a function over the system states which measures ‘distance from goal’. A rational user will interact to decrease this distance. We have published more detailed, but related, models in other papers; see [9, 3].
3.3
The base model
The base model is a schema. It sets out a generalised collection of issues which capture important aspects of interactive systems, and in this case particularly concentrates on rational, goal driven user behaviour. There are several entities we wish to bring into this model; firstly there is the state of the device, secondly the user’s beliefs about the state of the device, thirdly a goal state which captures what the user wants to be true, fourthly a collection of transitions over the device state known as ‘actions’ and lastly a collection of transitions over the user’s belief state known as operations. We look at each of these in more detail.
WP25
3.3.1
Cognitive plausibility
Butterworth, Blandford & Duke
12
The state spaces and transitions
The device state is simply modelled in the traditional manner as a mapping from variable name to value. We assert that there is a collection of device states D and that the current value of the device state is denoted dev . D = ˆ ... dev : D
(13) (14)
The belief state is more subtle. Much effort (e.g. [25]) has been addressed to proposing epistemic logics which capture the features of knowledge and belief. The user may ‘know’ something, or may not know something, but do they know they know, or, even more tricky, do they know that they do not know? And so on. Unsurprisingly the mathematical basis of these logics is complex and they are not simple to reason with. The representational form of the user’s beliefs is left open so that the analyst can select the most appropriate form depending on the required analysis. We assert that there is a collection of belief states B and that the current value of the belief state is denoted bel . B = ˆ ... bel : B
(15) (16)
Note that there is no requirement that the belief state be in any sense a ‘correct’ representation of the device state. Indeed, an interesting line of analysis comes from mismatches or incompleteness between the two states (e.g. [4]). The goal is a statement of what the user wants to be true. It is expressed as a description of the cognitive state and the goal is said to be satisfied once the belief state in some way implies the goal. Within this model of behaviour the user interacts until the goal is satisfied, i.e. when the user believes the goal to have been satisfied. We assert that the goal is a belief. goal : B
(17)
Actions are transitions in the device state. They are functions from previous device state to next device state. There is a finite collection of them denoted α1 . . αn . α1 : D → D .. .
(18)
αn : D → D Operations are transitions in the belief state. There are a finite collection of them denoted o1 . . on . o1 : B → B .. (19) . on : B → B
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
13
Actions and operations are paired together in the system and are considered to occur simultaneously. Once the user invokes a device action their belief state will be updated too. In several cases the user ‘tracks’ the state of the device, i.e. the user updates their beliefs about the state of the device without actually verifying the belief by observing the visible state of the device. For example a user who copies a selected piece of text from a text editor will track the belief that the selected text has been copied to the paste board; the device does not make the fact visible. Note that we may pair the identity action (which does not change the device state) with operations. For example we may have an operation representing the user looking at the device that updates the belief state according to what beliefs the user can learn from the display. Such an operation does not require any change in the device’s state. 3.3.2
Rationality
A simple model of an interactive system asserts an initial condition for the device and belief states and then asserts that subsequent behaviour is described by a sequence of action/operation pairs. Such a model effectively captures the user behaving randomly; invoking actions without rhyme or reason. Such models can be useful as shown by the work of Dix [14] which treats the user as such a random variable and reasons about properties that the device offers to the user. As it stands, without further assumptions about the user, our model describes all possible system behaviour, even very strange behaviour. Our base model asserts stronger assumptions about the user than simply random behaviour, i.e. we assert that the user is ‘rational’ and build in assumptions about rational behaviour derived from the discussion we gave above. To capture rationality we assert that there is a function R that takes the current belief state to a natural number, which is a measure of the ‘distance’ from the goal. R : B →N
(20)
R(bel ) = 0 occurs when the goal state is achieved and if R(oi (bel )) < R(bel ) then we assert that the system would move closer to the goal state after the operation oi occurs. We then assert that it is rational to invoke an action/operation pair when the effect of the operation in that pair is to reduce R(bel ). We further assert that the user is goal driven; i.e. they continue to interact until the goals are satisfied, in other words an action/operation pair will only be invoked when R(bel ) > 0. Although we assert that the user will continue to interact until the goal is achieved there will also be degenerate examples where the user cannot achieve their goals and the user will in this case cease interacting when they cannot progress any further. There is an initial condition init which establishes what is true about the system before the interaction starts. We have now sufficient apparatus to propose our base model. It is shown in figure 8. The base model is a program fragment such that init is established and then a loop is entered into. There are n guarded commands in the loop, one for each action/operation pair. The commands simply assert that the belief and device states are updated by the action and operation. The interest is in the guard. Each guard asserts that an action/operation pair αi , oi can be invoked if. . .
WP25
Cognitive plausibility
14
Butterworth, Blandford & Duke
baseModel = ˆ init; do
R(bel ) > 0 ∧ 0 ≤ R(o1 (bel )) < R(bel ) ∧ bel ∈ dom o1 ∧ dev ∈ dom α1
→ bel , dev :
bel = o1 (bel0 ) ∧ dev = α1 (dev0 )
[] .. . []
(21)
R(bel ) > 0 ∧ 0 ≤ R(on (bel )) < R(bel ) ∧ bel ∈ dom on ∧ dev ∈ dom αn
→ bel , dev :
bel = on (bel0 ) ∧ dev = αn (dev0 )
do Figure 8: The base model • the goal is not already achieved (R(bel ) > 0), • doing so would move the system closer to the goal (0 ≤ R(on (bel )) < R(bel )), and • the current belief state can be transformed by the operation and the current device state can be transformed by the action (bel ∈ dom on ∧ dev ∈ dom αn ), i.e. their preconditions are satisfied. Note that the goal state goal is not explicitly mentioned in the base model; it is implicit in the rationality function R. There are several issues raised by this model, which we now outline and discuss.
3.4
Discussion of the base model
Despite our claim that this is a ‘general’ model it has limitations. Firstly the assumption that the device is passive means we cannot use it as the basis for system models where the device has autonomous behaviour or responds to influences other than those generated by the user. This model is inappropriate for control systems therefore, where the user employs the device as an intermediary to control some part of the fluctuating environment. Furthermore our definition of goal driven behaviour is based around goals being eventually achieved, and then behaviour ceases. It does not capture goal driven behaviour that has a more control oriented feel to it; it is inappropriate for interactions where the user is indefinitely endeavouring to keep the system in a goal state. Butterworth and Blandford [6] describe models which can capture this sort of behaviour, but for the examples used here they are not necessary. Because R is a function over the belief state this means that there will be some behaviours that are apparently irrational which can be captured as rational. Consider the
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
15
well known example of ‘monkeys and bananas’. A monkey has a goal of obtaining a bunch of bananas from a tree and in order to satisfy this goal the monkey must position a box under the tree and stand on it to get the bananas. The monkey has therefore two subgoals — to be under the tree and to be on the box. Obviously there are ordering issues in this example; the monkey must put the box under tree before it can climb onto it. Now assume the monkey is already under the tree and must therefore move away from the tree to get the box. Moving away from the tree may be viewed as externally irrational behaviour, because doing so moves the monkey away from the subgoal of being under the tree. However if the monkey has knowledge that moving away from the tree places it in a better position to achieve the overall goal then this can be considered to be rational behaviour. Because R is over the cognitive state (i.e. the user’s knowledge) then we can capture this sort of internally rational, if apparently externally irrational, behaviour. In the following section we demonstrate how we can instantiate this base model into the specific models of interactive systems from section 2.
4
Instantiating the base model
The process of deriving the system models is a case of laying down and formalising the assumptions about the user and device. We then instantiate the base model with these assumptions, all the time simplifying the resulting models so as to keep reasoning with them as tractable as possible. We first instantiate the state space, then the goal, then the action/operation pairs. We then show how each of the system models in section 2 can be derived.
4.1
The device and belief states
Recall that we divided the state space of the system into device state and belief state. The device state is the electronic diary; i.e. a set of events, so we instantiate the device state type D to be simply a set of events. D = ˆ P(Event)
(22)
The variable dev that denotes the device state now assumes that type, so we can assert that. . . dev : P(Event)
(23)
The user’s belief state contains the following information: the user’s beliefs about what events are in the electronic diary, the user’s beliefs about what is on the paper diary, and the user’s beliefs about what events remain to be added or deleted. Each of these beliefs is represented by a set of events. The belief state type B is instantiated as a 4-tuple as follows. . . B = ˆ P(Event) × P(Event) × P(Event) × P(Event)
(24)
. . . and the variable bel assumes that type. . . bel : P(Event) × P(Event) × P(Event) × P(Event)
(25)
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
16
The first set in the tuple represents the beliefs about the events in the electronic diary, the second the events on the paper diary, the third the events to be added, and the fourth the events to be deleted.
4.2
Explicit and implicit goals
Next we instantiate the goals and the related rationality functions, but before we do we need to look in more depth at what constitutes a user’s goal. Recall from section 2 that we stated that the requirements req assert that the Meeting Maker system should work to add the contents of the paper diary to the electronic diary. Recall also that we claimed that req was a statement of what the designer wants the user to do, but it may not be a statement of what the user would want to do. We now look at this claim in more detail. We ascribe to the user the goal to get all the events from the paper diary into the electronic diary. However this statement of the goal is ambiguous; what we really want to say is that the user gets all the events from the paper diary into the electronic diary and leaves the rest of the electronic diary unaltered. However let us return to the user; the designer may want the user to behave in this very logical fashion, but can we be sure that the user will behave this way? Unfortunately there is evidence that the user will not. In fact it can be argued that the user will actually interact to fulfil the ambiguously stated goal that we initially described. i.e. the user’s goal will in fact be pd ⊆ ed . We assert that the user typically approaches any interaction with two goals: an explicit goal which asserts that the user wants to change something about the world, and an implicit goal which asserts that the user wants to change nothing else about the world than that required by the explicit goal. The user will interact to achieve the explicit goal but may fail to interact to achieve the implicit goal. Our evidence for this is comes from the reporting of several systems [10] where problems are caused because as part of the interaction the system state is perturbed and needs to be reinstated subsequently. Consider an old fashioned automatic teller machine; the user inserts their card, types their personal identification number and the amount of money required, then collects their cash and then removes their card. This system is problematic; the user approaches the machine with the explicit goal of withdrawing cash from their account which they will quite happily fulfil. However the user also has a more implicit goal to not change anything else about the world, which they may well fail to achieve. In the case of the ATM system the user will interact to withdraw money, but once that goal is achieved they may walk away from the machine without retrieving their card. i.e. the implicit goal of not altering the state of the rest of the world, in particular the goal of keeping one’s ATM card in one’s pocket, is not fulfilled. Therefore we claim that the Meeting Maker user has an explicit goal to enter all the events on the paper diary in the electronic diary which we define as follows. . . user = ˆ ed : [pd ⊆ ed ]
(26)
The user will interact to achieve this goal, but may not interact to achieve the goal req (which captures both the explicit and implicit goal) which the designer may wish the user to.
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
17
We now propose a rationality function for the goal user . Recall from the previous section that the R function is a measure of how close the system state is believed to be to the goal. A value of 0 for R indicates that the goal state is reached. Ruser is the R function relating to the goal user and is defined as follows. . . Ruser ((ed , pd , toAdd , toDelete)) = |pd | − |pd ∩ ed |
(27)
i.e. the addition of events from pd to ed reduces Ruser (bel ), but changing ed by adding or removing events not in pd has no effect on the value of Ruser (bel ). Once the explicit goal is achieved, i.e. pd ⊆ ed then Ruser (bel ) = 0.
4.3
The action/operation pairs
Finally in the process of instantiating the base model we need to define the action/operation pairs for the Meeting Maker example. The are two sets of action/operation pairs we need to define: those for adding and deleting. 4.3.1
Adding events
Consider an arbitrary event ei ; the effect of adding it to the electronic diary is that it is in the electronic diary, that the user believes that it has been added and that it is removed from the set of events that need adding. The latter two of these effects comprise an operation (a transition to the belief state) whereas the former is an action (a transition to the device state). Recall that the base model expresses the actions and operations as functions over the state. So we propose two functions: addAct ei and addOp ei , the action and operation which describe the additions of event ei to the electronic diary as follows. . . (28) addAct ei (ed ) = (ed ∪ {ei }) addOp ei ((ed , pd , toAdd , toDelete)) = (ed ∪ {ei }, pd , toAdd \ {ei }, toDelete) (29) We also limit the domain of addOp ei so that it is only defined for belief states such that ei is in the set of events that need adding. (ed , pd , toAdd , toDelete) ∈ dom addOp ei ⇔ ei ∈ toAdd
(30)
This effectively declares a precondition for the operation, stating that it cannot take place unless ei is in the set of events to be added. 4.3.2
Deleting events
We now follow a similar process for deleting an arbitrary event ei . The action and operation are as follows. . . delAct ei (ed ) = (ed \ {ei }) delOp ei ((ed , pd , toAdd , toDelete)) = (ed \ {ei }, pd , toAdd , toDelete \ {ei })
(31) (32)
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
18
Again we limit the domains of these two functions so that events can only be deleted when they are in the electronic diary and the set of events to be deleted. This time we need to limit the domain of both the action and operation ed ∈ dom delAct ei ⇔ ei ∈ ed
(33)
(ed , pd , toAdd , toDelete) ∈ dom delOp ei ⇔ ei ∈ ed ∧ ei ∈ toDelete
(34)
This completes the definitions of the action/operations pairs necessary for the Meeting Maker example.
4.4
Deriving the one by one strategy
In the previous section we filled in the ‘empty spaces’ in the base model, by laying down the state space, goal definitions and action/operation pairs. However reasoning with this instantiated model is not simple, so we show how we can simplify it by a series of equivalences to the models we presented in section 2. The first simplification we can make is the observation that, assuming that the system starts in a state where the user’s beliefs about the device are correct, the user’s beliefs are always correct. Each action/operation pair that effects the device state effects the user’s beliefs about the device state in a completely equivalent manner. If the user adds event e1 then the user correctly believes that they have done so, and the same goes for deleting events. Therefore we do not need to make a distinction between the device state and the user’s beliefs about the device state; they both always have the same value and can therefore be represented by the same variable. This is of course a very extreme assumption to make, but if we can expose problems with the system making such powerful assumptions about the user then a real user, who will not have perfect knowledge of the device state, will still make the errors we expose, as well as others. Furthermore instead of parcelling up the system variables in dev and bel we simply use the individual variables. Therefore we simplify the state space to four variables: ed , pd , toAdd and toDelete, each of which represents the appropriate set of events, and where ed represents both the events that actually are in the electronic diary and the events that the user believes to be there. For the one by one strategy we instantiate the system initialisation to be the user transferring all the events on the paper diary pd to the set of events that need to be added toAdd . i.e. init = toAdd : [toAdd = pd ]. To instantiate the action/operation pairs for the one by one strategy we only consider adding events. So we assume that for every event e in E we instantiate a guarded command in the loop which describes the addition of e to the electronic diary. Now we look at these guarded commands to simplify them. First of all we concentrate on the guards. Instantiating the guard for adding arbitrary event ei using the rationality function Ruser
WP25
Cognitive plausibility
19
Butterworth, Blandford & Duke
results in the following expression. . . (|pd | − |pd ∩ ed |) > 0 ∧ 0 ≤ (|pd | − |pd ∩ (ed ∪ {ei }|) < (|pd | − |pd ∩ ed |) ∧ ei ∈ toAdd
(35)
The first two lines simply rewrite the guards from the base model in figure 8 with the definition of Ruser and incorporate the fact that adding event ei results in a diary with the value ed ∪ {ei }. The third line is a restatement of the precondition to the operation addOp ei , i.e. that ei must be in toAdd . There is no precondition to addAct ei therefore it is omitted. Now we see if we can simplify over this expression. We can make the assumption that toAdd ⊆ pd , based on the initialisation init that sets toAdd to be equal to pd and the fact that all subsequent operations only remove events from toAdd and do not effect pd , hence toAdd must always be a subset of pd . Any event ei is either in pd or not. If event ei is not in pd then adding it to ed does not affect the size of pd ∩ ed , so the second line in the expression reduces to false and therefore the whole guard reduces to false and it and the associated action/operation pair can be omitted from the model as they never happen. So all that leaves us with are the events that are in pd . Again we divide this set of events into those that are in toAdd and those that are not. If ei is not in toAdd then the guard fails because of the third line of the expression. If however ei is in toAdd then the second line reduces to true — the intersection of pd and ed must increase by one, therefore the value of |pd | − |pd ∩ ed | must reduce by one. The third line is true trivially. The first line must also be true because if there are still events in toAdd then not all of the events in pd have been added to ed and there must be a difference between the size of pd and the size of the intersection of pd and ed . So, if ei ∈ toAdd the guard is true, and it is false otherwise. Therefore we can reduce the guard to. . . ei ∈ toAdd (36) To take stock, if we assume that pd = {e1 . . en }, we now are at a system model that looks like this. . . toAdd : [toAdd = pd ] ; do e1 ∈ toAdd
→ ed , toAdd :
ed = ed0 ∪ {e1 } ∧ toAdd = toAdd0 \ {e1 }
[] .. .
(37)
[]
en ∈ toAdd
→ ed , toAdd :
ed = ed0 ∪ {en } ∧ toAdd = toAdd0 \ {en }
od i.e. we have a system that has as many guarded commands in its loop as there are events in pd . In an execution each of these guarded commands fires once and then disables itself.
WP25
Cognitive plausibility
20
Butterworth, Blandford & Duke
The order in which this takes place is non-deterministic, the semantics of the do-loop assert this. We argue that we can simplify this program by collapsing all the guarded commands into one command that non-deterministically selects an event from toAdd , adds it to ed and removes it from toAdd . The guard for this collapsed command is that there are events in toAdd . Hence we arrive at the following system model. . . toAdd : [toAdd = pd ] ; do
∃e • e ∈ toAdd ∧ toAdd = ∅ → ed , toAdd : ed = ed0 ∪ {e} ∧ toAdd = toAdd0 \ {e} od
(38)
. . . where we have effectively migrated the non-determinism of the do-loop to the nondeterminism of the existential quantifier that selects an event from toAdd . Now it is evident that, by a little twisting and turning, changing the existentially quantified variable e to a local variable, sequencing the conjunctions in the command and replacing ed = ed0 ∪ {e} by the device action addEvent(e), we have arrived at the program fragment oneByOne: oneByOne = ˆ
toAdd : [toAdd = pd ] ; do toAdd = ∅ → var e • e : [e ∈ toAdd ] ; addEvent(e); toAdd : [toAdd = toAdd0 \ {e}] od
To recap: we started with a base model which incorporated various ideas about rationality and goal driven behaviour. We then instantiated that model with information specific to the example in question, making explicit assumptions about the system as we did. We then simplified this instantiated model greatly by a series of equivalences and finally a little refinement. By this process we claim that the resultant model oneByOne has cognitive plausibility based on the assumptions we made in the process of deriving it. We now continue and do the same for the other two system models enMasse1 and enMasse2 , but we shall not go into as much detail as we did for the one by one strategy. (Many of the assumptions we make for the en-masse strategies are the same as for the one by one strategy anyway and do not need repeating.)
4.5
Deriving the en-masse strategies
For the en-masse strategies we assert that as initialisation the user enters an arbitrary set of events es into the electronic diary and then works out which events need to be subsequently added and deleted.
WP25
Cognitive plausibility
init =
Butterworth, Blandford & Duke
addSeries(es); toAdd : [toAdd = {e | e ∈ es ∧ e ∈ pd }] ; toDelete : [toDelete = {e | e ∈ es ∧ e ∈ pd }]
21
(39)
Now assume that we follow a similar process of instantiation as we did for the one by one strategies and we will finish up with an argument that a rational user will add events that have not been added by setSeries, but crucially we cannot argue that they will delete events, even events that are added incorrectly by addSeries. For a guarded command to be executed it must cause a reduction in the value of the R function and the guarded delete command does not. If we instantiate the 0 ≤ R(delOp ei (bel )) < R(bel ) with the rationality function Ruser we arrive at the following. . . 0 ≤ |pd | − |pd ∩ ed \ {ei }| < |pd | − |pd ∩ ed \ {ei }|
(40)
Assuming that ei is in ed , removing it either does not affect the size of the intersection between pd and ed or decreases it. It is not rational to delete events in ed that are also in pd as that would increase the size of the intersection, but deleting events incorrectly added by setSeries cannot increase the size of the intersection; therefore deleting events moves the system no nearer the goal. The guards for deleting events will reduce to false when instantiated with the rationality function Ruser and therefore deleting will not occur. Therefore we arrive at the program fragment enMasse1 . . . ˆ enMasse1 = addSeries(es); toAdd : [toAdd = {e | e ∈ es ∧ e ∈ pd }] ; do toAdd = ∅ → var e • e : [e ∈ toAdd ] ; addEvent(e); toAdd : [toAdd = toAdd0 \ {e}] od (Note that we can get rid of the fragment that sets up the set toDelete; it is not needed.) So we conclude that we can derive enMasse1 from the base model and the goal user , but not enMasse2 . We claim that enMasse1 is cognitively plausible, whereas enMasse2 is not.
4.6
Design implications for Meeting Maker
To step back from the mathematics, what we have done could be argued to not be very surprising, particularly with the en masse strategy. We gave the user an ‘incorrect’ goal and showed that with that goal they may use the system incorrectly; we do not need several pages of models to show us this. However what is much more interesting is that we gave the user the same ‘incorrect’ goal with the one by one strategy and showed that, even so, they
WP25
Cognitive plausibility
The interaction commences
→
22
Butterworth, Blandford & Duke
The user perturbs the system state in order to achieve their explicit goal
→
The user achieves their explicit goal
→
The user then reinstates the system state?
Figure 9: A scheme for error prone systems use the system correctly. What is it about the one by one strategy that makes it immune to this error? We argue that there are a class of systems that are prone to this type of error, namely systems which require that the user perturbs the system state in order to achieve some goal and then leaves it up to the user to reinstate the perturbed system state subsequently. A general statement of these types of systems is given graphically in figure 9. A solution to this problem is to prevent the user completing their interaction until they have reinstated the perturbed state. In the case of the en masse strategy we may design an interface which prompts the user to delete events after addSeries has been invoked and then prompts the user to add events. In this way the device compensates for the user’s lack of motivation to delete events, by insisting that the user deletes events before adding events (which the user is motivated to do). Another solution is not to look at the subsequent correcting loop, which could be argued to be always problematic, but to concentrate on addSeries so that it adds the paper diary precisely, thus rendering the subsequent correction loop unnecessary. This could be achieved by have the user create a series of events which the user must validate to be correct before it is added to the electronic diary.
5
Conclusion
We have shown a formal basis for an argument for the ‘cognitive plausibility’ of a system model. This argument is analogous to an argument for functional correctness of a system model. Functional correctness can be asserted by starting from a statement of requirements and transforming that statement by rules from the refinement calculus until a specific system model is reached. That system model is then asserted to be functionally correct. Cognitive plausibility can be asserted by starting from a base model of user behaviour and instantiating that base with assumptions about the user and the device with which the user interacts. That instantiated model can then be transformed, mostly by simplifying equivalences and by standard refinement rules, to a simple tractable model of system behaviour, in which many of the assumptions relating to the user have become implicit. Just as functional correctness should only be expressed in relation to some requirements. (e.g. ‘system x is correct with respect to requirements y’) then cognitive plausibility should only be expressed in relation to the assumptions that are made in the argument (e.g. ‘system a is cognitively plausible with respect to assumptions b on the base model c’). So our claim for the cognitive plausibility of oneByOne should read ‘oneByOne is cognitively plausible with respect to the assumption that the user works towards goal user on the base model baseModel .’ It is important to note that many the cognitive science issues introduced in section 3
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
23
have been considerably simplified in section 4; there is no distinction made between belief and device state, and the goal and R function have become very implicit.
5.1
Making assumptions explicit
We argue that the abstraction that is promoted by formality causes the analyst to think about the system in question in these more general terms. When we started investigating Meeting Maker we did not realise in advance that it was one of these sorts of systems. The analysis described in this paper made this fact clear, but debate remains as to exactly why users sometimes behave in this way. Implicit and explicit goals is one explanation; there are others [34]. Crucially our assumptions are explicit, as are the consequences of those assumptions. Our base model and the assumptions are not definitive. All psychology deals in shades of grey to a much greater extent than computer science. What our approach does is make explicit the assumptions that are made about the user and therefore open them to scrutiny. This then brings our work back to one of the tenets of Programmable User Modelling; that a modelling approach should force the modeller to make explicit their assumptions, first proposed by Young et al. [33]. Our later work has achieved this in a much more mathematical manner, with a more rigorous theoretical basis.
5.2
Specification ‘styles’
We have deliberately not introduced any new mathematics or notation in this paper. We believe that existing mathematical tools are sufficient to reason about most interactive systems, and the creation of new notations in many cases only confuses issues. That is not to say that choice of notation is not important, but that choice is not as important as the concepts that are expressed in the chosen notation; a notation should not obscure those concepts. It is important to realise that any model is really a communication between parties; the models we have presented in this paper are intended to communicate cognitive science issues to a formal audience. If we were attempting to communicate these ideas to a cognitive science audience we would have chosen a radically different form for the models. The notation will affect the style of a specification and to what uses it is put; for example the schema style of Z [32] allows for easy reasoning about composition of smaller components, but it may be difficult to reason about the interaction of those components. The uses to which a specification is put should dictate the notation, not vice versa. What has emerged from our work is a specification ‘style’ that we have expressed in several notations (MAL [31], TLA [23] and Morgan’s specification fragments as used in this paper). For any system there are an unlimited number of specifications of it, that represent different viewpoints and uses to which the specification is put. Various authors have proposed specification styles for classes of applications (e.g. Hall [18] described an object oriented style of Z specification, and Barros [12] has developed techniques for specifying relational databases). Our work is a move towards a style for specifying interactive systems where user assumptions are built in and can be tractably reasoned about. Also we have deliberately used only existing mathematical techniques and notations; we were interested to see what well defined and stable mathematical paradigms could say about interactive
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
24
systems, rather than deciding that HCI is in some way too strange to be dealt with using traditional formal methods work. The specification style shown by Blandford, Butterworth and Good [3] makes an explicit link to cognitive science; i.e. one can pick out bits of the specification and see whether it comes from the device or user model. The specification style shown by Butterworth, Blandford and Duke [8] makes a more implicit link to cognitive science, but results in a more elegant model. This paper has discussed a mathematical basis for this implicit link to cognitive science.
Acknowledgements The following have made helpful comments about the many previous drafts of this paper: John Cooke, Richard Young, Jason Good, Paul Cairns, Paul Curzon and Jo Hyde. This work is funded by EPSRC grant GR/L00391. For further details see the web pages at http://www.cs.mdx.ac.uk/puma/.
References [1] J. Anderson. The Architecture of Cognition. Cognitive Science. Harvard University Press, 1983. [2] P. J. Barnard. Interacting Cognitive Subsystems: A psycholinguistic approach to short term memory. In Ellis A., editor, Progress in the Psychology of Language, volume 2, chapter 6, pages 197–258. Lawrence Erlbaum Associates, London, 1985. [3] A. Blandford, R. Butterworth, and J. Good. Users as rational interacting agents: formalising assumptions about cognition and interaction. In Harrison and Torres [20], pages 43–60. [4] A. Blandford and R. M. Young. Separating user and device descriptions for modelling interactive problem solving. In K. Nordby, P. Helmersen, D. Gilmore, and S. Arnesen, editors, Human-Computer Interaction — INTERACT’95, pages 91–96. Chapman and Hall, 1995. [5] P. Bumbulis, P. Alencar, D. Cowan, and C. Lucena. Combining formal techniques and prototyping in user interface construction and verification. In P. Palanque and R. Bastide, editors, Proceedings of Design, Specification and Verification of Interactive Systems ‘95, pages 174–192. Eurographics, Springer Verlag, 1995. [6] R. Butterworth and A. Blandford. The principle of rationality and models of highly interactive systems. In M. A. Sasse and C. Johnson, editors, Human-Computer Interaction — INTERACT ’99, pages 417–425. IOS Press, 1999. [7] R Butterworth and A. Blandford. The roles played by models of interactive systems. Submitted to HCI journal. It is available as PUMA working paper WP23. See the world wide web pages at www.cs.mdx.ac.uk/puma, 1999.
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
25
[8] R. Butterworth, A. Blandford, and D. Duke. The role of formal proof in modelling interactive behaviour. In Markopoulos and Johnson [24], pages 87–101. [9] R. Butterworth, A. Blandford, D. Duke, and R. Young. Formal user models and methods for reasoning about interactive behaviour. In J. Siddiqi and C. Roast, editors, Formal Aspects of the Human-Computer Interaction, pages 176–192, 1998. [10] M. D. Byrne and S. Bovair. A working memory model of a common procedural error. Cognitive Science, 21(1):31–61, 1997. [11] S. Card, T. Moran, and A. Newell. The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates, 1983. [12] R. S. M. de Barros and D. J. Harper. A method for the specification of relational database applications. In J. Nicholls, editor, Proceedings of Z User Workshop, pages 261–286. Springer, 1991. [13] E. Dijkstra. A Discipline of Programming. Prentice Hall, 1976. [14] A. Dix. Formal Methods for Interactive Systems. Computers and People Series. Academic Press, 1991. [15] D. Duke and M. Harrison. Abstract interaction objects. Computer Graphics Forum, 12(3):25–36, 1993. [16] D. J. Duke, P. J. Barnard, D. A. Duce, and J. May. Syndetic modelling. HumanComputer Interaction, 13(4):337–393, 1998. [17] J. Good and A. Blandford. Incorporating human factors concerns into the design and safety engineering of complex control systems. In J. Noyes and J. Bransby, editors, People in Control: An International Conference on Human Interfaces in Control Rooms, Cockpits and Command Centres, pages 51 – 56, 1999. IEE Conference Publication Number 463. [18] A. Hall. Using Z as a specification calculus for object oriented systems. In D. Bjorner, C. A. R. Hoare, and H. Langmaack, editors, Proc. VDM’90: VDM and Z - Formal Methods in Software Development, pages 290–318. Springer Verlag, 1990. Vol 428 of Lecture Notes in Computer Science. [19] M. Harrison and H. Thimbleby, editors. Formal Methods in Human-Computer Interaction. Cambridge series on HCI. Cambridge University Press, 1990. [20] M. Harrison and J. Torres, editors. Design, Specification and Verification of Interactive Systems ’97. Eurographics, Springer, 1997. [21] B. John and D. Kieras. The GOMS family of user interface evaluation techniques: comparison and contrast. ACM Transactions on Computer-Human Interaction, 3(4):320– 351, 1996.
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
26
[22] B. John and D. Kieras. Using GOMS for user interface design and evaluation: which technique? ACM Transactions on Computer-Human Interaction, 3(4):287–319, 1996. [23] L. Lamport. The temporal logic of actions. ACM Transactions on Programming Languages and Systems, 16(3):872–923, 1994. [24] P. Markopoulos and P. Johnson, editors. Design, Specification and Verification of Interactive Systems ’98. Eurographics, Springer, 1998. [25] J. J. Ch. Meyer and W. van der Hoek. Epistemic Logic for AI and Computer Science. Cambridge University Press, 1995. [26] C. Morgan. Programming from Specifications. Prentice Hall international series in computer science. Prentice Hall, 1990. [27] A. Newell. The knowledge level. AI Magazine, pages 1–20, 1981. [28] A. Newell. Unified theories of cognition. Harvard University Press, 1990. [29] P. Palanque and R. Bastide. Petri net based design of user-driven interfaces using the Interactive Cooperative Objects formalism. In F. Patern`o, editor, Proceedings of the Eurographics Workshop on Design Specification and Verification of Interactive Systems ’94, pages 215–228. Eurographics, 1995. [30] P. Palanque and F. Patern`o. Formal Methods in Human-Computer Interaction. Formal approaches to computing and information technology. Springer, 1998. [31] M. Ryan, J. Fiadeiro, and T. Maibaum. Sharing actions and attributes and modal action logic. In T. Ho and A. Meyer, editors, Proceedings on the International Conference on Theoretical Aspects of Computer Science, pages 569–593. Springer Verlag, 1991. [32] J. Spivey. The Z Notation: A Reference Manual. Series in Computer Science. Prentice Hall International, 1989. [33] R. Young, T. Green, and T. Simon. Programmable user models for predictive evaluation of interface design. In K. Bice and C. Lewis, editors, Proceedings of CHI ’89: Human Factors in Computing Systems, pages 15–19. Association of computing machinery, 1989. [34] R. M. Young, P. Barnard, A. Blandford, and M. Harrison. The unselected window scenario. Technical Report CP52, Amodeus ESPRIT BRA 7040, 1994.
A
Proofs
There follow sketch proofs for assertions made in the body of the paper.
WP25
A.1
Cognitive plausibility
27
Butterworth, Blandford & Duke
req oneByOne
We sketch out the major refinement steps that take req to oneByOne. Firstly we introduce a logical constant result which is set to the value required for ed , i.e. ed0 ∪ pd con result • ed : [ed = ed0 ∪ pd ]
ed :
result = ed ∪ pd , ed = result
(41)
We can then introduce the variable toAdd , set it to the initial value of pd , rewrite the occurrence of pd in the pre-condition of the subsequent specification and assert that the value of toAdd is the empty set on termination. We can then rewrite the post-condition result = ed as result = ed ∪ toAdd because we have asserted that in the post-condition toAdd is empty.
ed :
result = ed ∪ pd , ed = ed0 ∪ pd
toAdd : [toAdd = pd ] ; result = ed ∪ toAdd , (42) ed , toAdd : result = ed ∪ toAdd ∧ toAdd = ∅
We now use the expression result = ed ∪ toAdd as a loop invariant, toAdd = ∅ is the loop guard and we propose the following loop variant; ∅ ⊆ toAdd0 ⊂ toAdd , i.e. the size of toAdd decreases. Based on this we can now rewrite the second specification in the sequence as a loop, as follows. . . do
toAdd = ∅ → ed , toAdd :
toAdd = ∅ ∧ result = ed ∪ toAdd , result = ed ∪ toAdd ∧ ∅ ⊆ toAdd0 ⊂ toAdd
(43)
od We now work on the loop body. The precondition and the postcondition both assert that result = ed ∪ toAdd , but the postcondition also asserts that the size of toAdd reduces. Hence we, by simple set theory, conclude that ed must have the difference between the initial and final values of toAdd added to it. i.e. the body of the loop reduces to. . .
ed , toAdd :
ed = ed0 ∪ (toAdd0 \ toAdd ) ∧ ∅ ⊆ toAdd0 ⊂ toAdd
(44)
(Note that the logical constant result has been factored out of this specification and it can now be removed.) Now we tighten the precondition to assert that toAdd is reduced by just one event e, and correspondingly that event is added to ed .
∃e • e0 ∈ ed ∧ ed , toAdd : ed = ed0 ∪ {e} ∧ toAdd = toAdd0 \ {e}
(45)
We can then split this specification up and sequence its components making e a local variable. To conclude we place it in the context of the initialisation and the loop as
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
28
follows. . . toAdd : [toAdd = pd ] ; do toAdd = ∅ → var e • e : [e ∈ toAdd ] ; ed : [ed = ed0 ∪ {e}] ; toAdd : [toAdd = toAdd0 \ {e}] od
(46)
Now, rewriting ed : [ed = ed0 ∪ {e}] as addEvent(e) gives us the one by one strategy from figure 3. Therefore we can conclude that req oneByOne.
A.2
req enMasse1
We prove that enMasse1 is not a refinement of req by counter-example. If enMasse1 is correct with respect to req then the final size of ed must be equal to the size of ed0 added to the size of pd . ed = ed0 ∪ pd ⇒ |ed | = |ed0 | + |pd |
(47)
We argue that enMasse1 can terminate with ed being greater than |ed0 |+|pd | and therefore cannot be a refinement of req. addSeries and addEvent are the only two fragments in enMasse1 that effect ed and neither of them decrease the size of ed . Therefore if addSeries adds a set of events that is larger than pd then there is no way that the size of ed can be subsequently reduced. addSeries adds an arbitrary set of events to ed , so there is no guarantee that it will add a set that is smaller than pd , hence we can argue that enMasse1 may terminate with |ed | < |ed0 | + |pd | and therefore is not correct with respect to req.
A.3
req enMasse2
Demonstrating the req enMasse2 is a case of showing firstly that. . . ed0 ∪ pd = ed0 ∪ ((es \ toDelete) ∪ toAdd )
(48)
(i.e. that ed0 ∪ pd , the value required of ed , is obtained by adding es to ed0 then removing toDelete and adding toAdd ), and then showing that the strategy enMasse2 adds es and toAdd to ed0 and removes toDelete. ed0 cancels out on both sides of the above equation so that leaves us having to show that. . . pd = (es \ toDelete) ∪ toAdd (49) Now we substitute the definitions of toDelete and toAdd . pd = (es \ {e | e ∈ es ∧ e ∈ pd }) ∪ {e | e ∈ es ∧ e ∈ pd }
(50)
This formula can be seen to be true; if you take all the events from es that are in es, but not in pd then add all the events that are not in es, but are in pd , you must finish up with pd itself.
WP25
Cognitive plausibility
Butterworth, Blandford & Duke
29
Now we need to show that es and toAdd are added to ed and toDelete is removed. This is done by following through a refinement very similar to that shown in proof A.1, therefore it is omitted.
