Action View and Action History View Mechanisms for Cooperative Work Environments Yahiko Kambayashi

Kazimierz Subieta

Kenji Fujita

Integrated Media Environment Experimental Lab. Faculty of Engineering, Kyoto University Sakyo, Kyoto 606-01, Japan fyahiko,subieta,[email protected] Fax: +81-75 753-59-65 ce [TaKa93], which supports the oce work in distributed environments utilizing the object-oriented database technology. Here VIEW stands for Virtual Interactive Environment for Workgroups 1 . The system is based on server-client architecture and supports a multi-user environment. The Action View and the Action History View mechanisms are implemented using HP Distributed Smalltalk [HP94]. The idea of the action view and action history view mechanisms is based on the analysis of popular scenarios of CSCW. Common hypermedia-based presentation systems are based on the WYSIWIS principle (What You See Is What I See): the users share the same displays. For many applications such a work mode is undesirable, e.g., for security/privacy reasons, or because of overloading of a work partner's screen by irrelevant information. Thus it is necessary to restrict data objects that can appear on remote displays. This can be done by several methods, for example, by explicit determining which objects have to be exported outside, or by subdividing the screen area into private and public parts. In many situations, however, these methods are cumbersome and unreliable. We propose the method based on object views , which restrict kinds of objects that may appear on remote screens. Another problem concerns a situation when collaborative work has to be done by the partners at dierent time. In this case a remote work partner A should see not only objects that appeared previously on the screen of B, but should also have the possibility to reply all the actions performed by B. Again, it would be desirable to restrict the information shown in these replays, because of security/privacy and possible overloading of a screen. In this paper we propose a method of the information export restriction for CSCW systems. It is based on the well-known idea of database views . A view for a database (object base) can be de ned as a

Abstract In this paper two new view mechanisms, called action view and action history view, are presented. They are developed for the VIEW Oce, a system which can be used to realize a distributed virtual oce environment. The mechanisms support coordination of users by means of the database technology. For security/privacy and other reasons it is not desirable to show all contents of a user's screen to other users. The action view mechanism makes it possible to select exported screen objects. Histories of actions shown on a screen are recorded, thus coordination among users working at dierent time is supported. The recording uses a symbolic form and histories' elements are marked by timestamps. Redundant operations can be removed from a record. The action history view mechanism deals with such histories of actions observed on screens. It supports further replays of action histories; the replay speed is variable. Data and sequences of actions satisfying some conditions required by a remote user can be also selected. For this purpose the action history view language (AHVL) has been developed. The paper presents both AHVL and its user interface.

1 Introduction

In this paper we present a new view mechanism for CSCW (Computer-Supported Cooperative Work) [GrSa87, LMY88, EGR91] called action view . It supports cooperation of users satisfying security/privacy constraints. The security/privacy issues are important for CSCW systems, as for many applications they are a prerequisite for the computer-aided collaboration. To make the cooperation among users working in dierent time possible we also developed an extension of this mechanism called action history view . These mechanisms are implemented for a recently developed system, called VIEW Of-

1 Formerly VirtualOce, but it has been renamed because "virtual oce" has became a general term.

1

mapping of each possible database state (a database instance) into another (virtual) state. For the purposes of CSCW it is necessary to represent not only database states, but also operations that have been performed to change the states. Thus we introduce the concept of database actions as a combination of objects' states and operations that have been applied to the objects. As in case of objects shown on screens, for user coordination it may be necessary to present database actions at other workstations. Again, due to security/privacy constraints it is undesirable to present on other workstations all such actions. For example, assume that users A and B work together for project , but user A also works for project while user B does not. Hence operations related to project should not be seen by user B. Thus, selection of database actions is necessary. Generalizing the concept, an action history is de ned as a sequence of actions shown on a display, and a database action history is de ned as a sequence of database actions. A function which maps such a database action history into its part is called an action history view . This concept corresponds to a view for a temporal database [Snod94], de ned as a part of data in the whole history of the database (for example, see [XuKa87]). Action histories can be stored by the video technology which enables the user to record sequences of display contents of each workstation. Conventional data compression techniques, such as MPEG, are commonly applied to reduce the required storage. It is not possible, however, to realize exible operations on such a history, because automatic separation and structuralization of data and operations are unattainable. Thus we decided to use the symbolic recording for an action history. It consists of operations stored in a symbolic form together with the data that have been modi ed by them. Basic advantages of the symbolic recording method over the video based recording method are as follows:

The following issues should be taken into account when considering the operation history recording:

. An operation history should make it possible to record repeated "undo" and "redo" operations.

. In case when several versions are recorded, only

one version need to be stored in the original form. Other versions can be generated from one of the stored versions and from a sequence of operations that have been applied to it.

. In order to reduce the number of user's operations,

frequently used sequences of operations should be detected and the user can select one of the sequences.

Such problems are already partly solved in some CSCW systems. For example, GINA [BeSp90, BeSp92] uses an operation history to search speci c operations in order to undo them. It also realizes the replay function. The view functions which are discussed in this paper, however, are not implemented in this system. In the VIEW Oce system we implemented all GINA functions together with new ones, in particular, with the mentioned above view mechanisms. We argue that although the recording of operation history is not a new idea, these view mechanisms are not considered before. The rest of the paper is organized as follows. In Section 2 we indicate basic problems in supporting coordination of users working at remote workstations, and present typical solutions of them. Section 3 outlines the VIEW Oce system. Section 4 presents the Action View mechanism. In Section 5 we discuss issues related to recording and playing an action history. Section 6 outlines the Action History View Language (AHVL), and Section 7 presents its user interface. Section 8 contains our concluding remarks.

2 Realization of Action Views and Action History Views

. Access to operations and data can be restricted

thus security/privacy and other constraints can be satis ed.

In this section we indicate basic problems in supporting coordination of users working at remote workstations and present typical solutions of them. These problems motivated us to introduce the mechanisms that are the subject of this paper.

. Logically redundant operations can be eliminated.

A pair consisting of an operation and its twin undo operation can be eliminated together. Intermediate locations of a mouse need not to be recorded.

2.1

Object Views and Action Views

. Retrieval of a partial history can be speci ed in

The following well-known methods are used to support coordination of users working at remote workstations:

The amount of data required to store an action history by the symbolic recording is much smaller than that by the video recording; nevertheless, the erase operation was introduced to remove unnecessary histories.

. Video cameras

a high-level language (for example, " nd the part where operation a is applied to objects b and c").

They are very popular to accomplish conversation among several users. Due to them the eye contact and awareness can be realized easily. The communication cost for video, however, is much higher

2

. (c) Change of a display method

than that for characters. In particular, if video signals are sent by several users via an Ethernet LAN, pictures move very slowly.

For example, if a user wants to see larger characters due to an eye problem, s/he can change the character size as well as the window size. S/he can also distinguish icons by changing their colors.

. Control and observation of remote workstations

For example, Timbuktu oers a function to observe displays on remote workstations. This system enables two users to work together by displaying the screens of both workstations. Exchange of data among them can be performed by copy-andpaste operations.

The presented functions cannot be supported by the methods that we have described at the beginning of this section.. Moreover, classical approaches to database views (e.g., determining a view through an SQL query) also do not satisfy well our requirements. To this end we have developed a new concept of a deputy object [PeKa95]. It is used to de ne objects that are strongly related, but not identical. The relationship between an object and its deputy objects is of dierent quality from the conventional inheritance relationship assumed in classical object-oriented approaches. A deputy object b corresponding to some original object a can have different attributes, dierent values, and dierent methods. For example, objects in Fig.1 a, b, and c can be regarded as deputy objects of the objects at the lefthand screen of Fig.1. These deputy objects have all basic properties of their originals, except some secondary attributes such as their locations on a screen, their size, etc. We argue that the distant cooperation among users requires also a new object view idea which would make it possible to map and represent objects' changes. This kind of a view is called here an action view . An action view maps both a database object's state and actions performed on it. The action view concept can be further extended by considering sequences of such action views; the sequence re ects succeeding changes of the object's state. Such a sequence is called an action history view . An action history view can be de ned by a logical expression. The result of the view is displayed for the user. Note that in this way we can obtain many interesting eects; in particular, some object may appear or disappear on the screen, depending on succeeding results of its modi cation. For example, if user A can see all documents containing the keyword "database", then a document may appear or dissapear on the screen of the user A if such a keyword is added or removed by updating operations.

Both methods are widely used in CSCW applications; however, none of them oers view functions. We argue that for privacy/security this functionality is very important. For example, some window may display user's personal mails, which should be kept secret from others.

a) selection B

A C

B

A b) modification of the display

C

B

A C

c) changing a display method

B

A C

Figure 1: Possible changes of a display Typical object view functions are shown in Fig.1. The following functionalities are distinguished:

. (a) Selection of objects to be displayed

If some displayed objects should not be seen by others due to security, privacy or an overloaded screen, the user shoud have the possibility to hide them.

2.2

Methods of Realization of Action History Views

. (b) Change of objects' locations

Each user may have his/her own preferences concerning objects' locations. For example, objects arranged as seen on the left-side screen can be presented for another user as shown in Fig.1b. In particular, if there are overlapping windows at the original display, another user can locally rearange them (on his/her workstation only) by standard changing location functions.

The following methods can be considered as relevant to realize action histories and action history views.

. Video tape recording

It is economical, since a video tape can store 6 hours or even more. The method is especially useful when a cooperation history contains discussion among users with the use of a video recorder.

3

. Digital video recording with compression

have introduced advanced view functions, which are the subject of this paper.

The popular MPEG format makes it possible to store pictures longer then one hour in a single CVD (Compact Video Disc). As known, screens with no change are not recorded by this format. In consequence, because usually during the program presentation there are intervals with no movement, we can record much longer time.

. The system should enable the user to retrieve objects by their logical properties as well as by their physical locations in the oce space.

. Real oce and virtual oce: In a display it should

be possible to de ne a window representing a real existing oce, as well as a window representing an imaginary oce, which re ects essential aspects of the collaborating group.

. Symbolic form

Operations on the screen are recorded in the database as database structures which can be accessed and processed by standard data manipulation methods. If the operations are recorded with time stamps, we can replay previous work. This makes it possible various view functions to be realized; in particular, data at any moment can be selected. This method requires structuralization and separation of objects shown on a display, thus practically cannot be applied for windows which display video pictures.

. Real and virtual equipments: Each equipment

unit in the oce space can be mapped as a single object on the screen, but the system should enable the user to de ne virtual equipment. For example, a super fax can be de ned as a combination of email and fax, with the assumption that a fax is sent if the corresponding email address is unknown.

. Real and virtual persons: It should be possible to

. Composite form

see a group of people as a single person. If some user charges a virtual person with a particular job, the fact that this job is done by more than one person is transparent for him/her.

It consists of a video part and an attribute part. The attribute part can be considered as a simpli ed sort of the symbolic form. For example, "start and nish the use of document A" can be recorded, thus one can retrieve a part of the objects' history where the document A is used. This method makes it possible to realize some restricted version of the previously discussed view mechanisms.

. Integration of a personal work space and a group work space. In most groupware systems, there is a clear distinction between a personal work space and a group work space. In real oce personal and group work cannot be separated. The VIEW Oce makes it possible to integrate the personal and group work spaces.

The symbolic form is the most powerful for further methods aiming the \intelligent" behaviour of a CSCW system; however, it is dicult to implement. The main source of the diculty is the necessity of manual indexing of documents and other objects being the subject of the user collaboration. In the following we discuss appropriate implementation techniques which make it possible to reduce this overhead. Note that in many practical cases we cannot avoid the video recording to store the video part, thus a combination of the symbolic form and the video forms is the most reasonable.

. Integration of various kinds of communication equipment, which is required for interaction.

. Security Management: Because an oce consists

of many individuals performing various jobs, traditional forms of security are not sucient. We argue that new security forms are required [LLK93], smoothly integrated with user interfaces.

. Capability for constructing a personal oce space.

In a real oce, a person can arrange his/her own working space. This property have to be supported by VIEW Oce. In particular, an oce worker can prepare a virtual oce for each project s/he is doing. This function enables him/her to represent clearly the structure of an oce. Fig.2 shows an example display of personal space and group space.

3 Outline of the VIEW Oce The VIEW Oce is designed to support various aspects of humans' work in their oce. In this paper we roughly present its characteristics; more comprehensive description can be found in [TaKa93]. Both the action view and action history view mechanisms are implemented as parts of this system. During the design of the system we attempted to realize the following con icting requirements and features:

4 The Object/Action View Mechanism

. The system should support cooperation among

In this section we discuss a language for the de nition of action views and for dealing with related security problems. The language is based on the object-

users without sacri cing independence of them; in particular, should support privacy. To this end we 4

The is a Smalltalk [GoRo83] block expression which for each argument object returns a boolean value. The examined object is identi ed by the prede ned variable self, as shown in the following example:

de ne view project "VIEW" owner "fujita" condition [self class = Folder] The same language is used for an action view. Such a view enables the user to see operations which have been applied to the objects selected by the view. To cope with security and privacy problems the system makes it possible to assign to each combination of an object and a user the following three kinds of security permissions:

. Existence permission

In case when existence permission to access an object is not granted to a user, the user cannot recognize the object, i.e., cannot know that the object exists. A GUI icon which represents the object does not appear in the display.

. Read permission

In case when read permission to access an object is not granted to a user, the user cannot execute or open the object.

. Write permission

In case when write permission to access an object is not granted to a user, the user cannot execute operations which might change attribute values of the object.

Frequently, the de nitions of permissions for every combination of a user and an object is operationally cumbersome. To simplify the task, the system enables the administrator to de ne the permissions by a single statement which combines a set of users and a set of objects. As follows from the above assumptions, an object is readable if and only if the existence permission and read permission concerning the object are granted to the user and the object is not hidden by the view which is de ned for him/her.

Figure 2: Personal space and group space oriented paradigm. A user's operation is de ned as a series of user's input operations that causes to send a single message to an object. For example, when a user clicks a mouse button on an object in the window, drags it to another object and then releases the button, then a message is sent. This series of inputs is de ned as an operation. The object view language is de ned to specify objects that appear on remote displays. Its syntax is the following:

5 Recording and Playing an Action History

::= de ne view [project ] [owner ] [condition ]

In this section we discuss basic notions which are relevant to the action history idea. The main goal is the mapping of all essential features of objects' states , as well as actions performed on the objects. The mapping should support further operations on a history, such as selection of sub-histories, reply of a whole history or its part, etc. We view a history as a sequence of snapshots , where each snapshot re ects either objects' states,

::= As seen, all parts , and are optional. Objects satisfying the speci cation are selected as the result. 5

of objects should be restored according to the Aold component of this snapshot. Then, the method determined by the message component can be executed with proper object's states. If the method depends on absolute time and/or other variable or undetermined factors of the system's environment, the result objects' state can be dierent from Anew . Thus, the Anew component makes it possible to restore the nal objects' state which was at the original time, and then, to show it to the user. In order to replay as perfectly as the original pictures that had been shown on the screen, all user input (the time when a mouse button is clicked, the location of a mouse cursor at each time, and so on) should be recorded into the history. In this case, however, the history requires a bit larger amount of memory. Moreover, the Smalltalk programming environment makes it dicult to recognize a mouse cursor location if its button is not clicked or released. For this reason in a snapshot we record only key input, mouse click, mouse release, cursor location of a mouse when it is clicked or released, and the time when the operation ends. Because of the lack of information, the speed of operations of application may not be the same as the original one. However, in general, the user can change the speed of the replay depending on the importance of operations.

actions performed on objects, and time dependencies. There are several possibilities to determine time moments when snapshots have to be made and what a snapshot should contain. We considered that for our system the most convenient is the event-driven paradigm. Because any change to a database is triggered by a user operation, we associate a snapshot with such an event. The objects' states between events are not changed, hence snapshots made between events cannot bear a new information. In Smalltalk an event is represented as a message sent to an object; a message triggers some method associated with an object's class. Following this concept and terminology, snapshots of our action histories have the following components:

. R: an object identi er of an object, a receiver of a message;

. M : the message having the object identi er as the parameter;

. Aold : it represents attribute values of objects before the method determined by the message is executed

. Anew : it represents attribute values of objects after the method is executed;

. I : user's input which triggered the operation; . T : the time when the execution of the method is

6 The Action History View Language

nished.

. S : a name of a user who triggered the operation. To avoid redundancies in action histories, if some attribute values are not changed, instead of copying Aold or Anew , a link to previous values is generated. Note that action histories may contain secret information, thus for security reasons the access to histories must be restricted. A snapshot of a history is accessible to a user if and only if all components of the snapshot are readable to the user, according to the previous de nition. A stored action history can be used to replay sequences of displays at dierent time. Let a history h =

be a sequence of snapshots hi . The user can replay the whole history, or only such its snapshots that satisfy a given condition; in such cases the order of displayed snapshots is preserved. The user can also replay the history in the reverse order, or in arbitrary order, possibly with the duplication of some snapshots. To this end the technique is as follows. Let s = be a sequence of natural numbers, possibly with repetitions. The user can reply h in the order determined by s; that is, the sequence of snapshots for reply is . For example, if s = then the snapshots of h will be replayed in the order h1 ; h2 ; h4 ; h3 ; h2 . The algorithm of replying action histories has peculiarities connected with the necessity to restore objects' states. Before each snapshot is replayed, a state

The Action History View Language (AHVL) and its interpreter enable the user to choose snapshots to be replayed from a particular history, to determine their order, and to restrict their content. The AHVL search conditions are determined in visual GUI panels, Fig.5 and Fig.6. The rst panel makes it possible to determine basic search attributes. Its syntax is the following.

::= [Users f , ...g] [Search Expression ] [Time Interval ] ::= [from ] [to ] ::= j now j now - ::= select j skip The Users eld speci es the list of user names. It selects snapshots from the current history that were 6

operated by a user from this list. If this eld is left blank, all snapshots of the history are considered. The Time Interval eld makes it possible to select snapshots that were performed during the speci ed time. The user can specify it in several formats, in particular, in the format which is relative to the actual time (now). As in the previous case, if the time interval is not speci ed, all snapshots are considered. The eld Search Expression enables the user to specify arbitrary conditions concerning the history. It is usually rather easy to design a language for specifying such conditions assuming they are static; that is, do not take into account the order of snapshots to be replayed. To this end we use standard Smalltalk block expressions. For many applications, however, this kind of conditions is too restrictive. For example, the user is interested only in repeated sequences of actions \read document ", \read document ", \change document "; he is not interested in other sequences and other snapshots that may occur inside this sequence. Thus in our design we attempted to support the user with dynamic conditions expressing time dependencies between snapshots to be replayed. The method that we apply in AHVL is based on regular expressions . As well-known, a regular expression is a grammar specifying some collection of sequences of symbols. Let

states according to \signals" being marks assigned to history snapshots. (There are some details of our design which are not interesting for the subject of this paper.) We also generalize the idea for the case when a snapshot can be marked by more than one symbol. An example of a more complex regular expression and the output sub-sequence it yields is shown in Fig.3.

Figure 3: Selecting a sub-history through a regular expression The second panel of AVHL, Fig.6, is used for the speci cation of more detailed search conditions and to mark snapshots by symbols such as A, B, C, in Fig.3. The user is allowed to create as many such panels as s/he wants. The syntax of this panel is as follows.

Symbol = fA, B, C,... g

be an alphabet of some symbols. The syntax of regular expressions E is de ned as E ::= `"' j Symbol j E `'E j E `j'E j E `3 ' j `('E `)'

" is a denotation of an empty string, `' is a concatenation (as usual, we do not write it in expressions), `j' is an alternative, `3 ' is an iteration. For our purposes it was convenient to use `.' as an abbreviation of Symbol 3 ; i.e., the dot denotes any string in this alphabet. Assume snapshots from the current history are marked by a symbol from the alphabet Symbol and several snapshots can be marked by the same symbol. Then we can use a regular expressions to identify a sub-sequence of snapshots which matches a string determined by the regular expression. For example, assume that in the history

the snapshots are marked by correspondingly. A regular expression BA selects the sub-sequence of snapshots

; a regular expression C. selects the sub-sequence

(it must start from C, and then the dot accepts every string of symbols), and the regular expression (AC )3 selects the subsequence

(because (AC )3 produces all strings AC, ACAC, ACACAC, ... ). For implementation we reasonably generalized this idea, assuming that a regular expression is converted into a non-deterministic automaton (being simultaneously in several states), which changes its

::= symbol [class ] [message ] [name ] [owner ] [project ] [condition ] The is obligatory for the panel; it is used for marking snapshots that satisfy conditions determined in it. The conditions can specify:

. : name of a class of the object which received a message;

. . . . . 7

: message name; : receiver object's name; : receiver object's owner; : receiver object's project; : arbitrary condition written as a Smalltalk block expression.

then the message associated with this snapshot is sent. All snapshots of the current history are shown in the sub-window at the window bottom. Each snapshot shows a message sent by a user's operation in a format similar to Smalltalk message sending expressions. Speci cations of objects are enclosed in parentheses. A triplet such as `3 4thFloor Space' presents the identi er of an object, the object's class name, and the object's name. The display can also show a primitive object such as `Integer' or `Point' instead of the triplet. By scrolling the sub-window and selecting a snapshot, the current snapshot number is automatically set. By pressing the button `>' the current snapshot is replayed, by restoring the display according to the Aold component of the snapshot and then, sending the message associated with it. By pressing the button `