A session management service controls the underlying services application sharing and telepointing, annotation and sketching, and audio/video conferencing.
A Framework for Synchronous Tele-Cooperation Thomas Schmidt Siemens AG Corporate R&D Division Stuhlsatzenhausweg 3 D-66123 Saarbrücken
Jean Schweitzer Siemens AG Corporate R&D Division Stuhlsatzenhausweg 3 D-66123 Saarbrücken
Michael Weber Distributed Systems Dept. University of Ulm Oberer Eselsberg D-89069 Ulm
Abstract. Computers are ever more thought of to support cooperative work over distances. Thus tele-cooperation tools and environments are needed which bring the meeting and joint-working onto everyone’s desktop. The following paper introduces a framework for such tele-cooperation. The required cooperation-awareness is achieved by a session model around which the building blocks of the entire service are configured. A session management service controls the underlying services application sharing and telepointing, annotation and sketching, and audio/video conferencing.
1 Introduction Information processing and data transmission have made the world smaller since the first images were drawn on cave walls. Today, the technical environment for high speed data interchange enables humans around the world to interact with each other. While a face-to-face meeting involves all human senses and all means of expressing, currently the most technically based interaction is restricted to a few ways of communication only. But the rapid technical development enables the integration of all important means of communication among humans into a computer based environment. The face-to-face meeting situation is mirrored by so-called “synchronous tele-cooperation tools” with the users sitting in their rooms in front of their computers. The applications realizing these tools can be divided into cooperation-aware applications especially designed to be used within groups, and into coordinating tools allowing existing cooperation-unaware single-user applications to be run in a group context [1], [2]. These coordination tools are necessarily realized as cooperation-aware applications. Thus, looking from a higher level of abstraction, all tools supporting synchronous group work can be seen as cooperation-aware. However, the inflation in the development of single features makes it necessary to structure the whole bunch of currently available techniques. This paper presents a framework for synchronous tele-cooperation which enables synchronous joint working in
heterogeneous workstation environments comprising different hardware platforms. Specifically, SNI RW420 workstations with IRIX 5.x, SNI’s Pentium PC with SINIX 5.x and Sun Sparcstations with SunOS/Solaris will be supported. Most of the experiences which form the base of this paper are gained by R&D projects carried out in the frame of DELTA, RACE and BERKOM II, especially ARAMIS [3], CIO [4], ECOLE [5] and BERKOM-MMC [6].
2 The Session Model Working in a group context requires the awareness of the ongoing cooperation on two main levels, the system level and the user interface level. This paper focuses on the system level. The system has to possess and process information concerning the involved machines and users. This information may comprise dedicated user roles. The role may be associated with rights like read or read/write access on data objects. The user relevant information has to be extracted from the system and presented on the user level by an appropriate interface realizing the group-awareness for the user, e.g. different user inputs may be mapped onto different visualizations of cursors or modified objects. The overall frame within cooperative work takes place is described by the session model shown in Fig. 1. This model is derived from the application sharing specific model [7] by extending it to the general class of cooperation-aware applications. The model describes cooperation in an abstract way and serves as the basis for the design and implementation of the components on the system level and the user interface level. The left-hand side of the diagram is based on an abstraction hierarchy, whereas a corresponding hierarchy is reflected in terms of user participation on the right-hand side. The abstraction hierarchy consists of three levels: The top-level abstraction is the world, a virtual space in which sessions take place. The potential participants of sessions are users. The users attending a session are called participants. One user out of these, the chairman, leads the session. This chairman usually is the initiator of the session and he decides on the session participation policy. According to that policy, users can freely join, they can request the chairman to be allowed to join, or they can be invited to join the session. The session forms the frame synchronous cooperative work is embedded in. This frame, the central component of synchronous tele-cooperation, is realized by the Session Management Service. A session comprises several cooperation-aware services reflecting the resources and tools also brought into action in ordinary meetings commonly used by the participants. The following services are administrated by the Session Management:
World
User
1
1 1
mc Session 1
Chair
1
1 1
Floor Holder c
mc Service
mc m
1
m
cooperation aware Services Multimedia Application Sharing & Telepointing
1
c
Participant 1
mc 1
1
Caller
Annotation & Sketching A/V Conferencing
1: related to 1 entity c: related to 0 or 1 entity m: related to 1..m entities mc: related to 0..m entities
Fig. 1. Session Model
-
Multimedia Application Sharing performing the common synchronous use of single-user applications together with Telepointing performing joint pointing,
-
Annotation and Sketching providing whiteboard layers,
-
A/V conferencing providing audio/video communication between the participants.
Moreover the Session Management Service is open to the integration of further cooperation-aware services. Services can be involved in a session depending on the session policy only by the chairman or any other session participant, who is referred to caller. Some services may require a sequential user input. The assignment of the right to provide this input is subject to floor control. At any one time there is one participant at the most holding the floor.
3 Session Management In the context of a session users may be invited by the session chairman, or they may request to join or leave an existing session, the chairman can assign and revoke the floor, several services may be involved by the participants. The Session Management Service (SMS) coordinates all these operations necessary from the start of a session until its end. To perform these tasks the SMS provides a set of generic operations which are grouped into core session and participant management, floor control and service management. The SMS also includes the Management Information Base (MIB) which holds session-static as well as session-dynamic information (Fig. 2.). User Interface
User Interface
User Interface
Session Management Service Telecooperation System
MIB
MM-Application Sharing and Telepointing
Annotation and Sketching
A/V Conferencing
Other Services
Fig. 2. Session Management and controlled services
3.1 Core Session and Participant Management The SMS provides the following operations for session and user management: open:
An empty session is created. The issuer becomes the chairman and decides on the participation policy.
close:
A session is terminated. Only the chairman can close a session.
invite:
The chairman invites a user to join the session. The invited user may join the session.
join:
A user requests to join the session. It depends on the policy whether he joins straight away or whether the session chairman decides to allow joining.
drop:
A participant is removed from the session by the chairman.
leave:
A participant leaves the session.
shift_chair: The role of the chairman is shifted to another session participant. The
operation can be issued by the session chairman only. set_policy:
The chairman sets the current session policy which lays down the rules of joining a session.
3.2 Floor Control In a session there have to be means of assigning the floor to one participant at a time. This user is designated with the right to provide input to the application currently shared. We distinguish between floor policies and floor mechanisms [8]. Floor policies describe how participants request the floor, how it is assigned and released. Floor mechanisms are the low-level means to implement the floor policies. Different floor control policies are supported: -
The session chairman assigns the floor to a session contributor which returns it to the chairman after use.
-
The current floor holder passes the floor to another session contributor who requested it.
-
Floor requests are queued. As soon as the current floor holder releases the floor, it is assigned to the participant heading the request queue.
At a time each session follows a specific floor policy which might be changed by the chairman. The Floor Control Service is embedded into the SMS. Four primitives are required as floor mechanisms: request:
A participant requests to get the floor. Depending on the current policy the floor is granted or not.
release:
The current floor holder releases the floor.
assign:
The floor is assigned to a participant.
revoke:
The floor is taken from the current floor holder.
The information who is currently holding the floor is passed on from the SMS to underlying services.
3.3 Service Management The SMS controls the other involved services. Such control does not only comprise the start and termination of the respective service, but also the changing of the number of session participants and the change of the floor holder. open:
A participant requests to start a service.
close:
A service is terminated.
add_user:
The session management requests the service to expand the list of participants by a new one.
remove_user:
The session management requests the service to shorten the list of participants by the leaving one.
assign_floor:
The token is assigned to a dedicated participant. This allows him to provide input.
3.4 Management Information Base The MIB holds session-static and session-dynamic information. Static information relates to users, such as their addresses and names which might be stored exploiting an existing directory service. Dynamic information relates to session-specific data, such as the current set of participants and their roles, or the current floor holder. The MIB can be accessed by all services to get and retrieve relevant information.Thus the MIB offers a facility for inter-service data exchange. The following operations are provided to enter and update information stored in the MIB: create/destroy:
A data object is created/destroyed in the MIB.
change:
Attributes of the data object are changed.
un/register:
A service is un/registered by the MIB in order to be informed about changes of a data object
indicate:
A service is informed about changes of a data object
4 Multimedia Application Sharing and Telepointing To share an application means to provide its output to all session participants and to grant the input right to one of them at a time. The basic idea of application sharing is to intercept the window protocol traffic between the application and its terminal server. The Multimedia Application Sharing Service allows the running of single-user cooperation-unaware multimedia applications in a group context. The Multimedia Application Sharing Service can be divided into the Application Sharing Component [9] and the Audio/Video Sharing Component. Application output is distributed to all session participants by multiplexing the output stream (see Fig. 3.). The application executes on one machine and its user interface is displayed onto several machines. I.e. the requests from the application to the window server are multiplexed and sent to the remote window servers as well. The remote window servers then produce the same graphical output on their terminals. Only input from the current floor holder is passed to the application. The input of other participants is filtered to maintain the single-user behaviour of the application being shared. The Telepointing Service provides each user with a set of globally visible pointers. As soon as a telepointer is moved into a shared window it becomes visible for all other participants. Therefore the Telepointing Service tracks the movements of locally used telepointers and distributes these movements to the remote sites. Coordinate transformations have to be performed to let remote telepointers appear in the same position in
current floor holder Window Server1 Shared Application
ASS Window Servern User Input Application Output ASS - Application Sharing Service
Fig. 3. Application Sharing by Distributing the Application’s User Interface
the shared window as on the source site. An adjustable update frequency on remote screens avoids telepointers turning into rubbers when being moved. Telepointers also have to be synchronized with related audio streams coming from the Audio/Video Conferencing. This can be achieved by delaying the presentation time of telepointers until the corresponding audio is received, or by dropping intermediate positions of the telepointer when audio is faster.
5 Annotation and Sketching The Annotation and Sketching Service is a generic annotation utility, to provide sophisticated functionality for annotating shared documents. Additional to the joint editing facilities the Annotation and Sketching Service provides comfortable means to annotate arbitrary applications, especially documents with text and graphics. In order to provide sophisticated annotation facilities and keep the original application unchanged, the working sheets of the Annotation and Sketching Service are transparent. A user of the system has the impression, as if the output of his document is covered by a transparency containing his annotations. Using this technique, it is possible to see usual output of the application and the annotations on the 'transparency' at the same time (see Fig. 4.). The annotations are not inserted in a shared document, but are visible to all participants of a conference. The document can be edited in the usual way, after switching from the annotation mode to normal input mode. In order to provide a homogeneous integration and to keep the user interface as simple as possible, only the most important functions are integrated into the Session Management’s user interface. The current version of the Annotation and Sketching Service does not support semantically linked annotations. Annotations are called semantically linked, if they are adap-
Annotation Service
ShapeMask
Annotation Service sh. Appl.
Shared Application Fig. 4. Annotation service using X-Shape extension
ted to changes made in the original document automatically. For example a user has marked one sentence in a paragraph of a document under review. If additional text is inserted in this paragraph, the paragraph is moved to another location on the page or even the page has been changed, the annotation will be visible at a wrong place. It is possible to adjust the annotations automatically, if such situations occur. Future versions of the Annotation and Sketching Service will support semantically linked annotations.
6 A/V Conferencing Cooperative work requires support by audiovisual communication. Therefore the system is enhanced by the Audio/Video Conferencing Service. The Audio/Video Conferencing Service is based on an A/V communication service which allows to multiplex and distribute audio and video sources to remote participants (see Fig. 5.). It also includes audio and video processing capabilities such as compressing and decompressing A/V data, or mixing audio channels. It can be used for a separate video conferencing session or in conjunction with other services to enhance communication. For audio and video data transport services providing isochronous delivery between the
participants are required. Thus a negotiation of suitable Quality-of-Service parameters with such a transport system has to be performed. AV Conferencing Service
AV Communication Service
Audio System
Window System
Transport System control flow to/from AV conference service AV conference peer protocol
audio/video data flow
Fig. 5. AV Conference Service Architecture
7 Conclusions and Future Work The paper introduces the session as a framework within synchronous cooperative work takes place. In the context of a session users may be invited by the session chairman, or they may request to join or leave an existing session, the chairman can assign and revoke the floor, or participants may involve several services. The framework is realized by the Session Management Service (SMS). The SMS provides a set of generic operations to coordinate all activities necessary from the start of a session until its end. These operations are grouped into core session (open/close a session) and participant management (e. g. invite a participant), floor control (e.g. assign/revoke the floor) and service management (e.g. start/terminate a service). The SMS also includes the Management Information Base which holds session-static (e.g. users addresses) as well as session-dynamic information (e.g. participant role). The MIB can also be accessed by all services to store and retrieve relevant information. In the next phase a conference level will be installed above the session level. The conference service maintains short- and long-term conferences and deals with the subjects of storing and recovering session contexts, address management which includes group support by a name service, a cooperative scheduler and voting support. Furthermore, group-awareness will be implanted into the data level, e.g. by a distributed virtual group-filesystem, document managers and databases, and into the presentation level, i.e. into the window manager.
Fig. 6. Screenshot of the telecooperation system
8 Acknowledgements We thank all our internal and external colleagues being involved in the projects BERKOM II, CIO, ARAMIS and ECOLE for valuable contributions and discussions.
9 References [1]
Reinhard, W., Schweitzer, J., Völksen, G. Weber, M.:“Ways to a CSCW Framework: Concepts and Architectures”; IEEE Computer, May 1994.
[2]
Cronjäger,S., Reinhard, W. , Schweitzer, J.: “Functional Components for Multimedia Services”. Proceedings, International Conference on Communications (ICC ‘93). Geneva, 1993
[3]
Armbrüster, H., Humer-Hager, T., Pütz, K. J.: “Pilotprojekte mit ATM-Netzen”; telcom report, Siemens AG, März/April 1994.
[4]
Dermler, G., Gutekunst, T., Ostrowski, E., Pires, N., Schmidt, T., Weber, M., Wolf, H.: “JVTOS - A Multimedia Telecooperation Service Bridging Heterogeneous Platforms”; Proceedings, International Conference on Broadband Islands, Hamburg, 1994.
[5]
Dietel, C. et al: “A multimedia environment for distributed learning”. ISDN applications in education and training, IEEE, London, 1994.
[6]
Herrtwich, R. et al.: “The BERKOM MMC Service”. Proceedings ACM Multimedia ‘93 Conference, Anaheim, 1993.
[7]
Gutekunst, T., Schmidt, T., Schulze, G., Schweitzer, J., Weber, M.: “A Distributed Multimedia Joint Viewing and Tele-Operation Service for Heterogeneous Workstation Environments”; Proceedings, GI/ITG Workshop on Distributed Multimedia Systems, pp. 145 - 159. Edited by W. Effelsberg, K. Rothermel. Stuttgart, 1993.
[8]
Crowley, T., Milazzo, P., Baker, E., Forsdick, H., Tomlinson, R.: “MMConf: An Infrastructure for Building Shared Multimedia Applications”. Proceedings CSCW ‘90. Los Angeles, 1990.
[9]
Minenko, W.: “Transparentes Application-Sharing unter X Window”. in german; Internal Report, DFKI, Siemens AG, 1993.
[10]
Dermler, G., Gutekunst, T., Ostroswski, E., Ruge, F.: “Sharing Audio/Video Applications among Heterogeneous Platforms”; 5th IEEE Comsoc Workshop Multimedia ‘94, Kyoto, Japan, May 1994
