Integrating Coordination Features in PVM 1 ... - Semantic Scholar

Integrating Coordination Features in PVM O. Krone, M. Aguilar, B. Hirsbrunner

Institut d'Informatique, Universite de Fribourg Fribourg, Switzerland [email protected]

V. S. Sunderam

Computer Science Department Emory University, Atlanta, GA, USA

Abstract

[email protected]

In this paper we focus on coordination for parallel systems controlled by the PVM software package. We describe several extensions which allow the use of extended coordination features in PVM. The extensions are designed to support coordination metaphors like generative communication or client server programming. They t into the general philosophy of PVM by adopting an approach to coordination that retains the principles of coordination, while presenting a more pragmatic and traditional interface to the user.

1 Introduction In the area for parallel computing, the aspect of coordination becomes more and more important. This importance has recently been re ected with the development of coordination models and corresponding coordination languages like [Gel89, FA90, JAP93, BH94, JPB90] Although these models deserve a lot of attention from a theoretical point of view, their implementation bears to some performance problems. On the other hand, PVM is an ecient system which allows to program distributed applications using the standard message passing paradigm. Unfortunately PVM lacks some classical coordination features like generative communication or client server programming. In this paper we are bridging this gap by enhancing PVM with a set of primitives which allow the use of classical coordination features while preserving the eciency and programming model of PVM. This paper is organized as follows: in Section 2 we focus on coordination in general, followed by Section 3 which introduces the extensions for PVM. Section 4 presents a formal description of the interaction of the new PVM primitives with a new system part, called the name space. Three examples are sketched in Section 5, and Section 6 gives an overview of related work and concludes with some nal remarks and an outlook on future work.

2 Coordination Coordination has recently become a scienti c discipline for its own and is described for example as "managing dependencies among independent entities\ [TWM94]. This can be realized using a coordination language [CG92] to express and describe the relationships between the active entities running in parallel.

2.1 Coordination Models and Languages

How can a coordination language be characterized? "A coordination language is the linguistic embodiment of a coordination model\ [CG92]. Coordination has to be treated orthogonally to computation which means that you can not use a coordination language without a computation language. In general coordination will be needed for the following fundamental tasks of a distributed system: 1

C-Space

C-Space C-Space

Sender must know receiver

Central Mail Server

Sender/Receiver can be anonymous Interaction with several C-Spaces

Coupled Communication

Anonymous Communication Blackboard Model, centralized C-Space

Uncoupled Communication

Figure 1: Coupled communication and uncoupled communication

Delegation of requests: Services etc. (Client/Server Model) Speci cation of temporal behavior: Scheduling aspects Causal Dependencies: Description of the possible interactions between processes To ful ll these typical coordination tasks a general coordination model has to be composed of four components: (1) coordination entities as the processes or agents running in parallel which are subject of coordination, (2) a coordination media: the actual space where coordination takes place, (3) coordination laws to specify interdependencies between the active entities [PC94], and nally (4) a set of coordination tools. We can clarify the problematic by specifying what kind of knowledge a process or active entity has to have. It should have knowledge about:  with which other process to communicate with  how to establish a communication channel with other processes  what protocol should be used to \make" a dialog  what language has to be used to exchange data  how can a message from another coordination world be interpreted  how to handle inconsistent, local views of the system It is one of the main tasks of the coordination model to specify these items. To achieve coordination some sort of communication has to be established between active entities. One should distinguish two dierent approaches of communication metaphors: coupled communication and uncoupled communication, they will be discussed in the following section.

2.2 Coupled Communication versus Uncoupled Communication

Using communication as a method to achieve coordination implies some sort of exchange of information. This exchange of information can either be realized by means of speci c message passing routines (coupled communication), like in [Hoa78, Sun90] or by the introduction of a blackboard system (uncoupled communication) where information will be stored and hence made visible not only for a speci c destine but also for other entities. Information becomes \public", compared to message passing systems, (where it is \private") and all active entities which have access to this blackboard system can eventually read this information. Uncoupled communication or generative communication [Gel85] has several advantages compared to message passing systems. First of all, the information becomes a rst class entity, therefore an object to reason about. Furthermore information can become persistent, allowing several agents to consume the information even if the originator of the messages does not exist anymore. Figure 2

1 illustrates pure message passing systems and generative communication. The arrows denote the possible interaction ranging from point to point interaction over a centralized mail delivery protocol to totally uncoupled communication using a Coordination-Space (C-Space) similar to Linda's tuple space.

3 Advanced PVM Coordination Features In this section we describe the extensions we propose for PVM towards some \higher" programming model than pure message passing. In order to achieve this goal, we de ne (1) a global name space for processes, services and messages and (2) an instance called \Post" which is responsible for high level message passing and nally (3) abstractions to support the client-server programming model.

3.1 Global Name Space

We introduce a global name-service in PVM. New primitives are provided to register and unregister names in this name space. Names are used to identify either processes, messages or services. The scope of names is limited to the current parallel virtual machine de ned by PVM.

Processes

Processes can register and unregister to and from the global name space of PVM. By registering, the user can specify a symbolic name by which the processes will be identi ed in the future. Names can be transferred to other processes so that they can communicate with formerly unknown processes. The process of registration is dynamic, therefore allowing a dynamic change of the con guration of the parallel machine represented in the name space. Using symbolic names has the advantage that process identi cation can be done runtime and application independent.

Messages

The extension for messages is two folded: (1) messages can be identi ed by a symbolic name and (2) the semantics of messages will be extended towards uncoupled communication by allowing new message types like persistent messages (see Section 3.2). By giving messages a name we considerably change the standard semantics for messages in PVM. However, there is a problem with the naming of messages as far as the reuse of a message names is concerned. If a message name can only be reused after the destination has performed the corresponding receive, a sender process could be blocked in loop if it uses the same message name for the second send. There are two solutions for this problem: either the sender uses every time a fresh name for each message sent in the loop or we de ne a stream semantics on named messages. Streams have the advantage that they are a well known programming model. Process receive messages in the stream in a FIFO order.

Services

Adopting a client server model for PVM allows to combine two widely used coordination paradigms namely message passing and client-server programming in one programming tool. We do not present a classical synchronous remote procedure call interface known from for example [Sch93] but a combination of service invocation and service request calls combined with classical message passing. Again, we use the global name space as a repository for the storage of symbolic service identi ers. Processes providing a service have to register a service name in this name space. Processes requesting a service can get the service by addressing a service name and process name in the name space.

3

Process Name

Message Name

Blackboard

Process and Message Name

Figure 2: Postal Mail Deliver Service

3.2 Advanced Message Passing: Postal Mail Delivery

Using a centralized postal mail delivery system [MA94] for message passing allows for several extensions with regard to advanced message passing. Message can now be identi ed { like processes or services { by a logical name. The sending of message is not restricted to point to point communication anymore, but allows several modes:  Persistent: Message will be kept by the mail delivery server and can be accessed by anybody who knows the symbolic name of the message  Hold: Hold message a certain amount of time, after this the message expired and will be removed from the name space.  Instant: Send message to destination, do not keep it (old PVM standard mode)  Multiple-Copies: Keep multiple copies of the message; allows several but determined processes to receive the message Combinations of the modes are allowed, e.g., Multiple-Copies and Hold. Processes can retrieve information by either specifying the sender and the message name, only the message name, only the sender name, or nally by a wild card. The dierent modes are denoted by the dierent areas in Figure 2. The area \Blackboard" serves as a well-known communication area. Every process has per default access to this area. The area is used for the distribution of information to an unknown ordinance. Beside the extensions with regard to message passing, a postal mail delivery system has a couple of other advantages:  it supports migration of processes, because it maps logical site independent process names to physical site dependent process identi ers  it hides the physical site of a process from a user  it allows for highly portable and scalable algorithms since they do not depend on certain physical communication structures of the target machine. Note the Post can be seen as a system server which oers system services to send and receive messages using a specialized protocol. E.g. the Post could provide services for reliable communication or atomic broadcasts. To summarize, the extended control instance of PVM consist of an advanced message passing server which allows to treat message in a high-level way. Especially the possibility to keep message in the server, bridges the gap between coupled and uncoupled communication.

4

3.3 Client Server Programming

Basically client-server programming will be made possible by introducing two new primitives pvm bind svc and pvm req svc. The former primitive is used at server side to bind a service to a service name, whereas the latter primitive is used at client side to invoke the service. In order to request a service the user calls pvm req svc with the speci cation of the requested service, e.g., parameters and service identi cation. The call is nonblocking, therefore the process can continue to compute while the service provider is working. To nally receive the result a receive is used where the message name corresponds to the service name of the requested service, see Figure 3. To retain the general philosophy of PVM parameters have to be packed and unpacked. char date[32]; pvm initsend(); /* No parameters for this service */ pvm req svc("Time-Server", "time-of-day", pvm getsbuf()); ....

/* Do some useful work */

/* Wait for result, note message name == service name */ pvm crecv("Time-server", "time-of-day"); pvm upkstr(date);

Figure 3: Client Server programming with PVM

3.4 Primitives at a Glance

An overview of the proposed new primitives is given in Table 2. The primitives will be described brie y:

pvm register(char *proc-name, char *s1, ...,char *sn, (char *)NULL): Register proc-name to the general global name space of PVM. After this call the services provided by this process can be identi ed using s1--sn. A process can, but has not to provide services for other processes. This is done using the optional parameters s1--sn. The services have to be bound using pvm bind svc. pvm unregister(char *proc-name, char *s1, ...,char *sn, (char *)NULL): Unregister proc-name of the general global name space of PVM. After this call the process can not be identi ed anymore using proc-name. A process can partially unregister services it previously provided. This is done using the optional parameters s1--sn. pvm bind tid(char *proc-name, int tid):

This function is used to bind a PVM task identi er to a symbolic name which must be unique for the whole distributed application. pvm bind tid can be called by both, the creator of the process and by the created process itself. A process can either be identi ed by its run-time dependent task identi er or by its runtime and architecture independent symbolic name proc-name.

pvm bind svc(char *svc-name, void (*f)()): Bind function f to a service identi er svc-name. The service can then be exported using pvm register. The two step approach to rst bind a service and then register it allows

for easy modi cation of services without changing the well known service name used by other processes. The identi er svc-name is process local, therefore several processes might locally bind a service to the same symbolic name. pvm bind svc has to be called by the process which provides the code for the service function f. 5

Result of receive from msg-name message msg-name from from  msg-name message msg-name from  sender from   receive any message designated for this process

name

msg-name

Table 1: Possible parameter combinations for pvm

crecv

pvm bind msg(char *msg-name, int bufid, int flags): Bind actual message buer to message identi er msg-name. flags

character of the message, e.g., persistent etc.

is used to specify the

pvm csend(char *proc-name, char *msg-name):

Send a previously bound message. The semantics of this routine is analogous to the pvm send call. A message named msg-name will be sent to the process named proc-name. The call is asynchronous, therefore computation can continue while the message is in transit. The message buer can be reused after this call.

pvm crecv(char *from, char *msg-name):

Receive a named message. The speci cation of the message to receive is summarized in Table 1 ( denotes a wild card).

pvm req svc(char *service-provider, char *service, int bufid):

Invoke a service at client side. The parameters for the service have to be packed using pvm pk* functions. The result of the service will be received using pvm crecv. To transfer the result of the service, the service provider will send a message with name service to the service invoker.

Primtive pvm pvm pvm pvm pvm pvm pvm

register(char *, char *, ...) bind tid(char *, int) bind svc(char *, void (*f)()) bind msg(char *, int, int) csend(char *,char *) crecv(char *, char *, int ) req svc(char *, char *, int)

Description Register process with its services in name space Bind symbolic name to PVM's task identi er Bind a function f to symbolic name Bind buer id to symbolic name Send message to process (symbolically) Receive message Request service from server

Table 2: New PVM primitives, not shown primitives to unregister and unbind names Figure 4 shows a process p interacting with the name space and the postal mail-delivery system. In this example the process has registered to the name space by exporting a service (\time-of-day"), process q registered as \Time-client" and requested this service, the result of the service invocation can be accessed with the message name \time-of-day".

4 Semantics of the Interaction with the Name Space This section provides a basic formal description of the interaction with the name space of the new PVM primitives. We only sketch the basic principles e.g., entering new names into the name space, a full description of the semantics of the new primitives will be subject of an upcoming paper. 6

Service

Postal Mail Delivery

Messages Process Names

send

Message Names

Process

Blackboard

Interaction

receive

Process and Message Names

register register

bind_tid time-of-day

PVM Process p

bind_msg

PVM Process

Time-Client

Time-Server

time-of-day

q

Name Space

Figure 4: Overview of PVM's new coordination features The formal description is given by using the Z notation [Dil94]. Z is a well known speci cation language based on set theory. We assume that the reader has a basic knowledge of the Z notation, where necessary we explain the basic principles of Z. We have three dierent types of names in our name space: [ProcessName ; MsgName ; SvcName ] The result of a state change in the name space can either be ok or error . Response ::= ok j error

The global name space consists of names for processes, messages, and services (ProcessName , MsgName , SvcName ). Processes (Pnames ) are identi ed by a symbolic name and its corresponding PVM task identi er and a set of service identi ers. Messages (Mnames ) are modeled using a partial function from a sequence of message buer identi ers to associated message names to the destination of the message. We choose a sequence of buer identi ers to underline the stream semantics of messages sent with the same message name. NameSpace Pnames : (ProcessName ! N ) ! fSvcName g Mnames : (seq N !7 MsgName ) ! ProcessName

For initialization all sets are empty. NameSpaceInit =b [NameSpace j Pnames = ; ^ Mnames = ;] 0

0

0

4.1 Process Names

Put a new process name into the space. Input variables are the symbolic name and the task identi er1 . Either the process name does not yet exist in the space, then add it with its services (EnterProcNameProc ) or the process name is already in the space then check that we do not use a service name twice (EnterProcNameAddSvc ). 1 In Z input variables are decorated with a ?, output variables with a !,
denotes a state change, whereas  indicates that the global state remains unaected.

7

EnterProcNameAddSvc
NameSpace tid ? : N ; p ? : ProcessName ; s ? : SvcName r ! : Response 91 t : fSvcName g j fp ? 7! tid ?g 2 dom Pnames ^ fp ? 7! tid ?g 7! t 2 Pnames ^ : (fs ?g  t )  Pnames = Pnames  f(fp ? 7! tid ?g 7! (t [ fs ?g))g Mnames = Mnames r ! = ok 0

0

Simply add the new process into the space, since it has not yet been registered in the space, we can safely add it without any restrictions. EnterProcNameProc
NameSpace tid ? : N ; p ? : ProcessName ; s ? : SvcName r ! : Response { New process, p? and tid? unused { fp ? 7! tid ?g 2 = dom Pnames Pnames = Pnames [ f(fp ? 7! tid ?g 7! fs ?g)g Mnames = Mnames r ! = ok 0

0

The following schema speci es the error situation for entering process names into the names space. The process name is already registered in the space and the process uses a service name twice. EnterProcNameError NameSpace tid ? : N ; p ? : ProcessName ; s ? : SvcName r ! : Response fp ? 7! tid ?g 2 dom Pnames ^ (91 t : fSvcName g j fp ? 7! tid ?g 7! t 2 Pnames ^ fs ?g  t  r ! = error ) The actual interface consists of a schema for the successful invocation and for the handling of error situations. EnterProcName =b EnterProcNameProc _ EnterProcNameAddSvc _ EnterProcNameError Basically the interaction with the name space of pvm register and pvm bind tid can now be modeled as EnterProcName .

4.2 Message Names

We distinguish between the sending and receiving of messages. Since the sending of message is a two phase operation (bind and send) we rstly have to bind the message (MsgBindName ). We use this schema only in combination with the (MsgNameSend ) schema.

Send

Bind a message buer id (bu d ) to a symbolic message name. The result of the operation is the mapping of the bu d to the message name (inout ). The output will be used as the input for 8

MsgNameSend 2 . MsgBindName NameSpace m ? : MsgName ; bu d ? : N inout !; x : seq N !7 MsgName (let x == (dom Mnames )  { Message name m? already used { 9 t : seq N j t  fbu d ?g = ; ^ ft 7! m ?g 2 dom Mnames _ { New message name { (x B fm ?g = ; ^ t = hi)  inout ! = ft a hbu d ?i 7! m ?g)

For the actual sending, we have to check whether the destination process exists. The input of this schema is the output of MsgBindName . MsgNameSend
NameSpace m ? : MsgName ; dest ? : ProcessName inout ? : seq N !7 MsgName ; r ! : Response (9 i : N j fdest ? 7! i g 2 dom Pnames  Mnames = Mnames  finout ? 7! dest ?g) Pnames = Pnames r ! = ok 0

0

We model two error conditions, one for the binding of messages, another for sending of messages. A bind error occurs if an identi er will be mapped twice to the same name. MsgBindError NameSpace m ? : MsgName ; bu d ? : N r ! : Response { Buer identi er has been mapped already to a name { 9 t : seq N j t  fbu d ?g 6= ; ^ ft 7! m ?g 2 dom Mnames  r ! = error

The following schema describes an error situation for sending of a previously bound message. This is the case if the destination process does not exist or if the message name is unknown. MsgNameSendError NameSpace m ? : MsgName ; dest ? : ProcessName r ! : Response { Unknown destination, or unknown message { 9 i : N ; s : seq N j fdest ? 7! i g 2 = dom Pnames _ fs 7! m ?g 2= dom Mnames  r ! = error

The combination of pvm bind msg and pvm csend is described with the schema SendMsgName which takes into account some basic error situations. SendMsgNameOk is de ned using Z's t S lters all elements t which occur in the sequence S; a concatenates two sequences; B is Z's range restriction operator; comments are enclosed in 2 

9

pipe mechanism, this allows to use the output of the MsgBindName schema as the input of the MsgNameSend schema. SendMsgNameOk =b MsgBindName >>MsgNameSend 3 SendMsgName =b SendMsgNameOk _ MsgBindError _ MsgNameSendError

Receive

For the receiving of a message we have to check whether the destination has already been registered and whether the appropriate message exists in the name space. MsgNameReceive models only the reception of messages explicitly designated for the process (denoted by self ). It does not remove the message name form the name space, since we speci ed only the entering of names into the name space in this Section. MsgNameReceive NameSpace m ? : MsgName ; from ?; self : ProcessName r ! : Response { Check whether sender exists { 9 tid ; bu d : N ; s : seq N j ffrom ? 7! tid g 2 dom Pnames ^ fs 7! m ?g 7! self 2 Mnames  r ! = ok

The interaction of MsgNameReceive .

pvm crecv

with the names space is semantically described with the schema

5 Examples The following examples underline the usefulness of the introduced primitives. We brie y sketch how the extensions can be used for load balancing purposes, for a parallel I/O subsystem and for the implementation of a message passing oriented coordination language called CoLa [BH94].

5.1 Load Balancing

Because services can dynamically be registered and unregistered from the name space we can implement a load balancing scheme. Suppose we have several servers providing the same service (replication of services) on dierent physical machines. Once the load on a speci c machine becomes too high, the server simply unregisters its services from the name space therefore decreasing the load on the machine.

5.2 Parallel Input/Output

Persistent messages are perfectly suited as the base to implement a parallel I/O subsystem. If, for instance, the same disk block will be requested over and over again by a client, the le server could make this data persistent to avoid unnecessary disk accesses. This is possible even if the system is implemented using a client server model because the result of a service request will be sent as a message to the client. 3 >> denotes Z's piping mechanism which relates the output variables of the rst schema to the input variables

of the second schema.

10

5.3 CoLa

The coordination language CoLa is especially aimed at providing a universal platform for the development of applications in the eld of distributed arti cial intelligence in a massively parallel environment. Coordination between CoLa processes is based on a message passing send/receive mechanism. CoLa supports dynamic virtual process topologies which are represented by a set of processes and logical communication links between these processes. These communication links can be implemented as named message streams and the change of the topology as a special system service of the Post.

6 Conclusion

6.1 Related Work

In the area of Coordination for Open Distributed Systems Laura [Tol94] is designed for the coordination of services in open distributed systems. This is done by introducing a service-space shared by all agents which is used for requesting and providing services. Agents providing a service have to dispose a description of a service-oer, and agents requesting a service have to put a servicerequest form into the space respectively. In some sense this similar to our name space, but in our opinion it would not work for massively parallel systems because the costs to maintain a global unique service-space for all agents would be simply too high. The integration of message passing and client server programming for distributed computing in OSF's Distributed Environment has been described in [Wu93].

6.2 Summary

In this paper we proposed extensions to the PVM programming model which allow a higher degree of expressiveness to coordinate parallel applications. We extended PVM by introducing a global name space for messages, services and processes which will be used to implement non-classical message passing semantics and client server programming model. The interaction with this name space has been formalized using the speci cation language Z. The proposed extensions t into the general philosophy of PVM.

6.3 Future Work

The extensions, especially the postal mail delivery service and the name space, will be used to implement a system that allows interoperability at coordination level. The idea is to de ne several Coordination Spaces which are characterized by the coordination model used inside the space [OK95]. Inter coordination will be achieved by the abstract de nition of coordination performatives like the performatives of KQML [FF94] for message passing. The actual interpretation and transformation of a coordination performative from one coordination space to another will be realized by the introduction of a special control instance S inside the coordination space (like the facilitators in KQML). Here the global name space and the advanced message passing capabilities of PVM can help to allow e.g. coordination between a pure LINDA like system and a pure PVM like controlled system. A similar project is de ned in [Usz94] where a Interoperability Architecture is proposed which which is built around a family of distributed languages. A specialized component called Interoperability Server is used as the interface to the dierent environments. Our plans for the future include an implementation of the extensions for the PVM version of the Parsytec PowerExplorer where we use a version of PVM adapted for multi threaded programming [OKH95] as the base process management and communication platform. 11

References [BH94]

M. Aguilar O. Krone B. Hirsbrunner. CoLa: A Coordination Language for Massive Parallelism. In Proceedings ACM Symposium on Principles of Distributed Computing (PODC), Los Angeles, California, August 14{17 1994. [CG92] N. Carriero and D. Gelernter. Coordination Languages and Their Signi cance. Communications of the ACM, 35(2):97{107, February 1992. [Dil94] Antoni Diller. Z An Introduction to Formal Methods. John Wiley & Sons, second edition, 1994. [FA90] I. Herman P. Spilling F. Arbab. An Overview of Manifold and its Implementation. Technical report, CWI, Department of Interactive Systems, Amsterdam, 1990. [FF94] T. Finin and R. Fritzson. KQML - A Language and Protocol for Knowledge and Information Exchange. Technical report, University of Maryland, Baltimore, MD 21228, 1994. [Gel85] David Gelernter. Generative Communication in Linda. ACM Transactions on Programming Languages and Systems, 7(1):80{112, 1985. [Gel89] N. Carriero D. Gelernter. Linda in Context. Communications of the ACM, 32(4):444{ 458, 1989. [Hoa78] C.A.R. Hoare. Communication Sequential Processes. Communications of the ACM, 21(8), August 1978. [JAP93] P. Ciancarini J.M. Andreoli and R. Pareschi. Interaction Abstract Machines. In P. Wegner G. Agha and A. Yonezawa, editors, Reserach Directions in Concurrent Object Oriented Programming. MIT Press, Cambridge Mass., 1993. [JPB90] Le Metayer J. P. Ban^atre. The Gamma Model and its discipline of Programming. Science of Computer Programming, 15:55{77, 1990. [MA94] Beat Hirsbrunner Marc Aguilar. Post: A New Postal Mail Delivery Model. In K.M. Decker R.M. Rehmann, editor, Programming Environments for Massively Parallel Distributed Systems, pages 231{237. IFIP WG 10.3, April 1994. [OK95] M. Aguilar O. Krone. Bridging the Gap: A Generic Distributed Hierarchical Coordination Model for Massively Parallel Systems. In Proceedings of the '95 SIPAR-Workshop on Parallel and Distributed Computing, Biel, Switzerland, October 1995. [OKH95] M. Aguilar O. Krone and B. Hirsbrunner. Pt-pvm: Using PVM in a multi-threaded environment. In 2nd PVM European Users' Group Meeting, Lyon, September 13{15 1995. [PC94] Thilo Kielmann Paolo Ciancarini. Coordination Requirements for Open Distributed Systems. -Release, 1994. [Sch93] M. Bever K. Geihs L. Heuser M. Mulhauser A. Schill. Distributed Systems, OSF DCE and Beyond. In A. Schill, editor, International DCE Workshop, number 731 in LNCS, Karlsruhe, October 7{8 1993. Springer Verlag. [Sun90] V.S. Sunderam. PVM: A framework for parallel distributed computing. Concurrency: Practice and Experience, 2(4):315{339, December 1990. [Tol94] Robert Tolksdorf. Coordination in Distributed Systems. PhD thesis, Technische Universitaet Berlin, 1994. 12

[TWM94] K. Crowston T. W. Malone. The Interdisciplinary Study of Coordination. ACM Computing Survey, 26(1):87{119, March 1994. [Usz94] K. Zielinski G. Czajkowski A. Uszok. Coordination through Interoperability in Distributed Processing. In ECCOP Workshop on Models and Languages for Coordination of Parallelism and Distribution, Berlin, 1994. Springer Verlag. [Wu93] Y.H. Wei C. Wu. Integrating RPC and Message Passing for Distributed Programming. In Schill [Sch93].

13