Automating Type-safe RPC - CiteSeerX

Automating Type-safe RPC Miguel Mira da Silva [email protected]

Dept of Computing Science University of Glasgow Glasgow G12 8QQ Scotland Abstract It is generally accepted that the possibility of a procedure in one program to call a procedure in another program is a useful paradigm for developing distributed applications. However, in a distributed environment, programs should be allowed to be built and then changed in an independent manner, thus allowing inconsistent types to be declared in the expected and the actual remote procedure interface. If one cannot guarantee type consistency, the program executing the remote procedure would be vulnerable to invalid call requests. The paper describes how a persistent programming language can be used to implement an RPC mechanism that automatically supports type-safety. It is typesafe since the expected interface is checked against the actual remote procedure interface before a call is attempted and automatic because user intervention is only that needed otherwise (that is, to call a local procedure).

Contents

1 Introduction 2 Design Issues

2.1 The RPC Mechanism 2.2 Automatic Type-safety 2.3 Server and Procedure Binding

1 2

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

3 Persistent Programming Languages 3.1 Napier88

5

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

4 Programming Interface 4.1 4.2 4.3 4.4

Generating Client and Server Stubs Using the Generated Stubs Example of Generating and Using Client Stubs Example of Generating and Using Server Stubs

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

5 Argument Semantics

: : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

6 Binding and Type-checking

Sessions Capabilities Binding Service Support for Incremental Change

13 13 15

15

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

7 Transport Protocol

7.1 Client Stubs Manager 7.2 Server Stubs Manager

7 8 9 11

11

: : : : : : : : : : : : : : : : : : : : : : : :

6.1 6.2 6.3 6.4

6

7

: : : : : : : : : : : : : : : : : : : :

5.1 Argument Passing by Value 5.2 Packing and Unpacking Arguments 5.3 Shared Values

2 3 4

15 16 17 19

20 : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

8 Implementation 9 Summary and Future Work Acknowledgements References

21 21

21 22 23 24

1 Introduction A distributed application is usually composed of a number of programs communicating across a network. These programs are executed on machines with potentially incompatible architectures, and often written in dierent programming languages. Remote procedure call (or RPC, for short) is a paradigm for providing high-level communication between these programs [BN84]. Using an RPC mechanism, a distributed application can have a procedure in a program calling a procedure in another (remote) program. In a distributed environment programs should be allowed to be independently built and then continuously changed. This independence permits programmers to be concerned with their own programs, and only need to understand the external interface to other programs. In this context, one cannot easily guarantee that the types of the expected arguments always match those of the actual remote procedure. Type-checking must be enforced in order to prevent programmers from making (even more) errors, particularly those which might endanger the integrity of the whole system (such as using an integer as a reference). The possibility of type mismatches is especially relevant in a distributed application, as there are usually some programs providing a service to many other programs which consume that service. If a failure in a service consumer has consequences in a service provider, the failure also aects other consumers, not directly related to the faulty one. This happens because when a consumer needs to use a service from a provider, this one could have crashed as a result of an error which happened in another consumer of the same provider. This paper describes the design and implementation of an RPC mechanism that automatically guarantees strong type-safety. The mechanism is type-safe since calling a remote procedure is only possible after the caller program has demonstrated it knows the correct number, order and types of the arguments of the remote procedure. The type equivalence test is strong because the mechanism checks the actual type structure, in opposition to user-de ned names which can lead to errors if used across programs. Moreover, type-safety is automatic because it is achieved without requiring any user intervention in excess of that already needed for calling a procedure in a type-safe language. The research work described in this paper is dierent from other work because the RPC mechanism takes advantage of many advanced features found in Napier88 [MBCD89], a persistent programming language; it does not necessarily oer more than other commercial implementations or research prototypes. Persistence is important for building an automatic type-safe RPC mechanism mainly because it enormously eases the task of implementing the system. Napier88 provides several interesting facilities apart from orthogonal persistence: rich type system 1

including higher-order procedures, parametric and abstract data types; threads and semaphores; a type-safe callable compiler; and a library of bulk data types. These facilities, together with their clean integration, oer an excellent development environment for large applications manipulating complex inter-related data. Throughout the paper it is shown how these facilities oered by Napier88 helped in implementing a type-safe RPC mechanism. The rest of the paper is structured as follows. Section 2 motivates the reader for automatic type-safe RPC and the binding problem. Section 3 introduces persistent programming languages in general and Napier88 in particular. The next four sections are related to the RPC design and implementation: the interface to the programmer is presented in section 4, section 5 discusses arguments semantics, section 6 describes binding and type-checking and section 7 the transport protocol. A short implementation report is presented in section 8. Section 9 concludes with a summary and planned future work.

2 Design Issues 2.1 The RPC Mechanism The RPC mechanism is usually described in terms of two main modules ( gure 1): a client (the caller program) and a server (the callee program). The client normally executes on one machine and the server on another, but in the mechanism nothing prevents the client and the server to be executed on the same machine. For each side there is a further partition into three main layers:

user program where user procedures are resident stub procedures for packing and unpacking values into and from network messages transport protocol for exchanging these messages across the network When a client user program issues a call to a remote procedure in a server user program, it actually executes a normal procedure call to a local stub procedure mirroring the remote procedure interface at the server side. The code for this client stub is generated from information about the remote procedure name and types of its arguments. The client stub packs the arguments into a message (byte array with some known interpretation) and calls the transport protocol level that sends the message to the server as a packet (uninterpreted byte array), using a low-level communication protocol (sockets, for example). Then the client stub blocks, waiting for the result message. 2

Server

Client Procedure Call Procedures Message Stubs Packet Transport Network

Figure 1: The RPC Mechanism On the server side, the transport protocol level is listening to the network, waiting for incoming packets. When one arrives, it builds a message and calls the appropriate server stub passing the message as its single argument. The server stub unpacks the arguments from the received message, calls the remote procedure1, packs the result into a new message and passes it to the transport protocol, which sends it back to the client. On the client side, the transport protocol is waiting for the result packet; when it arrives, a message is built and passed to the client stub, which unpacks the result value and nally returns it to the client program.

2.2 Automatic Type-safety The RPC mechanism as described is not type-safe. The client and the server are usually independently developed and changed, inevitably leading to inconsistencies between the types of the arguments at the remote procedure and its local representative, the stub. An experiment using Sun/RPC [Sun90] was attempted. A procedure for remotely printing an error message is initially declared as int printerror(int) and later changed to int printerror(string) (for example, to change from an error code to an error string). If the server is updated but not the client, when the client calls the procedure again after the change, the server simply crashes. The crash is a consequence of the server trying to use as a string what is in fact an integer. This type mismatch is so easy to produce because Sun/RPC assumes that both the client and the server use stubs generated from the same interface speci cation, without It is a normal call to a local procedure, however. The term \remote procedure" is used in this paper to designate procedures supported by a server to be called remotely. 1

3

verifying if the assumption is actually true. In order to deal with evolving interfaces, Sun/RPC alternatively oers \protocol versions" so that servers have the opportunity to change the interface by adding a new version to the speci cation. The client would continue to use the old version as long as required, then using the same new speci cation to generate stubs for the new version. In a development environment using Sun/RPC, type-safety is not automatic because it is achieved only with strong programmer interaction (for maintaining consistent speci cations) and intervention (for creating and maintaining those versions). Perhaps even more important, the mechanism does not guarantee that type mismatches (and, as a consequence, crashes) will not occur. Recently, research in RPC has started to address type-safety more rigourously. Network objects [BNOW93], a modern RPC well integrated into Modula-3, allows methods to be called on objects across the network. Great care was taken with respect to type-safety, and because the native type codes generated by the Modula-3 compiler are local to one program, the compiler had to be changed in order to compute a ngerprint for every object type. Two types have the same ngerprint only if they are structural equivalent. Using ngerprints, network objects prevent calling methods on objects with invalid types, even between programs across the network. With type structural equivalence, membership of a type is de ned by some properties of the values themselves, as opposed to name equivalence, where membership is an attribute designated at the time a value is created. Structural equivalence oers a concept for type equivalence independent of a particular declaration in one program [CBC+90]. In a distributed application containing several programs, only structural type equivalence can guarantee that independently declared types are compatible. The RPC mechanism described in this paper uses structural equivalence between types. Although, there is no need to compute ngerprints (as happens with network objects) because type equivalence is already structural in Napier88, and type representations are available at run-time.

2.3 Server and Procedure Binding Another important issue is related to binding. Before a client calls a remote procedure, it must choose an appropriate server supporting the desired procedure (server binding), and within that server the right procedure (procedure binding). Information to identify the server can be manually provided by giving a network address, either alongside the procedure interface or later at run-time before using the stubs. Also, a number may be used which identi es the desired procedure at that server. This is the solution adopted by many RPC mechanisms including Sun/RPC [Sun90]. Hamilton [Ham84] extended the syntax for calling a remote procedure to accept a server address. Network objects [BNOW93], on the other hand, always know where to direct the call, because the remote reference to an object includes information about its location. 4

Having this information embedded within the interface or in the client code does not leave much space for exibility (for example, to change from one server to another). Also, procedure numbers force client and server programmers, which should independently build and change programs, to agree and synchronise with respect to these numbers. Information about servers and the procedures they support can alternatively be managed by means of a binding service which helps not only with both server and procedure binding but also with type-safety. A binding service is a special server that stores interfaces of remote procedures and controls the right to call them, and there is only one binding service in the whole system. A binding service was chosen to the RPC described in this paper because it oers a compromise between the convenience and security of automatic type-safe RPC and the eciency of manual server and procedure binding. Birrel and Nelson [BN84] use a distributed database to store server addresses and their supported remote procedures names, without attempting to control argument types. The ANSA architecture includes the notion of a trading service [DH93] responsible for passing information from servers to clients, allowing queries to be made on the information stored. The binding service works as follows. Before a server starts supporting remote calls, it must register the interfaces of the remote procedures it supports in the binding service. A client, before calling a procedure in a remote program, must also contact the binding service with the remote procedure interface. If that remote procedure is supported in a server, the binding service answers with the address of that server (server binding), the identi cation of the procedure in the server (procedure binding) and an authorisation to call it (type-safety). Thus, it is impossible for a client to call a procedure without proving before that it knows the procedure name and the correct number, order and types of its arguments.

3 Persistent Programming Languages Conventional programming languages manipulate values which are resident in memory. If these values are to be used later by the same or any other program, then the le system or some sort of database system must be used. A persistent programming language provides values to remain available as long as they are required for computation, thus eliminating the need for les or databases to be used. If persistence is orthogonal to type, the provision of persistence will apply equitably to values of any type. Orthogonal persistence [ACC82] gives the programmer freedom to choose the best types independent of the length of time the values will persist. This freedom eliminates the burden of having to choose two dierent representations for storage and manipulation, and to write the code to translate between them. If all values of any type may persist the time required by the application, then no 5

dierence exists between data for short or long term storage. A consequence of this is that programmers do not need for explicitly write code to move and translate data between the programming language and the database system. The amount of code needed for this transference and translation, and the conceptual burden that results from having two data models, has been called the \impedance mismatch" problem. Persistent programming languages solve this problem by incorporating persistence into the programming language. The RPC mechanism presented in this paper was implemented in Napier88, a persistent programming language described next.

3.1 Napier88 Originally developed as part of the PISA project [AMP87], Napier88 [MBCD89] is a strongly typed, procedural persistent programming language. The implementation this paper refers to was built by the Persistent Programming Research Group at the University of St Andrews, Scotland. It is oered as a compiler to an intermediate language, an interpreter for that language and a store already populated with standard procedures (also called the standard store). The standard store contains procedures for input and output, network access, threads and semaphores, string and byte operations, type interrogation and, in general, for type-safe manipulation of the stored data (including procedures). In Napier88, persistence is identi ed by reachability from a persistent root. The persistent store is calculated as the transitive closure of this root. Values are made persistent by putting them in this closure. Those values not reachable are garbage collected, so there is no need for an explicit call to free storage space. An environment is a collection of tuples with an identi er, a type, a value and a constancy (that is, the directory concept of a le system). As environments may contain environments themselves and the root of the persistent store is an environment, they are used in Napier88 to build an hierarchical structure such that variables in these environments will persist by reachability from the persistent root. Napier88 oers a rich type system which can be interrogated [Con91] described by a set of base types and a set of constructors. The set of base types includes integer, real, boolean, string and null, amongst others; structure, vector and variant are examples of type constructors. Type equivalence is structural. Napier88 also provides higher-order procedures, that is, procedures are just another type which can be created and put in the store. Finally, the language supports recursive and parametric types, including parametric procedures. Also supported in Napier88 is type-safe linguistic re ection [Kir93], which allows a program to extend or modify its own behaviour at run-time. This is achieved by allowing programs to access the compiler while executing, which is a standard procedure in the store. This way, programs may alter themselves by creating new program fragments 6

or even new types at run-time, which are compiled and integrated into the current execution. Type-safe re ection means that all re ective operations are type-checked. Not part of the language, but well integrated with it, is a library of parametric bulk data types [ABC+93]. This library, implemented in the University of Glasgow, includes lists and maps, amongst other bulk types, all with a similar interface. The library facilitates the task of building complex applications because most needed general code already exists.

4 Programming Interface The programming interface characterises how the programmer actually interacts with the mechanism. Two main issues must be addressed when designing an RPC mechanism: rstly, how to pass information about the procedure interfaces to the stubs generator in order for it to create the client and server stubs; secondly, how the program uses these stubs from within the language. These issues are examined next in turn. After that, examples of both generating stubs and using them is given for the client and the server.

4.1 Generating Client and Server Stubs The programmer must provide the stubs generator with procedure interfaces (also known as signatures), that is, information about the remote procedure names and the types of their arguments2. Usually, this is achieved by an interface written in a \procedure declaration language" which provides the stubs generator with these signatures. This approach necessitates a separate program to compile the signatures and generate the stubs. In fact, Birrel and Nelson [BN84] use a pre-processor to produce the stubs, and the same approach is still used by most modern RPCs, such as the Sun/RPC [Sun90]. In contrast, the RPC mechanism described in this paper does not use an external stub generator to compile the signatures of the remote procedures written in a dierent language. There are two main reasons for this: 1. a separate language (in addition to the existing language) is always harmful as it increases the complexity of the programming environment: programmers need to learn a new language, even if derived from a well-known one, and also to translate and maintain consistent types between the procedure declaration and its use in the program; and The paper refers to arguments as including the result, if any. Procedure arguments are also known as parameters, but in Napier88 terminology this word is used for parametric types. 2

7

2. avoiding a separate language is made possible in Napier88 because:   

the type system permits to extract the necessary type information at runtime; the callable compiler allows the compilation of the generated stubs in a type-safe manner at execution time; and the compiled stubs can be stored for later use (procedures are just another type that can be made persistent).

Type information could also be extracted from the program text itself at compile-time, but our scheme is more exible in that it supports creating client and server stubs at any time.

4.2 Using the Generated Stubs After generating the stubs, the programmer then needs to use them from within the language. Hamilton [Ham84] decided to extend the language with special syntax for calling the remote procedures; he argues that extending the language adds extra syntax for RPC control information (such as server binding and error handling) and also emphasizes the semantic distinction between calls to local and remote procedures. Birrel and Nelson [BN84] provide such information in the form of extra arguments to the call, a technique also used by Sun/RPC [Sun90]. Wai [Wai88] also extended the language in a manner similar to Hamilton for server binding, without attempting to deal with errors at the language level. It was chosen to not extend Napier88 with extra syntax for calling remote procedures because: 1. extending the language would force programmers to learn new syntax to use the mechanism, which can be a major drawback for its acceptance; 2. a language extension also causes changes to the compiler that would dicult porting the mechanism into future implementation releases of the language; and 3. some simpli cations are possible because currently the mechanism does not attempt to cope with errors (though dealing with errors does not necessarily forces a language extension). If the syntax for calling remote procedures is identical to that used for local ones, the RPC mechanism does not oer the semantic distinction argued as important by Hamilton and others. With identical syntax, programmers should take great care when calling remote procedures, especially those aspects related with arguments semantics (discussed later in section 5). 8

4.3 Example of Generating and Using Client Stubs An example is showed in Napier88 of how to call the stub generator at run-time to create client stubs and another of how to use them. Unlike a traditional RPC, which uses a separate program for compiling the procedure interfaces, the client stub generator accepts a Napier88 environment (see subsection 3.1) with procedure signatures. The signatures are extracted from the environment and the stubs generated and compiled, all at run-time. Programmers also use client stubs with a normal sequence of Napier88 statements. The client stub generator is oered as a procedure called makeClientStubs which accepts an environment containing dummy procedures used for signature extraction, obtains their signatures, generates and compiles the respective client stubs, and nally replaces these dummy procedures with the client stubs. After calling makeClientStubs, the programmer may immediately utilise the environment containing the generated stubs or put it in the persistent store for later use. makeClientStubs uses several procedures found in the standard store. The standard procedure scan inspects the environment with the uninitialised procedures. Another standard procedure called getType accepts any value as its single argument and returns its type, which is used in this context to return a procedure signature (in Napier88, a procedure is just a normal value with type). After all stubs for the procedures found in the environment have been generated, the code is then passed to the standard procedure compile, the run-time callable compiler. The compiled code is then executed and, as a result, the stubs are created and then returned to the client program. Figure 2 shows how a client creates stubs for interacting with an example message server. The purpose of the message server is to store messages in a database, each indexed by an integer generated by the server. In the server, clrMsg clears (or resets) the database of messages, putMsg creates a new message returning its identi er and getMsg accepts an identi er returning a previously created message. The procedure uninitialised[T](string->T) implements a dummy procedure with a return value of type T. This is not a special procedure in the sense that it was designed exclusively for the RPC mechanism, but a normal procedure in the store commonly used by many Napier88 programmers. After calling makeClientStubs to create the client stubs, the programmer may then use these stubs in a normal sequence of Napier88 statements. Figure 3 shows how the client stubs generated with the previous example can be used to test the message server. After this sequence of statements, the server must now hold a message "test msg" indexed by some locally generated number.

9

! creates an empty environment to hold the signatures let ClientProcEnv := environment()

! insert into it the signature for clrMsg

in ClientProcEnv let clrMsg := proc() uninitialised void("clrMsg")

! insert the signature for putMsg

in ClientProcEnv let putMsg := proc( s:string -> int ) uninitialised[int]("putMsg")

! insert the signature for getMsg

in ClientProcEnv let getMsg := proc( i:int -> string ) uninitialised[string]("getMsg")

! construct the stubs for all three signatures

let ClientStubEnv := makeClientStubs( ClientProcEnv )

Figure 2: Generating Client Stubs ! obtain the client stubs from the environment

use ClientStubEnv with clrMsg: proc(); putMsg: proc( string -> int); getMsg: proc( int -> string ) in

! call the client stubs

begin clrMsg() let nbr := putMsg("test msg") let msg := getMsg(nbr) end

! throw away stored messages on the server ! store a message on the server ! retrieve that message from the server

Figure 3: Using Client Stubs

10

4.4 Example of Generating and Using Server Stubs On the server side, the interface oered to the programmer by the mechanism is somewhat similar to the client side, but its implementation has three important dierences: 1. the stubs generator this time uses the processing part of the procedures in the environment passed as argument; 2. the server stubs are stored by the stubs generator directly, instead of being returned to the server program; and 3. the programmer now calls a procedure which waits for incoming requests from the clients, instead of calling the server stubs. To generate the server stubs, the programmer calls makeServerStubs with an environment containing implemented procedures. Although these implemented procedures need to be persistent in order to be called later by the server stubs, the environment with the implemented procedures need not be persistent itself, as the process of generating the stubs will store these procedures by reachability from the server stubs (which will be made persistent). Figure 4 shows how to create the message server. Procedures beginning with m create and manipulate maps, one of the data types available in the Napier88 library of bulk types [ABC+93]. A map provides a representation of mappings between any two types and one is used, called mapOfMsgs, to associate a number (integer) with each message (string). Because mapOfMsgs is used by these procedures and these are persistent by reachability from the persistent stubs, the map itself persists. After creating the server stubs, the programmer calls serverStubMngr to start accepting incoming call requests from clients ( gure 5). When a incoming call arrives, serverStubMngr selects the appropriate server stub (based on the message, as explained next) and calls the stub.

5 Argument Semantics With a syntax for calling remote procedures identical to that used for calling local procedures, one would expect the semantics of calls to local and remote procedures to be the same. Unfortunately, this is not true, as the physical separation of the application in address spaces (between the client and server programs) imposes restricted semantics on argument passing and the range of transmittable types. 11

! create \equal" and \less than" for integers, and the map

let eqInt := proc( p1:int; p2:int -> bool ); p1 = p2 let ltInt := proc( p1:int; p2:int -> bool ); p1 < p2 let mapOfMsgs := m empty[int,string](eqInt,ltInt)

! create a variable to hold the number of stored messages let nxtMsgId := 0

! set up an empty environment to hold the procedures

let ServerProcEnv := environment()

! implements and inserts clrMsg into the environment

in ServerProcEnv let clrMsg := proc() mapOfMsgs := m empty[int,string](eqInt,ltInt)

! implements and inserts putMsg into the environment

in ServerProcEnv let putMsg := proc( msg:string -> int ) begin nxtMsgId := nxtMsgId + 1 m isu insert[int,string](mapOfMsgs,nxtMsgId,msg) nxtMsgId end

! implements and inserts getMsg into the environment

in ServerProcEnv let getMsg := proc( i:int -> string ) m find[int,string](mapOfMsgs,i)

! construct the stubs for all three procedures makeServerStubs( ServerProcEnv )

Figure 4: Generating Server Stubs ! start accepting call requests serverStubMngr()

Figure 5: Using Server Stubs

12

5.1 Argument Passing by Value A remote procedure is, by de nition, executed in a dierent address space from where it is called. This is the reason why most RPC mechanisms pass arguments by value. However, some RPC mechanisms do pass arguments by reference. For example, Wai [Wai88] implemented an RPC in PS-algol [ACC82] (a predecessor to Napier88) which supported call by value semantics not only for scalars but also for pointer values, instead of deep-copying data structures across programs. Wai argues that this is essential for building a distributed persistent language. Network objects [BNOW93] pass these objects by reference, but normal objects by value. But, again, the objective is to oer a distributed programming system. The objective of this work is to automate type-safe RPC, not to build a distributed language. This is the reason why we chose to pass arguments by value, not only for scalars but also for complex objects. The programmer must be aware that the semantics for argument passing in calls to remote procedures are dierent from those to local ones. Passing arguments by value means that the arguments are copied between the client and server programs. As these arguments may have references to other values, these must also be copied, and so on until the transitive closure from the original arguments have been copied. Even if the values are the same in the client and server programs, the arguments now have dierent identities because they were copied from one program to another and built again. This means that, if the argument is returned again to the calling program, it will be a dierent value with the Napier88 equality test, even if both actually represent the same (structural) value.

5.2 Packing and Unpacking Arguments In the design of an RPC mechanism, one must try to support as many types as possible. However, the remote procedure is executing in a dierent address space from where the invoking procedure resides. This fact restricts the range of types supported by the RPC mechanism. Each restriction may happen because of some inherent diculty or simply because its implementation and/or execution are too expensive. Due to limited resources it was decided to support only a limited number of types and constructors, and instead investigate other, perhaps more important, research issues (for example, sessions as described in subsection 6.1). The RPC mechanism described in this paper supports only the most common base types: integer, real, string, boolean and null. It also supports the following type constructors: vector (one dimensional array), structure (labelled record type) and variant (labelled discriminated union), and the recursive composition of these. The implementation generates a pack or an unpack routine for each type found in 13

a procedure signature when creating the client or the server stubs. These routines are placed in the store (indexed by type) to be later reused, should one of these types re-appear as an argument of another signature. This is possible only because Napier88 supports higher-order procedures and orthogonal persistence. Types are separated in simple types and complex types. Simple types cover a subset of Napier88 base types: integer, real, string, boolean and null3. Complex types are those which use one of the supported constructors: vector, structure and variant. Pack and unpack routines for simple types were hand-written, compiled and stored, and are simply called from the generated stubs. Routines for packing and unpacking complex types are generated when the types are rst discovered, then compiled and stored for later use. However, the interpreter (for the intermediate language to which Napier88 is compiled) already oers a set of (not necessarily type-safe) low-level procedures that perform a similar function, and it could be argued that the mechanism should take advantage of their existence. These low-level procedures were not used because using them would prevent the RPC mechanism to be ported for future releases or dierent implementations of the language. Complex types can be quite complicated, as they are created by recursively applying constructors to other complex or simple types. In order to lower the level of complexity of the implementation, complex types are attened before the actual pack and unpack routines are generated. Flattened complex types are limited to one constructor for each composition. For example, a vector of a structure with only one integer eld, represented in Napier88 as type ComplexType is *structure(i:int) becomes, after being attened rec type Type1 is *Type2; Type2 is structure(i:int) (where rec means recursive de nition). These new attened types are dierent from the original ones, and have dierent names. It may appear that generating pack and unpack routines for these new types will invalidate their use with values belonging to the old (pre- attened) types. However, since Napier88 type equivalence is structural, both the new and the old types are equivalent. The only other RPC implementation in a modern persistent language that we are aware of, that being developed for Tycoon, uses \general polymorphic intern/extern linearization" procedures [Mat94], thus Tycoon/RPC is type-complete (including higher-order procedures and threads). Procedures similar to these in Tycoon also exist in Napier88 for implementing persistence, but (at least in Napier88) they have the undesirable side-eect of also packing the whole closure statically bound to the arguments. We chose instead to build our own linearization procedures in order to experiment with several binding schemes. 3

There is only one value of type null: nil.

14

5.3 Shared Values When traversing the graph of referenced values, a value already copied may be found. This occurs when two dierent values share some other value. The algorithm for deepcopying arguments must preserve the semantics of this sharing [HL82]. Sharing values within a call is important because this is the base of many important data structures, for example, a doubly linked list. For each packed value an identi cation for it is created and packed alongside the value. When the same value is about to be packed again, the packing algorithm just sends its identi cation, instead of packing the same value a second time. On the other side, the unpacking algorithm stores the unpacked values together with their identi cations, preserving sharing semantics in the destination program. The packing and unpacking algorithms as described not only preserve sharing semantics, but also perform ecient deep-copying across programs, by avoiding to copy shared values repeatedly. However, the value may change between invocations of the same remote procedure. This implies that shared semantics is maintained only within a call to a remote procedure.

6 Binding and Type-checking Before calling a remote procedure, a client needs to bind to the server supporting that particular procedure. The client also needs to know the identity of the procedure in the server program, in order to avoid sending the procedure signature each time it issues a remote call. Finally, the server must check that the packed arguments in the received message are in number, order and type as those of the remote procedure.

6.1 Sessions Static or dynamic type-checking may be used locally to ensure that both a client and a server are within themselves type-safe. However, as any client and any server in a distributed environment should be allowed to be independently built and changed, one cannot easily guarantee that between them type mismatches will not occur. A straightforward solution to this problem would be to have one \godfather" ensuring that any server would meet the needs of all its clients, and that the needs of all clients are met by their servers. This solution would require network-wide coordination to change anything in the system, from adding a new server to altering the signature of a remote procedure. In a real distributed system, with autonomously evolving clients and servers, this solution is clearly impracticable. 15

The other extreme solution would be for a client to send enough information for procedure binding and type-checking in each call to a remote procedure. This approach has two advantages: only pairwise check of signatures is necessary, when and if needed (what could be called \lazy checking"); and any change to the system can incrementally take place and propagate across the network. This solution has the disadvantage that the control information and processing time needed in each call can be too expensive when compared to the procedure call itself. A compromise between the previous two radical solutions is possible, gaining some positive features of each approach. The concept of session is introduced. A session describes the relationship between a client stub and a server stub. Its requirements are established at the start of a session and maintained until the end of a session. A server can then change a procedure interface during a session, but keep this change to itself; at the start of a new session, the change is made visible to the outside world and the system evolves. The cost of signature matching is incurred only once per session. Programmers may choose session lengths, as the client can terminate the current session and start a new one at any time. Sessions provide the conceptual framework for incremental evolution. Servers may change procedure signatures or start supporting new procedures at any time; although, clients would be aware of these changes only at the beginning of a new session for every procedure. This way, the system evolves incrementally, with changes being propagated from servers to clients at convenient times. Sessions are unrelated to the execution time of either clients or servers, because information on sessions persists across program invocations. In Napier88, threads are persistent, thus a server may even interrupt its execution and restart later at exactly the same state. Also, because the (local) store updates its contents in a transactional way, sessions obey locally to the principles of \all or nothing". But session consistency between clients and servers is not based on any notion of \distributed transaction"; instead, sessions are implemented by using capabilities and a binding service, as described below.

6.2 Capabilities The client, before calling a remote procedure, should own a capability for it. The concept of capability was rst proposed by Dennis and van Horn [DvH66], and Morrison et al [MBC+90] provide a good overview of its relationship with type systems. Although, capabilities are probably most well known in the distributed system area by being a basic concept in the Amoeba distributed operating system [MT86]. In generic terms, a capability is the concatenation of an identi er, a rights eld, and a password to prevent users from predicting or forging capabilities. Capabilities are created only by a trusted entity, and used by other (trusted or untrusted) entities in 16

the system. Here, the binding service is considered to be the single trusted entity in the system, and both clients and servers are untrusted. In this context, a capability is used for both identifying (procedure binding) and validating (type-checking) a call to a remote procedure. The rights eld is not used at present, although it may be used in the future. The binding service then takes advantage of the knowledge it owns about remote procedures and their servers to oer server binding. In this scheme, only the binding service creates capabilities that are guaranteed to be unique. A server registers a procedure in the binding service, and receives as a result its capability. Clients must acquire the capability for a remote procedure from the binding service before calling the procedure at the server. Each capability identi es a procedure. The capability is then used for identifying the remote procedure at the server, but also for type-checking. Using capabilities that way, they can be seen as just an optimisation for type-checking after the rst call, rather than a semantic concept. Capabilities are valid only within the boundaries of a session. The address of the server supporting a particular remote procedure is returned along with its capability to the client. This separates server binding from procedure binding and type-checking. The separation may be useful in the future, when the binding service would take advantage of the information it owns to better manage servers and procedures. The possibility for capabilities and sessions to be exchanged between clients, as supported by other RPC mechanisms, was deliberately avoided since it would introduce new problems well beyond the scope of this research.

6.3 Binding Service The cost of session set-up can be reduced and its exibility increased by centralizing information on sessions in a binding service. The binding service stores all the information related to servers and the procedures they support. Servers tell the binding service the procedures they support, and clients contact the binding service before calling these procedures. After this initial set-up, the client directly calls the procedure at the server, without contacting the binding service. The binding service is implemented as a normal server which supports two special remote procedures: import and export. Each time a server starts executing it must export to the binding service the name and types of the arguments of the procedures it supports, and that should be done before accepting any call requests from clients. Also, the client, before calling a remote procedure, must contact the binding service to import the capability of a previously exported remote procedure and the address of its server. 17

Binding Service

Client

Server

export( server, signature ) capability import(signature) server,capability proc( capability, arguments ) result

Figure 6: Using the Binding Service In order for the servers to export their procedures to the binding service, they must rst import the procedure export itself, so they need import. Also, clients must call import at the binding service, so they should also import the procedure import itself. To solve this recursion at both client and server, import and export have well know capabilities, so they do not need to be imported by the clients and servers. Both the client and the server must have client stubs for import and export, and the binding service must also generate server stubs for these procedures. The server exports to the binding service the signatures of the remote procedures it supports by calling export to all \not yet exported" procedures ( gure 6). Export receives as arguments a server address and a procedure signature, and returns a capability for them. The server must then associate this capability with the corresponding server stub. If the binding service receives a signature from a server which was already previously exported by another server, it simply stores both instances for future reference (returning a dierent capability this time). This allows dynamic recon guration of servers, without requiring programs to change. The binding service may be used both for load balancing (choosing an alternate \not so loaded" server) or resilience (changing the binding to another server if one fails); however, neither are implemented at present. Currently, the binding service just returns to a client importing a multiple exported signature the capability returned to the rst server which exported it. On the client side, before calling the remote procedure, each client stub checks if it already has the capability. If not, the stub itself calls import with its own signature, receiving the capability and the address of a server supporting the procedure, and only then calls the remote procedure. Later, the stub would discover it already has the capability, so it is unnecessary to import the procedure again. In order to known its own type, the client stub calls the standard procedure getType which accepts a value (of any type) and returns a type representation. A list (an18

other bulk type found in the library [ABC+93]) with tuples type representation, capability, server address stores the interfaces of all exported remote procedures at the binding service, and a similar list caches capabilities at the client. Both the client and the binding service use the standard procedure EqualType (to check if two types are structural equivalent) to nd a tuple in the list. At the server, a call to the correct stub is issued when a message is received, based on information stored in a map from capabilities to server stubs. When a client receives a capability that enables it to call a remote procedure at a server, it eectively starts a new session. This session will last as long as the server supports the procedure. When the server stops supporting the procedure, the client will note that event because its next call will fail.


6.4 Support for Incremental Change Each capability is valid for calling a remote procedure with some interface at a server for the duration of a session. This behaviour allows a server to change a procedure implementation without starting a new session, if the interface itself does not change. The server stub generator simply replaces the old server stub with the new one, which calls the new implementation; the client will not notice the change. This behaviour is desirable as it allows server evolution with minimal consequences. That is, an internal change in a remote procedure do not need usually to be propagated to its clients, so the mechanism hides these kind of changes by default. The server can always force the clients to notice the change by revoking support to the existing procedure, and then start supporting a new remote procedure with the same interface. Another typical change occurs when a server starts supporting a new implementation of the remote procedure with a dierent interface. According to the ideals of incremental evolution, the server should not stop supporting the old version. Instead, the server should support both the old and the new interface, allowing the client to migrate from one interface to the other at a convenient time. Even more interesting, if one integrates these two previous kinds of incremental change, they allow the server to change the implementation with the old interface (without the client noticing the change) in order to incorporate, if desired, (part of) the functionality of the new interface. The example given in subsection 2.2 is supported by having two instances of the procedure printerror, a int printerror(int) and a int printerror(string). Initially, all clients use int printerror(int), but as int printerror(string) starts to be supported at the server as well, each client decides when it is ready to change to the new interface. 19

Sun/RPC [Sun90] oers \protocol versions" to achieve a similar eect. Using the same example, Sun/RPC has int printerror(int) as \printerror version 1" and int printerror(string) as \printerror version 2". Clients bind to one of the printerror versions. Even though Sun/RPC also supports incremental evolution, the concept of session enforces much stronger type-safety than that provided by Sun/RPC, as procedure signatures are type-checked at the beginning of each session.

7 Transport Protocol The RPC mechanism is built on top of sockets. Sockets are oered in Napier88 through a set of procedures found in the standard store which provide a similar interface to that used in other languages (for example, as described in [Sun90]. These procedures actually implement a protocol, built on top of conventional sockets, for setting up a connection, as a Napier88 program is identi ed by the machine name where it is being executed and the store directory, whilst bare sockets operate on connection identi ers. The transport protocol at the client and the server use these sockets to exchange packets between programs across the network. Instead of using sockets as the underlying protocol, it could have been decided to use higher level packages such as Stacos [Mun93], a tool for deep-copying objects between two Napier88 stores. Stacos copies the actual value representation in the store, and thus is heavily dependent of the homogeneous representation of values, that is, both programs should be written in Napier88. Sockets were chosen because: 1. Stacos was built speci cally to Napier88 and it was decided not to take any decision that would invalidate its implementation in other (potentially nonpersistent) languages; and 2. Stacos does not provide the exibility necessary for dealing with errors; although in the current implementation error handling is not supported, it would be better for this to remain an open question. Exchanging packets between programs is only part of the problem. The packets themselves must also conform to some structure. For example, Sun/RPC uses XDR [Sun90], a standard for the description and encoding of data, useful for transferring data between dierent computer architectures. DCE/RPC [Ope90] allows distributed applications to run over heterogeneous systems, and instead of using a single external data representation, remote calls are tagged with a description of the calling machine's basic data representations. For this implementation it was chosen to support a very simple, ad-hoc structure for packets that can later be changed to a standard such as XDR. 20

7.1 Client Stubs Manager After packing the arguments into a packet, a client stub calls its client stub manager in order to send the packet to the server. The client stub manager implements the transport protocol at the client side, and it is responsible for sending the packet to the server. After sending the packet, it waits for the answer packet from the server, and returns it to the stub again. In the current implementation a new socket connection is created for each packet.

7.2 Server Stubs Manager The server stub manager is called once by the programmer, and it waits for client requests in a busy wait fashion, actively reading the socket until a packet is received. When one arrives, it extracts the capability from this packet and identi es the procedure to be called in a type-safe manner. The server stub manager then calls the correct stub with the rest of the packet as its single argument. The stub calls the desired procedure, returns a packet containing the return value, and the manager sends it back to the client. The server stub manager creates a new thread for each incoming packet, in which context the procedure will be executed. It is up to the programmer to implement the necessary policies in order to guarantee the concurrent execution of several remote procedures. In Napier88, a thread is created by calling a procedure named start that accepts a procedure without arguments and returns a Thread abstract data type. Later, the thread can be suspended, restarted or killed. Multi-threading in Napier88 is implemented by the interpreter, including the scheduling policy. The server stubs manager waits for client requests in a busy wait fashion, actively reading the socket until a packet is received. Although this may appear as a waste of CPU resource at rst glance, it is not so severe if one admits that most scheduling policies will try to share CPU time in a fair time slice, should this resource be required by another program. Also, most important servers are assigned to a machine of their own. On the other hand, the program could wait for a socket event, but this facility is not currently supported in Napier88.

8 Implementation The RPC mechanism was implemented in Napier88 running on a Sun workstation. The hand written code is about 4,700 lines of Napier88. Three Sun workstations connected by an Ethernet have been used for tests, but as server locations are already full Internet addresses, the mechanism should work without modi cation in any Sun 21

Server

Client Client Program

Client Stubs

Client Stubs Manager

Generator of Client Stubs

Generator of Server Stubs

Remote Procedures Server Stubs

Generator of Routines for Pack & Unpack function call

code generation

Server Stubs Manager

Figure 7: The RPC Implementation Architecture workstation accessible from the Internet. No performance gures for calling remote procedures existed at the time of writing. The RPC mechanism is implemented as two main modules of code, client and server, which share a common module for generating code for packing and unpacking data structures ( gure 7). Both the client and the server are organised in three layers: user program, stub procedures and stubs manager. The program has local procedures, and in the case of the server also contains remote procedures. Stubs pack and unpack data structures, and on the server side also call the appropriate remote procedure. The transport protocol level is implemented by a pair of stub managers for the client and the server, which on the server side also calls the appropriate server stub. This functionality is commonly realized by a \dispatcher" procedure in other implementations. The programs call either the client or the server stub generator for creating the respective stubs. The client and server stub generators are dierent from each other, but they share the same set of procedures which generate code for packing and unpacking complex data structures.

9 Summary and Future Work This paper describes the design and implementation of an automatic type-safe RPC mechanism built in Napier88, a persistent programming language. Type-safety is an important feature as it guarantees the same interface between the expected and the actual remote procedure, thus eliminating an important source of severe errors. It is strongly enforced by the notion of structural equivalence between types and supported by sessions. Sessions achieve incremental change in a type-safe manner. Type-safety is automatic because no programmer intervention is needed, apart that 22

required otherwise for declaring normal procedures and calling them. The distributed database currently being implemented using the RPC mechanism, which will be subject to a number of dierent changes, is expected to demonstrate the adequacy of the research work described in this paper. Many Napier88 features are used in the implementation, such as orthogonal persistence, higher-order procedures, a run-time callable compiler, the rich type system and a library of bulk types. These, together with their well-suited integration, helped to implemented the mechanism in a number of aspects, including:

type system which allows interrogation at run-time, excludes the need for a separate

language for declaring procedure signatures callable compiler supports the compilation of the stubs at run-time in a type-safe manner orthogonal persistence is used to store stubs and associated data structures, and enables the concept of session to last longer than the execution of programs It is very important to stress that the RPC mechanism is entirely built in Napier88; the language was not extended nor was a separate language needed. This also permitted stronger type-safety than is usual with minimum eort, as Napier88 type equivalence is already structural. After some reluctance in the beginning, especially that related with the complexity introduced by the new persistent dimension, Napier88 proved to be an excellent development platform for designing and implementing an automatic type-safe RPC mechanism prepared for incremental change. We are currently investigating ways of extending the number of data types supported by the RPC mechanism, especially the richer ones such as polymorphic and abstract data types. Also, support for procedures as elds in structures would oer the concept of \interface" as found in other RPC mechanisms. These experiments will certainly lead to even more interesting issues in binding, and its relationship with distributed systems in general.

Acknowledgements Malcolm Atkinson for his many ideas and very stimulating comments and criticism. A number of other people in Glasgow must also be acknowledge: Peter Dickman, Paul Philbrow, Laurent Daynes, and Rich Cooper. Bernd Mathiske in Hamburg for his comments on Tycoon/RPC. The Persistent Programming Research Group in St Andrews developed this robust Napier88 implementation. 23

The research work leading to this paper was supported partly by JNICT and partly by INESC. The author's research environment in Glasgow is also partially supported by the EC projects FIDE2 (ESPRIT 6309) and IMIS (ESPRIT 6548).

References [ABC+93] M.P. Atkinson, P.J. Bailey, D. Christie, K. Cropper, and P.C. Philbrow. Towards bulk type libraries for Napier88. Technical Report FIDE/93/78, ESPRIT Basic Research Action, Project Number 6309|FIDE2 , 1993. 36pp, including index. [ACC82] M.P. Atkinson, K.J. Chisholm, and W.P. Cockshott. PS-algol: An algol with a persistent heap. ACM SIGPLAN Notices, 17(7):24{31, July 1982. [AMP87] M.P. Atkinson, R. Morrison, and G.D. Pratten. PISA: A persistent information space architecture. ICL Technical Journal, 5(3):477{491, May 1987. [BN84] A.D. Birrel and B.J. Nelson. Implementing remote procedure calls. ACM Transactions on Computer Systems, 2(1):39{59, February 1984. [BNOW93] A. Birrell, G. Nelson, S. Owicki, and E. Wobber. Network objects. In Proceedings of the Fourteenth ACM Symposium on Operating System Principles (5th{8th December 1993, Asheville, NC), pages 217{230. Association for Computing Machinery, 1993. [CBC+90] R.C.H. Connor, A.L. Brown, Q.I. Cutts, A. Dearle, R. Morrison, and J. Rosenberg. Type equivalence checking in persistent object systems. In A. Dearle, G.M. Shaw, and S.B. Zdonik, editors, Implementing Persistent Object Bases, Principles and Practice, pages 154{167. San Mateo, CA: Morgan Kaufmann Publishers, 1990. Proceedings of the Fourth International Workshop on Persistent Object Systems, Their Design, Implementation and Use (Martha's Vineyard, USA, September 1990). [Con91] R.C.H. Connor. Types and Polymorphism in Persistent Programming Systems. PhD thesis, University of St Andrews, 1991. [DH93] J.P. Deschrevel and A.J. Herbert. The ANSA Model of Federation and Trading. Architecture Projects Management Ltd., Cambridge (UK), February 1993. [DvH66] J. B. Dennis and E. C. van Horn. Programming semantics for multiprogrammed computations. Communications of the ACM, 9(3):143{145, 1966. 24

[Ham84]

K.G. Hamilton. A Remote Procedure Call System. PhD thesis, University of Cambridge Computer Laboratory, 1984. [HL82] Maurice Herlihy and Barbara Liskov. A value transmission method for abstract data types. ACM Transactions on Programming Languages and Systems, 4(4):527{551, October 1982. [Kir93] G.N.C. Kirby. Re ection and Hyper-Programming in Persistent Programming Systems. PhD thesis, University of St Andrews, 1993. [Mat94] Bernd Mathiske. Comparing Napier88/RPC with Tycoon/RPC. Private communication, December 1994. [MBC+90] R. Morrison, A.L. Brown, R.C.H. Connor, Q.I. Cutts, A. Dearle, G.N.C. Kirby, J. Rosenberg, and D. Stemple. Protection in persistent object systems. In J. Rosenberg and J.L. Keedy, editors, Security and Persistence. Proceedings of the International Workshop on Computer Architectures to Support Security and Persistence of Information (Bremen, West Germany, 8{11 May 1990), Workshops in Computing, pages 48{66. SpringerVerlag in collaboration with the British Computer Society, 1990. [MBCD89] R. Morrison, A.L. Brown, R.C.H. Connor, and A. Dearle. The Napier88 reference manual. Technical Report PPRR-77-89, Universities of Glasgow and St Andrews, 1989. [MT86] S. J. Mullender and A. S. Tanenbaum. The design of a capability-based distributed operating system. The Computer Journal, 29(4):289{300, 1986. [Mun93] D.S. Munro. On the Integration of Concurrency, Distribution and Persistence. PhD thesis, University of St Andrews, 1993. [Ope90] Open Software Foundation, Inc., 11 Cambridge Center, Cambridge, MA 02141. Distributed Computing Environment: Overview, May 1990. [Sun90] Sun Microsystems, Inc., 2550 Garcia Avenue, Mountain View, CA 94043. Network Programming, SunOS 4.1, March 1990. [Wai88] F. Wai. Distributed Concurrent Persistent Programming Languages: An Experimental Design and Implementation. PhD thesis, University of Glasgow, April 1988.

25