Published: In Proc. European Research Seminar in Advanced Distributed Systems (ERSADS’97), Zinal (Valais, Switzerland) - March 17-21, 1997
New Paradigms for Distributed Programming Jan Vitek
[email protected], Object Systems Group, University of Geneva, Geneva, Switzerland
The explosive growth of telecommunication networks is presenting distributed computing with new challenges. Distributed computing in-the-small as represented by LANs and small configurations of trusted machines cooperating towards a common goal have yielded to massively distributed computing systems where millions of untrusted computers running heterogeneous operating systems on top of varying hardware trade services and resources. These new problems requires methodological and technological advances. This position paper discusses new paradigms for building distributed applications for massively distributed computing systems.
1 Introduction The face of computing is changing faster than ever. In the last five years, the explosive growth of both Internet and WWW have redefined the computer as a network connection appliance, with the motto that the network is the computer. Technically, network connectivity is a given. Computer systems from workstations down to palmtops and even cellular phones provide some degree of network access. Network appliances that connect television sets to networks will soon enter homes. At the same time, a digital economy is being created. The driving force behind all of these change is software. The WWW, for instance, is the largest and simplest distributed system ever built. Software engineeringwise, it is an ad hoc solution to a difficult problem: the problem of developing software for massively distributed computing systems such as the Internet. Massively distributed computing systems (MDCS) distinguish themselves from traditional systems on 4 counts: scale, connectivity, heterogeneity, and autonomy. First, scale, MDCS offer a volume of information and number of services which is some orders of magnitude larger than traditional distributed systems. The number of nodes is also much larger; today millions, in the future up to billions of nodes. Second, connectivity is highly variable both in performance and in reliability. Some nodes have high bandwidth and reliable connections; others such as laptops or cellular phones [11] are only intermittently connected. At the same time, the cost of connect-time forces users to work off-line. Sharing of the communication infrastructure by a large number of nodes as well as data volume variations further threatens performance. The network is sparse and is modified dynamically as nodes and links are taken on- and off-line. The communication infrastructure is subject to occasional partitioning caused by events such as physical breakdowns, radio interference and blackouts, and network congestion. In general, therefore, it will not be possible to place bounds on communication delays and relative speeds of computations. Third, heterogeneity of nodes. MDCS comprise nodes of different computational power: from the anorexic cellular phone to the supercomputer. Technically, these nodes have little in common except the basic ability to exchange data over communication lines. They run different software with different operating systems on top of different hardware. And finally, autonomy means that nodes belong to different owners, be they individual users, large corporations, governmental agencies or universities. These owners, quite properly, insist on having full control over their own resources, e.g. specifying how much resources can be used, when and by whom. Thus different security policies must coexist in MDCS. MDCS represent a challenge to the computer science community. The challenge is to devise programming paradigms suited for developing applications for massively distributed computing systems and to develop the technology needed to support these paradigms. On the technological side the key issues includes security, ease of use, scalability and efficiency. This technology will have to support applications that access large numbers of resources located on geographically dis-
2
Jan Vitek
tant hosts administered by different organizations. These applications will manage large number of users working concurrently with different access right and different levels of trust. The technology must support coordination and communication between applications. And above all the technology must provide comprehensive and flexible protection mechanisms. Paradigms and technological solutions developed for conventional distributed systems may fail to scale to MDCS. Distributed applications designed using the client/server (C/S) paradigm separate software components in two, possibly overlapping, categories: clients and servers. Servers export an interface which describes a set of services. Clients use it to request services. Central to the model is the that the location of computations is an implementation detail which can either be explicitly stated by the programmer as an afterthought or completely hidden in the communication abstractions. At the antipode from the location transparency of C/S, a number of researchers have studied paradigms which make location explicit. Their efforts have focused on computation mobility, i.e. the ability to reconfigure at run-time the bindings between a computation and its computational environment. This work take some of its roots in the concept of Knos [17]. But since its inception this idea has reverberated in different communities, often being reinvented: AI uses the term mobile agent. Industry tried and failed to apply the paradigm to the Internet [20]. Telecommunication researchers have named them messengers [16]. Weaker incarnation have been studied [13][15] [14][10].
2 Distributed Programming Paradigms Four distributed programming paradigms display increasing levels of mobility: client/server (C/S), code on demand(COD), remote evaluation(REV) and mobile computation(MC). We compare paradigms in a scenario where a source host tries to access data of a target, we consider: • Resources: the data as well as physical resources. • Control: the flow of control of the computation. • Code: the code required by the computation. • Data: additional data need for the task at hand. • Execution state: the state of a computation. Table 1 (from [4]) summarizes the differences between paradigms. Notably, C/S has the constraint that code must be pre-installed on the target. COD allows the source to provides code, but otherwise restricts it to being passive waiting to be executed by the target. REV is the union of C/S and COD, the source is active and it provides the code. MC is the most general model as it extends REV with the ability to move computations.
Paradigm client/server code on demand
Source control + data code
Target Mobility code + resources control + data control + code resources remote evaluation control + code + resources control + code + data data mobile computa- control + code + resources control + code + tion data + exec. state data + exec. state
Initiator source target source source
2.1 The Client/Server Distributed Programming Paradigm The Client/Server model structures applications in groups of computations, called servers, that offer services to the user computations, called clients. Clients and servers may run on the same or on different machines, all of their communication being channeled through abstract communication
New Paradigms
3
mechanisms built on top of the basic facilities for sharing an exchanging information of wide area networks. Usually, the overhead of connection-oriented protocols is avoided by basing communication on a simple connectionless request/reply protocol. For most applications a much more convenient mechanism is the, so called, remote procedure call (RPC) which hides all of the network communication machinery at procedure call boundaries. With RPC, a procedure located on another machine can be called just as if it was a local procedure. One of the goals of RPC is to mimic local PC (LPC) semantics. This goal has been criticized on the basis that a RPC is fundamentally different from a LPC and that it is not possible to design reliable and efficient programs ignoring this. Another criticism of most RPC packages is that they do not allow the transfer of complex user-defined data structures, in particular pointer based data structures. This has been solved to some extent by Network Objects [2] and Emerald [9] among others. These two systems also allow remote references. References to objects on other machines. This introduces problems as it increases interdependencies. What are the pros and cons of C/S? An advantage is independence of language, operating system or hardware. Of course, language independence breaks down as soon as complex values are used. A String object in Java is different from a Smalltalk string. Another advantage is hiding distribution from the programmer. This is exactly what Waldo et al. have criticized [19]. The disadvantages of RPCs are threefold: (1) they are connection oriented, (2) they offer no mechanisms for implementing consistent and systematic security policies, and (3) they favor coarse grain interfaces. (1) An interaction may require a number of RPCs the connection has to be maintained during that time. (2) For security each call and reply must be authenticated and protected from prying or modification. Existing RPC packages either ignore the problem or provide mutually incompatible solutions. (3) Each RPC server must exports an interface, the procedures that may be called by clients. The cost of an RPC is high, it includes network communication, authentication, encryption, packaging argument data. So, it is desirable to minimize the number of RPCs needed for any particular task. This leads to coarse grain interfaces. Unfortunately, coarse grain interfaces are not flexible. The alternative is a fine grain interface which decomposes the original interface in a number of smaller units, like going from CICS to RISC. The problem with the fine grain interface is that the cost of each individual call is such that the speed deterioration will be intolerable.
2.2 The Code On Demand Distributed Programming Paradigm Code on demand (COD) is a paradigm for building run-time extendible software. Applications are extended with new functionality by downloading, linking and executing code. This amounts to remote dynamic linking. Although, dynamic linking is a well understood technique, some new twists are introduced as it is necessary to ensure portability across different machine architecture as well as security as the code is not fully trusted. COD has proved successful to extend WWW browsers. The paradigm has been implemented: Java is the most successful, other implementations include an Oberon-based system called Juice, a Caml browser called MMM, and the language independent software fault isolation technology from Calusa [5]. The limitation of COD is that mobility is restricted to code only, thus it is not possible to trigger from a remote host the loading of a code fragment.
2.3 The Remote Evaluation Distributed Programming Paradigm Remote evaluation (REV) lets a node send a request in the form of a program to another node. A computer that receives such a request executes the program in the request and returns the result to the sending computer. Control flows from the client to the remote host and back to the client. The code of the program as well as any required arguments are moved to the target machine. All resources needed during the computation are bound to local resource of the target machine. The name remote evaluation was coined by Stamos and Gifford in 1990 [13]. REV was presented as an extension of RPC which enhanced flexibility and improved performance by reducing network traffic. In some sense the UNIX remote shell program rsh is a simple case of REV. Other systems
4
Jan Vitek
supporting REV include distributed databases, e.g. R*, Falcone’s NCL, PostScript, and the Javabased Mole system [14], and M0 [16].
2.4 The Mobile Computation Distributed Programming Paradigm One last alternative to C/S is to base distributed programming on mobile computations (MC). The idea in a nutshell is to allow running computations to move between computing environments. MC removes the distinction between client and server, both are computations, either one of which may move to the other’s location. With MC the security emphasis must concentrate on protecting computations. Local communication can be cheap, but it must also remain secure. With cheap inter-computation communication fine grained interfaces become practical. Thus changing the programming style. The minimal requirements for MC is support for relocating a computation to another host and the ability to communicate with other computations. Thus, one may imagine a mobile programming language as a traditional programming language with a single additional command, the move primitive, which move a computation to a designated host, and, for instance, a secure shared memory where all computations can exchange data. RPC mechanisms are not necessary. If we want to communicate with a server located on a remote host, then we need only to create a computation that encapsulate the required interaction and to move it to the remote computer Fig. 1. The notion of location explicit. In fact, the concepts of a location and of bindings of a computation to a location must be given semantics. It is essential that computation be allowed to access local resources and be allowed to interact with other running computations collocated on the same node. With this approach we get run-time portability of programs. Programs are ported dynamically, while running, to other machines with a different hardware and software base by the mere fact of moving to that machine. Mobility of computations implies mobility of the code required by that computation.This mobility is under the program’s direct control. How do mobile computations address issues of MDCS? MC is well suited to off-line operation where a mobile user may compose and prepare a computation, connect to the network just long enough to move the computation to the compute server, then connect again to receive the results as a computation. The flattening of the client/server relationship has the advantage of flexibility, for instance, computation can always take place on the more powerful node. A useful property when faced with anorexic hosts. One other advantage of mobile computations is that they reduce the interdependence of systems as every node is self contained. The effect of taking a node off-line is that no existing computation may leave and no new computation may arrive. Mobile computations can be used to perform remote software upgrades in MDCS, a computation may either enhance an existing server or replace it all together. For this application it is mainly the mobility of code that is exploited. At the time of this writing there is no mature MC system available for experiment. The outstanding problems are general inefficiency and lack of flexible security mechanisms [10][14][6][8].
Client
Server fine grain interface
Communication package
Communication package
Execution Environment
Execution Environment
Communication subsystem Fig. 1 A client/server application architecture using mobile computations.
New Paradigms
5
3 Perspectives: Mobile Object Systems A number of MC related research directions have to be investigated: • minimal and portable representation for code and state, • integration of computational environment and local operating system, • efficient and secure inter-computation communication mechanisms, • secure execution of untrusted communicating applications, • lightweight protection domains for computations, • development language for mobile computations. At the University of Geneva we are currently working on mobile computations in the framework of the object-oriented paradigm (SPP Project ASAP). We model mobile computations as interacting mobile systems of objects. Our view is that a mobile object system is a protection domain in which a group of tightly coupled objects is allowed to execute. The semantics of the protection domain ensure that all interaction between different object systems are secure and that resources used by each object system are controlled by the enclosing computational environment. Mobility implies that all ties to a computational environments must be severed, the object system moved to an another environment and the bindings to the new environment reestablished. An implementation of mobile objects should meet the following list of requirements: • mobility of data, code and execution state; • portability and heterogeneity—different operating systems/hardware; • multiple high level languages—e.g. Java, C++, to define mobile computations; • efficient execution model—achieve execution speeds comparable to C code; • efficient use of network resources—minimize the volume of data transfer; • security of the host—protect the host machine from rogue computations; • security of the execution environment—protect the run time system; • security of the computations—protect computations from each other. From a technological standpoint, mobile object systems require a high level programming language in which programmers may describe object-based mobile computations. Programs written in the high level language are run in an executable language, this second language is a suitable low level representation of the high level program, for instance byte code or native code. A third language, the transfer language, is used to move programs. Thus the process of moving computations requires a two step translation first from high level to transfer language (at the source), then the code may move, and finally from transfer to executable language (at target).
high level language Ctrans transfer language Cexe byte code interpreter hardware
native code hardware
Often, either the high level language or the executable language is used for transfer. In the case of Java, for instance, the transfer language and the executable language are usually the same, namely Java byte codes. The first part of the ASAP project uses a Java-based REV system called MOLE developed by the University of Stuttgart [14] and extended for the needs of the ASAP project. MOLE uses Java as a MC development language. The computational environment is the Java run-time extended with a few classes. Although quite powerful, the MOLE platform is inherently limited by its use
6
Jan Vitek
of Java. This causes security, efficiency and expressiveness problems. The second phase of the project consist in the development of a secure intermediate language named SEAL which extends the semantics of Java byte codes with security features needed for mobility. SEAL thus replaces Java byte codes as a transfer language. Our aim is to generalize the design so as to ease compilation of different high level programming languages to SEAL. Thus, SEAL does not prescribe a high level language (although we plan to design one). SEAL differs from existing designs because the concept of protection domain is made explicit in the transfer language. We can guarantee safety, as it is not possible to express unsafe programs in SEAL (the meaning of safety will be defined in greater detail), This choice allows to reason about the safety of mobile code and to optimize much of the dynamic checking at compile time. As a language, SEAL provides many of the standard trappings of object-oriented programming languages. These include: static typing, structural subtyping, bounded parametric polymorphism, dynamic binding, a special form of multi methods. A less conventional choice is that there are no classes and no inheritance. The innovative feature of SEAL is the concept of sealed object A sealed object defines a protection domain. All objects owned by a sealed object are protected by it. Each sealed object executes within a computational environment, all resources used by the sealed object (memory, processing time, etc.) are controlled by the enclosing environment. In SEAL, a computational environment is, itself, a sealed object. Thus recursive protection structures can be constructed easily. Protection is explicit and insecure programs can not be expressed in SEAL. Protection is preserved until the second translation step Cexe, as this step is executed on the local machine by a trusted compiler it is possible to apply a number of safe optimizations to reduce the run-time cost. The results of the ASAP project will include the definition of the SEAL secure intermediate language as well as its implementation.
Bibliography [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20]
J.Bauman, C.Tschudin, J.Vitek, eds., Proc of the 2nd Wshp on Mobile Object Systems, 1996. A. Birrell, G. Nelson, S. Owicki and E. Wobber. Network Objects. Digital SRC R 115, 1994. L. Cardelli. Obliq: A language with distributed scope. Technical report, DEC SRC, May 1995. A. Carzaniga, A. Fuggetta, G.P. Picco, and G. Vigna. Understanding Mobile Code Systems through Classification. Technical report, Politecnico di Milano, April 1996. Colusa. Colusa Software White Paper. Omniware Technical Overview, 1995. D. Dean and D. S. Wallach. Security Flaws in the HotJava Web Browser, Princeton, 1995. R.S. Gray. Agent Tcl: A Transportable Agent System. In Proceedings of the CIKM'95 Workshop on Intelligent Information Agents, 1995. D. Johansen, R. van Renesse, and F.B. Schneider. An Introduction to the TACOMA Distributed SystemV1.0. TR 95-23, University of Tromsø and Cornell University, June 1995. E. Jul, H. Levy, N. Hutchinson and A. Black. Fine-grained mobility in the Emerald system. ACM Transactions on Computer Systems, 6(1), February 1988. F.C. Knabe Language Support for Mobile Agents. Technical Report ECRC-95-36. European ComputerIndustry Research Centre, Munich, Germany, December 1995. J. Markoff. AT&T Plans for Internet Over a $500 Wireless Phone. New York Times. Jul. 12, 1996. OMG. The Common Object Request Broker: Architecture and Specification CORBA, 1995. J.W. Stamos and D.K. Gifford. Remote Evaluation. In TOPLAS, 12(4):537-565, October 1990. M. Strasser, J. Baumann, and F. Hohl. MOLE - a Java based mobile agent system. In [1]. Sun Microsystems. The Java Language Specification, October 1995. C. F. Tschudin. An Introduction to the M0 Messenger Language. U Geneva, 1994. D. Tsichritzis, E. Fiume amd S. Gibbs, and O. Nierstrasz. Knos: knowledge acquisition and manipulation objects. ACM Tran. on Office Information Systems, 5(1):96, 112 1987. R.Wahbe, S.Lucco, T.E. Anderson and S. L. Graham. Efficient software-based fault isolation. In Proc. 14th ACM Symp. on Operating System Principles, pp. 203-216, 1993. J. Waldo, et al.l. A note on distributed programming. Technical report, Sun Microsystems, 1994. J.E. White. Mobile agents. In J. Bradshaw, editor, Software Agents. MIT Press, 1996.
