A User-Centric Approach for Grid Computing - Semantic Scholar

Open Grid: A User-Centric Approach for Grid Computing Walfredo Cirne

Keith Marzullo

Universidade Federal da Paraíba Departamento de Sistemas e Computação

University of California San Diego Computer Science and Engineering

KWWSZDOIUHGRGVFXISEEU

KWWSZZZFVXFVGHGXXVHUVPDU]XOOR

Abstract – The possibility of combining multiple machines geographically dispersed as the platform for the execution of parallel applications is enticing, having sparked considerable research in an area that has been called Grid Computing. Despite intensive research effort, though, those for whom Grid Computing has been aimed have been slow to adopt this new technology. We claim that there are two main reasons for that. First, the user has not been properly supported in her everyday activities in the sense that there is a lack of a cohesive gridwide working environment. Second, current efforts assume that the software they generate will be widely deployed across the whole grid, which is hard to achieve due to the specialized interest in high-performance computation. In this paper, we argue that the massive acceptance of Grid Computing technology depends on building solutions that are open (do not require a particular infrastructure), extensible (ease the addition of refinements) and complete (cover the whole production cycle). We then describe our system, called Open Grid, that implements our viewpoint and targets users of coarse-grain parallel applications. We hope that Open Grid will constitute itself into a step towards a widely used computational grid. Keywords – Grid Computing; Internet Computing; Massively Parallel Processing

I. INTRODUCTION Given the massive number of computers that are networked together, combining the power of thousands of processors seems a natural way to tackle computationally intensive problems. Moreover, projects like SETI@home [12] have shown that some applications can effectively utilize an extremely large number of geographically dispersed processors. (As of February 2000, SETI@home encompassed 1.6 million participants in 224 countries and computes on average at 10 Teraflops.) However, there are a number of technical and administrative issues in turning independent networked computers into a generic production platform for high-performance computing. Addressing these challenges underlies the Grid Computing research area [8]. The metaphor adopted by Grid Computing is the power grid: just as electricity is available upon demand from a power grid, computational power can be made transparently available upon demand from a computational grid. Considerable research has been done in the last few years towards the realization of this vision. Grid Computing infrastructure (which are essen-

tially distributed operating systems that support highperformance applications) such as Globus [7], Condor [11] and Legion [10] have been publicly available for years, and now we starting to see companies such as Entropia [6] whose goal is to commercially provide grid infrastructure. Despite this progress, though, those for whom Grid Computing has been aimed have been slow to adopt this new technology. Even those who have very coarse-grain embarrassingly parallel applications (which are quite amenable to running on a grid) rarely avail themselves of any of the available computational grid infrastructure. We believe that a major reason for this state of affairs has to do with the direction that Grid Computing design has taken. One can argue that Grid Computing research has fallen into two traps that had once waylaid earlier research into distributed operating systems: adopting a closed and inextensible architecture, and not supporting the workflow of developing a distributed application. Regarding this first aspect, current efforts to produce grid infrastructure assume that all machines involved will support specialized grid protocols [9]. In this paper, we argue that the massive acceptance of Grid Computing technology depends on building solutions that are open (do not require a particular infrastructure), extensible (ease the addition of refinements) and complete (cover the whole production cycle). In particular, we maintain that a grid-wide notion of working environment should be centered on the user, instead of dependent on a particular infrastructure, as in current designs. We then describe our system, called Open Grid, that implements our viewpoint and that we hope is a step towards a widely used grid.

II. THE PROBLEM When considering computational grid environments, it is useful to think of three kinds of participants. We'll give them names: Sarah, Bob, and David. Sarah is a programmer who wishes to run a coarse-grain parallel application (i.e., an application whose ratio of computation time per communications time is large). Bob is another programmer who has a fine-grained application. Both Sarah and Bob wish to run their applications on the grid in an effort to compute their corresponding results as quickly as possible. David is a systems administrator who controls some of the machines

in the grid. David’s goal is to make all users happy, but oftentimes he wants to give preference to particular users. Experience has shown that it is hard to develop and deploy infrastructure that simultaneously enables Sarah and Bob to easily use the grid, as well as gives David control over the resources he oversees. Attempts in doing so have not been as widely adopted as originally hoped. In order to more effectively transfer the Grid Computing technology from research labs to end-users, we can first concentrate of targeting less demanding grid users like Sarah. Moreover, Sarah’s problem is a very relevant one. The people who do data mining, massive searches (such as key breaking), parameter sweeps, Monte Carlo simulations, fractals calculations (such as Mandelbrot), and image-manipulation applications (such as tomographic reconstruction) have the same requirements as Sarah. The current Grid Computing infrastructure, on the other hand, has either ignored the differences between Sarah and Bob's requirements or has gravitated towards Bob's since they are more technically challenging. Moreover, by committing to a particular Grid Computing infrastructure (e.g. Condor pools [11], Globus resources [7], Computational Co-ops [4], or Nile farms [1]), both Sarah and Bob must restrict their application to utilize only processors available through such an infrastructure. Bob may be happy to do this, but Sarah might have access to many other processors elsewhere. And, since her application is coarse grained, using whatever processors she can get her hands on might be very fruitful. We believe that a Grid Computing technology Sarah will be willing to use has to be open, extensible and complete. By open, we mean that a solution should not preclude Sarah from using any computing resources she can muster. Her grid may be different than Bob's, or the grid of some other programmer with another coarse grain application. This implies that the Grid Computing infrastructure should provide an open environment in the sense that it cannot expect all machines Sarah can use to support some specialized grid protocol, as suggested in [9]. Of course, available Grid Computing infrastructures can be used, but such infrastructures should not be mandatory. Another was of stating the benefit of an open Grid Computing infrastructure is that it allows Sarah to bring her personal computing environment to the processors of her grid rather than being forced to use an environment imposed by the grid upon her. Extensibility is paramount because of the everchanging characteristics of a grid (such as load and availability) make improving an application’s performance a hard task. Research in the area points to application scheduling as crucial to achieve performance in the Grid environment [2] [12]. Application schedulers aim to improve the performance of the application by evaluating the current state of a grid to (i) select the resources to use, (ii) partition

the work among the selected resources, and (iii) submit the partitioned work to the corresponding resources. Unfortunately, successful application schedulers developed so far are closely coupled to the applications they schedule. This represents a serious obstacle because it frames simplicity and efficiency as mutually exclusive features. A way to address this difficulty is to provide a generic application scheduler as a default, and enable Sarah to change or enhance it as desired. This way, Sarah can start running without worrying about scheduling, and later improve her application's performance when it proves worthwhile to do so. Finally, Sarah’s interface to the grid should be complete. It should support the whole production cycle of the problem it targets, from development to production to maintenance, instead of just focusing on a particular aspect of the problem. In particular, a single set of abstractions should enable Sarah to develop, deploy, debug, and execute her application.

III. OPEN GRID There are two general issues that arise in enabling Sarah to use her grid as a platform for her (coarse-grain parallel) application. First, Sarah needs a grid-wide working environment, i.e. a set of abstractions that enable her to conveniently use her grid, in the same way that files and processes enable programmers to use a single computer. A working environment provides a common denominator that Sarah can rely upon when programming for her grid, despite the differences in the configuration of the multiple resources that comprise the grid. Moreover, a working environment is key in providing a complete solution for Sarah, one that eases managing input and output files, distributing application code, and otherwise carrying on daily computational activities, now based on a computational grid. In Open Grid, the User Agent provides a grid-wide working environment. Second, Sarah needs to manage the tasks that compose her application in a way that promotes the performance of her application despite the heterogeneous and everchanging nature of her grid (which may contain machines of different types, running under different load, and connected through different networks). Moreover, some of these machines may be unavailable at times: they may fail or become unreachable. To manually cope with the resulting complexity would jeopardize the gains Sarah realizes in using the grid. In Open Grid, the Task Manager has the responsibility of coping with this.

A. The User Agent We call the machines that Sarah already uses for her everyday tasks her home machines. The other machines that Sarah uses through Open Grid to farm out tasks for her application are called grid machines. In general, Sarah has good access to her home machines and has set up a com-

fortable working environment on them. The User Agent provides abstractions that make it convenient for Sarah to use the grid machines. The services provided by the User Agent for the grid machines are (i) remote execution, (ii) file transfer, and (iii) the playpen abstraction. Remote execution allows a process running on a home machine to start a process on a grid machine. File transfer allows for the movement of files between home machines and grid machines. Playpens allow Sarah to deal with files and storage over her grid. Playpens can either be temporary or permanent. Temporary playpens provides Sarah with temporary disk space in a manner that is independent of the local file system conventions of a given machines. Temporary playpens are implemented by creating a directory in a file system that can hold the amount of data specified by Sarah. Permanent playpens enable Sarah to distribute permanent files (such as binaries) across the grid. Permanent playpens are implemented as directories rooted on Sarah’s home directories. A grid machine loads files into permanent playpens from the home machines, using simple version number comparison to first check whether locally cached versions are in fact the correct versions. The User Agent services are implemented by the User Agent Daemon, which runs on grid machines, and the User Agent Server, which runs on home machines. Since the User Agent provides security-sensitive services (such as remote execution), the Daemon and the Server rely upon public-key cryptography to authenticate each other as being deployed by Sarah. The Daemon runs with whatever permissions David was willing to grant Sarah for that grid machine. A bootstrapping problem occurs in Open Grid: the Daemons and Servers themselves together comprise a distributed application that needs to be installed and monitored. This is the task of the User Agent Factory. We have built a User Agent Factory as a set of scripts that use FURQ WDE to start Servers and VVK to start Daemons. However, we cannot anticipate which mechanisms will be available for Sarah to access the machines that constitute her grid. We thus expect Sarah to customize the User Agent Factory to meet the needs of her grid; making it feasible for her to accommodate whatever new problems she faces in getting a connection to a new set of machines. A somewhat unusual feature of the User Agent is that it contains code provided by the user (namely, Sarah is expected to customize/write the User Agent Factory). That is necessary to make Open Grid an open environment. Since we cannot anticipate all that Sarah might have to do in order to access a given resource, Sarah contributes with code that customizes the Open Grid for her purposes. To keep things as simple as possible, Open Grid is designed to keep small the amount code that Sarah has to provide. But, of

course, Sarah can replace or extend any component of the system as she pleases.

B. The Task Manager The Task Manager, which runs on a home machine, is the Open Grid part of Sarah's application. Recall that Sarah’s application consists of a set of tasks that can be executed independently from each other. Sarah informs the Task Manager which tasks compose her application by sending the RJDGGWDVN message to the Task Manager. The Task Manager also provides control commands for Sarah to pause, kill, and monitor applications and tasks. Each task is composed by two executables (or scripts): the home task and the grid task. The home task runs on a home machine. The Task Manager invokes the home task when a grid machine becomes ready to be used. The name of the grid machine is passed to the home task through the 2*B352& environment variable. The home task performs all necessary set-up activity (such as creating and mounting playpens, transferring files, etc), remotely executes the grid task on machine 2*B352&, performs any finishing up activity (such as collecting results and deleting temporary playpens), and then informs the Task Manager that the task is completed by sending the RJWDVNGRQH message to it. Figure 1 shows a simple home task written in shell script, in which messages are sent to the Task Manager through the invocation of Open Grid commands. Note that RJFUHDWH SOD\SHQ, RJKRPHJULG, RJJULGKRPH, and RJUHPRWH H[HF are supported by the User Agent. KRPHWDVNVFULSW 7DVN3DUDPV µµ 7DVN,QSXWV µµ 3OD\3HQ CRJFUHDWHSOD\SHQ2*B352&C RJKRPHJULG7DVN,QSXWVJULGWDVN3OD\3HQ RJUHPRWHH[HF2*B352&JULGWDVN7DVN3DUDPV

∗

RJJULGKRPH3OD\3HQUHVXOW aUHVXOWGLU RJWDVNGRQH2*B352&

Figure 1: A simple task script The grid task runs on a grid machine and performs the task per se. Figure 2 depicts the actions that place in executing a task with Open Grid. Circles indicate components that are supplied by Open Grid and squares components that Sarah is expected to write. Two main concerns of the Task Manager are scheduling and fault recovery. In our context, scheduling consists of selecting which machine executes each task. What makes this non-trivial is that the machines that compose Sarah’s grid are not only different but also their availability varies in time due to the load generated by other users. In particular, we want to avoid assigning the last tasks in an application to slow or loaded machines, as such assignment largely increases the execution time of the application as a whole.

There has been work addressing this problem, which is primarily based on monitoring resources throughout the network [2] [14]. Unfortunately, such a monitoring depends upon Grid infrastructure that, by design, we want to avoid. We cope with this problem by replicating the last tasks of an application among many machines. This way, the unfortunate assignment of a task to a slow or loaded machine can

be neutralized by the latter assignment of the same task to another machine. Note that this strategy is only possible because tasks are independent. Likewise, task independence offers us a very straightforward way to deal with fault recovery. Failed tasks are simply restarted in the next available machine.

Home Machine Grid Machine

add-task (1) Task Manager task-done (4)

grid task

remote exec (3)

(2) home task

(3c)

(3b) User Agent Daemon

User Agent Server

playpen, file xfer, and remote exec (3a)

Figure 2: Sequence of events for running a task

IV. THE OPEN GRID PROTOTYPE We have implemented a first prototype of Open Grid. The prototype is somewhat simpler than what described in the previous section (in particular, the User Agent is implemented through VVK/VFS and has less functionality). Yet, it allows us to investigate the efficacy of our approach. We used the Open Grid prototype to run most of the simulations discussed in [5]. In total, we conducted around 600,000 simulations during a 40-day period, using 178 processors located in 6 different administrative domains (4 at University of California San Diego, 1 at San Diego Supercomputer Center and 1 at Northwestern University). The processors were in normal production (i.e., they were not dedicated to us at any point in time). The processors were either in Intel machines running Linux or in Sparc machines running Solaris. Using Open Grid, the 600,000 simulations took 16.7 days, distributed over a 40-day period (the remaining time was used to analyze the latest results and plan the next simulations). In contrast, our desktop machine (an UltraSparc running at 200MHz) would have taken about 5.3 years to complete the 600,000 simulations (had it been dedicated to only that task). Nevertheless, even more important than the achieved speed-up is the fact that we were able to use everyday machines located in different administrative domains as the platform for our application. In fact, note that the machines we used shared no common software except ubiquitous Unix utilities such as HPDFV, VVK, and JFF. In particular, Grid Computing software (more precisely, Globus [7]) was installed only in a single administrative domain. Moreover,

access mechanisms varied from one administrative domain to another. For example, we had to cross a firewall for the machines in one of the domains. Also, we were required to run at lower priority in four of the administrative domains. This lack of deployed infrastructure and the corresponding diversity in access mechanisms reinforced our impression that a solution for Sarah must be open and complete, allowing her to use whatever resources she has access to. Moreover, the grid-wide working environment provided by the User Agent freed us from worrying about which software and data was updated across the different administrative domains. We feel this was key to enable us to focus on our simulations (out final goal at that moment), making using our grid a productive endeavor.

V. RELATED WORK There has considerable activity over the last years in creating infrastructure to enable grid computing. Projects like Globus [7] and Legion [10] aim to provide comprehensive support for grid computing, a goal that is starting to be pursued commercially by companies such as Entropia [6]. Other Grid Computing projects have more focused goals, targeting high-throughput applications (as Condor [11]) and high-energy physics applications (as Nile [1]), for example. Yet other efforts have addressed specific aspects of the Grid Computing infrastructure, such as supporting the federation of independent sites into large-scale grids, as the Computational Co-op [4]. All these projects view Grid Computing infrastructure being deployed as universally available system services.

Open Grid, in opposition, adopts a user-centric approach for providing Grid Computing services. User-centric approaches are recognized as the best strategy for scheduling in grids, in what became known as application scheduling [2] [8] [12]. Open Grid takes the lessons of application scheduling a step further by complementing them with (i) a working environment that enables Sarah (our archetypical user) to conveniently use her grid in all phases of the production cycle, and (ii) a default scheduler that makes it possible for Sarah to start using her grid without investing time and effort to deploy a customized application scheduler. Open Grid is similar in concept to systems like SETI@home [12], Everyware [15], and APST [3]. These systems deploy grid applications that carry their own schedulers, create their own grid-wide abstractions, and run over a variety of system-centric infrastructures. Similarly, Open Grid enables Sarah to build a system with these three characteristics. Open Grid differs from systems like SETI@home, Everyware and APST in that such systems are tightly coupled with the applications they support, providing specific solutions for their application. Open Grid, on the other hand, is designed as a framework that Sarah uses to build her grid application. Open Grid allows Sarah to have her application running with the minimum of effort in dealing with grid concerns, while not precluding Sarah from putting such effort in when she deems worthwhile.

VI. CONCLUSIONS The Computational Grid is based on the ideas of distributed heterogeneous operating systems and on the needs of programmers writing large scale, high performance parallel applications. Up to now, though, most of these programmers have not benefited from Grid Computing. We feel that the current Grid Computing infrastructures have not served in this capacity because they have lacked being an open, extensible and complete solution. This paper introduces Open Grid, a Grid Computing solution designed to be open, extensible and complete and thus make possible deploying Grid Computing to the users it is intended to serve. Open Grid is open in the sense that it does not require (although can use) any infrastructure to be installed throughout the Grid. There are promises that systems like Globus [7] and companies like Entropia (which commercialize SETI@home-like infrastructure) [6] will provide a grid with a massive scale. But it will be some time (if ever) before one of these becomes dominant. Until then, an open user-centric approach is needed to allow a programmer to define her own grid with whatever computers she can gain access to. Open Grid is extensible in the sense that it makes it possible for a user (who we name Sarah) to customize any part of the system, enabling Sarah to take advantage of specialized knowledge of her application in order to improve performance. Yet, Open Grid implements sensible defaults

wherever they are possible, greatly reducing the gridrelated effort Sarah has to carry on to start running her application. Open Grid is complete in the sense that it supports all activities Sarah has to perform to effectively use her grid, from development and debugging, passing through deployment and execution, to result collection and maintenance. This is accomplished by providing a working environment that enable Sarah to view and reason about her grid as a whole. Open Grid is not a completely generic solution, though. Open Grid assumes Sarah to have a coarse-grain parallel application. While this assumption makes Open Grid of no value for a user (who we name Bob) with a fine-grain tightly-coupled application, there are a large number of applications that match Sarah’s requirements. We focus on Sarah (instead of on Bob) because her requirements are simpler, which makes a comprehensive solution for Sarah simpler than one for Bob. We see solving Sarah's problem as the natural first step for the mass deployment of Grid Computing technology.

REFERENCES [1] A. Amoroso, K. Marzullo, and A. Ricciardi. Wide-Area Nile: A Case Study of a Wide-Area Data-Parallel Application. ICDCS’98 – International Conference on Distributed Computing Systems. May 1998. [2] F. Berman, R. Wolski, S. Figueira, J. Schopf, and G. Shao. Application-Level Scheduling on Distributed Heterogeneous Networks. Supercomputing’96. [3] H. Casanova, G. Obertelli, F. Berman, and R. Wolski. The AppLeS Parameter Sweep Template: User-Level Middleware for the Grid. Supercomputing’2000, Nov. 2000. [4] W. Cirne and K. Marzullo. The Computational Co-op: Gathering Clusters into a Metacomputer. In Proceeding of IPPS/SPDP’99, April 1999. [5] W. Cirne. Using Moldability to Improve the Performance of Supercomputer Jobs. Ph.D. Thesis. Computer Science and Engineering, University of California San Diego, 2001. [6] Entropia Web Page. KWWSZZZHQWURSLDFRP [7] I. Foster and C. Kesselman. The Globus Project: A Status Report. Proc. IPPS/SPDP '98 Heterogeneous Computing Workshop, pg. 4-18, 1998. [8] I. Foster and C. Kesselman (editors). The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann Publishers. July 1998. [9] I. Foster, C. Kesselman, S. Tuecke. The Anatomy of the Grid: Enabling Scalable Virtual Organizations. To be published in Intl. J. Supercomputer Applications, 2001. KWWSZZZJOREXVRUJUHVHDUFKSDSHUVKWPO

[10] A. Grimshaw, A. Ferrari, F. Knabe, M. Humphrey. Legion: An Operating System for Wide-Area Computing. IEEE Computer, 32:5, May 1999: 29-37. [11] M. Litzkow, M. Livny, and M. Mutka. Condor - A Hunter of Idle Workstations. Proceedings of the 8th International Conference of Distributed Computing Systems, pages 104-111, June 1988. [12] SETI@home Web Page. KWWSZZZVHWLRUJVFLHQFHVHWLDWKRPHKWPO

[13] J. Weissman. Gallop: The Benefits of Wide-Area Computing for Parallel Processing. Journal of Parallel and Distributed Computing, Vol. 54(2), November 1998. [14] R. Wolski, N. Spring, and J. Hayes. The Network Weather Service: A Distributed Resource Performance Forecasting Service for Metacomputing. Journal of Future Generation Computing Systems, 1999. [15] R. Wolski, J. Brevik, C. Krintz, G. Obertelli, N. Spring, and A. Su. Running EveryWare on the Computational Grid. Supercomputing’99. 1999.