Agora: Grid Community in Vega Grid Hao Wang, Zhiwei Xu, Yili Gong, Wei Li Institute of Computing Technology, Chinese Academy of Sciences Beijing 100080, P.R. China
[email protected], {zxu,gongyili,liwei}@ict.ac.cn
Abstract. In grid environment, how to organize, discover and use grid resources is one of the basic research problems. In this paper, we introduce a new concept called agora, which refers grid community in our Vega grid project. We propose a principle on agora as one of the basic design principles of Vega grid, which is the community principle. We give a detailed definition of the agora as a 4-tuple（S, O, C, P）, representing the set of Subject, Object, Context, and Policy respectively. Our research methodology is comparative studies, drawing an analog between the grid and other platform systems such as OS, CPU, and MPI. We also study some properties about the agora, for example, grid law of inertia. This agora approach could be used to unify the grid invariability and dynamic variability effectively, develop the appropriate technology to support the dynamic and open properties of the grid, and deal with the complexity in the grid more efficiently.
1. Introduction The research work in this paper is a part of Vega Grid (one research project of the Institute of Computing Technology, Chinese Academy of Sciences). Vega is an acronym, which represents Versatile Services, Enabling Intelligence, Global Uniformity, and Autonomous Control. In Vega, three properties of the grid system we summarized are diversity, variability, and heterogeneity. Diversity: The grid system is a multi-person-multi-computer society, including various kinds of subjects and objects. Variability: The society is not stable or close, but dynamic and open. Heterogeneity: The heterogeneity in the grid represents various kinds of subjects, objects, software platform, hardware platform, use mode, administration mode, context, policies, protection mode, exception handling mode, safe mode, etc. The architecture of the grid is changing. Grid doesn’t appear the clear architecture and no widely known theory model like RAM and Von Neumann Model. Hence, we have to face the challenges of the complexity brought by these three properties. Further more; one of the core aims of Vega Grid is to enhance the usability by Enabling Intelligence. We hope the grid is easy to use like Web so to enlarge the
quantity of the users and the developers. In [7], we propose EVP model of grid resource space with three layers. The EVP model is helpful to implement the grid community. This paper will focus on discussing the community principle, especially the concepts and technology to implement the community. In section 2 of this paper, we will discuss the community principle in the Vega Grid. From section 3 to section 5, we will define the agora and discuss it in detail. The agora correlates with GSML [5], the grip [6], and EVP resource space [7]. In section 3, we will propose some assumptions and give a detailed definition of the agora. In section 4, we will discuss the implementation of the agora and give an example. In section 5, we will conclude and prospect the further research.
2. Review the community principle in Vega Grid The community principle in Vega Grid is: “The community being composed of the user and the resource, makes the user describe the requirement more abstractly, reduces the requirement of the degree of the knowledge structure, reduces unboundedness to boundedness, reduces inordinance to orderliness, and implements the enlarging effect by the context, then empowers user.” The community principle is a method to solve the open and out-of-order problem in the grid. The principle is based on the following two assumptions. (1) The finite assumption: Although the requirements of all users can be infinite and open in whole time, the requirement of one user is finite and close in a given moment or period of time. (2) The iceberg assumption: The orderly, bounded, and persistent group hides great energy, then can bring the enlarging effect which can’t be brought by a simple set of individuals. In a nutshell, the community is a 4-tuple ( S , O, C , P ) . “S” represents the set of “Subject”; “O” represents the set of “Object”; “C” and “P” represent context and policy respectively. The context and policy in one community shouldn’t have different meanings or be paradoxical. The policy is chosen intentionally and explicitly, but the context is formed naturally and implicitly. If we regard the community as the iceberg, the context is the part hiding under the water, which is the reason providing the enlarging effect. The community is somewhat like the group in Distributed Operating System, the VO (Virtual Organization), and the file system in Operating System. But the group and VO are usually the sets of the same kind of things, such as the subjects or the objects, and the community is usually the set of the two different kinds of things, such as the subjects and the objects. But because the VO has not been defined in detail by now, to avoid arguments, we may define the community as the detailed definition or the detailed implementation of VO.
The community makes the users describe the requirement more abstractly. In Vega Grid, the resources that can be used directly by users are those in the GSML pages, being called the effective resources; the resources in the community are the virtual resources; and the actually local resources in each grid node are the physical resources. In addition, as the address space technology and virtual address technology in the computer architecture, the abstraction is basic and essential to reduce the degree of the complexity. The community reduces the degree to the requirement of the knowledge structure. If we distinguish the users as the managers, the secretaries, and the engineers, the community and the GSML SDK can reduce the requirement degree of the end users’ knowledge structure. The managers can use the GSML pages directly, and the secretaries can compose the GSML pages. Only when the physical resources in the grid node must be converted to the virtual resources in the community, the engineers are necessary. The community reduces inordinace to orderliness. To a single end user, Internet, Web, and Grid are usually unbounded and out-of-order. With the community technology, what the user can see is a bounded and orderly sub grid which is far smaller. It reduces the complexity, enhances the usability, and then improves the efficiency. The community implements the enlarging effect. The community can use the information, the knowledge, and the processes which are implicit in the context and the policy, to complete the detailed work and keep the consistence. So the community implements the enlarging effect that is the work can be completed when the user only gives a little information.
3. Assumptions and properties about the grid and the definition of the agora In this section, we list some assumptions and some properties. From now on, we use “agora” represent the community in Vega Grid. 3.1. Grid Assumptions and the properties about Agora Assumption 1. There is only one grid. In this paper only one grid is considered. Property 1. (Grid Law of Inertia). Subjects, objects, and communities in the grid keep stable until some exterior factor (usually subject) changes them by certain means. This property says that there are no spontaneous and casual behaviors in the grid. Any change results from some exterior event, and the occurrence of the event is due to the action of some subject in a certain way. In the one-person-one-computer
world, it is easy to understand this. When we use application software, such as the word processor, the law of inertia is assumed potentially: a text will not be changed spontaneously. But in the multi-person-multi-computer open society, this does not go without saying. For example, when a user is browsing a database, he may find the data changed, though he does nothing, because the change is caused by another user. So the law of inertia should be stated explicitly. 3.2. Exterior Characteristics of Agora Assumption 2. Except possibly in the lowest level (physical layer), the agora is an indispensable and user-visible basic concept. In virtual layer and effective layer, the agora is visible to users, but it is not necessary for the users to use the agora explicitly. For example, the user only need mention the agora in the head part of a GSML page; the other possibility is that there is a default (so implicit) agora in the whole grid. This assumption is very strict. The agora is indispensable, which is like the communicator in MPI. Assumption 3. The agora is a persistent entity. This means that the agora must be kept in a persistent storage. In this point, the agora is similar to the file system, rather than the communicator in MPI. Assumption 4. The grid has a unique agora set. This means that every agora has a unique identity in the whole grid. Assumption 5. (Specific agora law of inertia). An agora has stability and dynamic. This is like a configuration file. The agora keeps stable (invariable) until some subject changes it by certain means. What is changed involves the interior characteristics of the agora (see to later), including changing our elements of the agora (subject, object, context and policy), or add/delete agora members, etc. Assumption 6. (Grid agora law of inertia). The agora of grid has stability and dynamic. This is similar to a file system. The agora set keeps stable (invariable) until some subject (or grip) change it by certain means. The change includes creating, converging and splitting communities. 3.3. Interior Characteristics of the Agora Definition 1. A agora A is a 4-tuple A=（S, O, C, P）, where S is the set of subjects, O is the set of objects, C and P are the sets of context and policy shared by the agora. We define the agora as the community in Vega Grid. Property 2. Every element of (S, O, C, P) is a abstract data type. The operations includes: add a new element (add), delete an element (delete), and update an element (update). Every operation is atomic. Every operation keeps the consistency of the
agora . The idea of property 2 is that operations on an agora should be simple, interleaved semantics and every operation makes the agora transfer from a good state to another good one. The operations in property 2 are the exterior actions in the law of inertia.
4. Implementation Consideration on Agora The main data structures of the agora implementation (visible to the exterior) are the context set C and the policy set P. And the main active component is the agora engine E. In the effective layer, subjects uses object oe =< Qe , Z e , De > . The agora automatically searches the corresponding object in the physical layer o p =< Q p , Z p , D p > , and the context set C and the policy set P are the parameters to the searching procedure. The searching procedure is carried out by the agora engine. At present, the agora engine in the resource locating of the Vega Grid adopts two technologies: dynamic deploying and routing addressing. In the Vega Grid the searching procedure is fully distributed, which is similar to distributed Google.
Fig. 1. The implementation of the agora
Besides the searching procedure, the grid agora should provide some functions, such as agora maintenance, automatic update and exceptional handling. We can refer to figure 1. Now we give an example that implement a grid service by the agora and appropriate technology. A user asks the grid for the matrix multiply services, which is matrixC = matrixA×matrixB . The resource is denoted as a 3-tuple < Q, Z , D > , where Q is the identification, Z is the parameters, and D is the specification. The user gives the information: (1) scientific computing; (2) matrix multiply; (3) the
source is the matrix A, and the matrix B; (4) the destination is the matrix C; (5) the specification is the least time. In this example, the user even doesn’t know A＝
1 8
7
2 ，B＝ 9 5
4
0 ，C＝ 6 0
0
.
0
In the effective resource space, the resource is denoted as a 3-tuple < Qe , Z e , De > , which is < {Scientific Computing, Matrix Multiply}, {Matrix A, Matrix B, Matrix C}, {the least time} >. The user can choose the information through the buttons in the GSML page, which is described as following: Scientific Computing MatrixMultiply Matrix A, Matrix B, Matrix C The Least Time By the Mapper between effective resource space and virtual resource space, the resource is mapped into < {100, 1000}, {10.0.2.58/working/MA.dat, 10.0.2.60/working/MB.dat, 10.0.2.68/working/MC.dat}, {99} >. In this 3-tuple, “100” represents the agora code of the scientific computing agora. “1000” represents the matrix multiply service code in the scientific computing agora. “100:1000” represents the exclusive service in the whole grid. “99” represents the policy code of the least time. By the Mapper between virtual resource space and physical resource space, we can get the physical resource. The grid service node providing the matrix multiply service is found. It is 10.0.2.65. The service is MPIMMService, representing the matrix multiply service. The directory is http://10.0.2.65:8080/ogsa/services/sample/mpi/MPIMMService. The virtual resource space transfers the matrixes to the physical resource space through the SOAP. The matrixes are put into the working directories. The matrixes are 10.0.2.65/now/MA.dat, 10.0.2.65/now/MB.dat, and 10.0.2.65/now/MC.dat. The physical resource is also , which is < denoted as a 3-tuple < Qp , Z p , Dp > {http://10.0.2.65:8080/ogsa/services/sample/mpi/MPIMMService}, {10.0.2.65/now/MA.dat, 10.0.2.65/now/MB.dat, 10.0.2.65/now/MC.dat}, {99} >. The grid service provides the interface MPIMM. The description about the interface is: The method mpi_multiply implements the matrix multiply service. It has three parameters (three matrixes). The process of transferring the matrixes is described as: POST /ogsa/services/sample/mpi/MPIServiceHTTP/1.0 Content-Type: text/xml; charset=utf-8 Accept: application/soap+xml,... User-Agent: Axis/1.1 Host: 10.0.2.65 ... 1 7 8 9 2 4 5 6 0 0 0 0 ... The string “in0”, “in1” and “in2” represent the matrix A, B, C respectively. The code of the method mpi_multiply is: public class MPIMMImpl extends PersistentGridServiceImpl{ public void mpi_multiply(String a, String b, String c){ String cmdarray[]={"mpirun -np 2 cmm",a,b,c}; try { Runtime.getRuntime().exec(cmdarray); } catch (IOException ex) { } } } cmm is the C program of the matrix multiply. The running environment is mpi-1.2.5.
5. Conclusions and Future Works As early as in 1979, Licklider proposed the idea of Multinet, and he indicated that the human society might come into a non-centralized controlled electronic commons, whose mode of operations would be one featuring cooperation, sharing, meeting of
minds across space and time in a context of responsive programs and readily available information. The grid agora is the development and embodiment of this idea. As the developing of the research, we would find that the grid agora technology is indispensable, and the benefits it brings are as follows: Reducing cost, especially the cost of using and developing; increasing online scalability; increasing capability of deep interactivity; decreasing the requirement on users, and accordingly extending the range of application.
References 1. I. Foster, C. Kesselman, J. Nick, S. Tuecke, “Grid Services for Distributed Systems Integration”, IEEE Computer, 35 (6), 2002, pp. 37-46. 2. K. Hwang, Z. Xu, Scalable Parallel Computers: Technology, Architecture, Programming, McGraw-Hill, New York, 1998. 3. J.C.R. Licklider, “Computers and Government”, in The Computer Age: A Twenty-Year View, M.L. Dertouros and Joel Moses (Eds.), MIT Press, Cambridge, Mass., 1979, pp. 88-129. 4. Z. Xu, W. Li, “Research on Vega Grid Architecture”, Journal of Computer Research and Development, 2002, 39 (8) 923-929. 5. Z. Xu, W. Li, D. Liu, H. Yu, B. Li, “The GSML Tool Suite: A Supporting Environment for Userlevel Programming in Grids”, Proceedings of PDCAT 2003. 6. T. Liu, X. Li, W. Li, N. Sun, Z. Xu, “Notes on a Runtime Construct for Grid”, Journal of Computer Research and Development, 2003, 40 (12). 7. W. Li, Z. Xu, “A Model of Grid Address Space with Applications”, Journal of Computer Research and Development, 2003, 40 (12).
