The nodes of Dawning 3000 use Power3-II and PowerPC. RS64-III microprocessors. It uses 64 computing nodes for high-performance computing tasks and 6 ...
Dawning Superservers and the Vega Grid Zhiwei Xu, Ninghui Sun, Wei Li, Taoying Liu Institute of Computing Technology Chinese Academy of Sciences Beijing, 100080, China ABSTRACT
In this paper, we outline grid computing research projects in China, focusing on the research efforts in developing Dawning superservers and the Vega Grid at Institute of Computing Technology, Chinese Academy of Sciences. In particular, we discuss the objectives, the design principles, and some key technologies of these projects. A common theme of these discussions is how to make a technology “grid enabling” or “grid enabled”. Keywords
Dawning Superserver, Vega Grid, GSML, grid community INTRODUCTION
Chinese scientists started research in grid technology in 1996. In 1999, a one-year national research project was launched to develop a China computational grid, under the sponsorship of the Ministry of Science and Technology (MOST). In 2002, MOST launched the second national grid project, which will last four years (2002-2005). Other central government agencies as well as some regional government bodies in China are initiating their own grid research projects for the next three years. For instance, Chinese Academy of Sciences is developing a Science Data Grid, which will eventually evolve into an eScience grid. The Ministry of Education is building a higher education grid, focusing on supporting universities faculties and students by sharing computing resources and information resources. Beijing and Shanghai city governments are considering utilizing grid technology to improve productivity in their manufacturing industry and biological industry. In this paper, we focus on the national grid projects sponsored by Ministry of Science and Technology, under its high-tech research and development program, popularly called the 863 Program. In particular, we discuss in more details the Dawning superservers and the Vega Grid research projects, which are key components of the 863 national grid projects. The rest of the paper is organized as follows: In Section 1, we give an overview of the MOST sponsored national grid research projects. In Section 2, we discuss the development of Dawning superservers, including Dawning 4000
systems, which are grid enabling clusters and will be installed at users sites in 2004. The Vega Grid project will be discussed in Section 3, including the VEGA design principles, the GSML software suite, and the concept of grid community. At the conclusion of this paper, we also point out how our grid research efforts are related to the international grid research community. 1. CHINA GRID PROJECTS
In 1999, MOST launched the first computational grid project in China [9]. Institute of Computing Technology (ICT), Chinese Academy of Sciences, was a leading research team of this national effort. The project was aimed to accomplishing the following design goals: to build a national-scale grid test-bed supporting high performance computing; to design and implement a grid system software suite; and to develop illustrative applications running on the grid test-bed. In December 2000, ICT and more than 30 partners established the first computational grid in China. This grid system consists of 10 nodes in seven cities using domestic high-performance computers, with computing power ranging from 10 Gflop/s to 300 Gflop/s. The grid system software provides single system image and a uniform API to users, who can submit batch jobs to the grid, instead of a particular node system. The grid software also provides functionality such as grid resource information service, account management, resource management and security management. The applications range from advanced manufacturing, weather forecasting, numerical wind tunnel, oil reservoir simulation, particle physics research, bioinformatics, terrain analysis in natural resource study, and scientific databases. Recently, MOST launched the second grid research project, to build a China Grid system by year 2005. This project is significantly different from the first one in the following four aspects: •
The applications developers and users in all walks of life in China gained more knowledge and insights of grid technology, and therefore are very enthusiastic in joining into grid development. Consequently, the application orientation of the China Grid project is changed from mainly scientific research and engineering simulation to broader ranges of services. The China Grid is not just a computing grid any more.
•
It is a service grid offering computing services, data services, information services, etc.
fully operational service grid capable of resource sharing and collaboration, with application grids running.
The development methodology has changed to an open one. For instance, the grid software in the first China computing grid was internally developed almost from scratch, based on LDAP. The new China Grid will utilize open standards and technologies such as OGSA/Globus [1] [2] and Web Services [3]. We believe that, just like Internet, there should be one global grid, of which China Grid is a part.
2. DAWNING SUPERSERVERS
•
From our experiences in grid development and interaction with users, we realize that grid should not be just an infrastructure, but it must provide corresponding interfaces that enable users to program and use the grid effectively. Therefore, the China Grid project must pay special attention to grid interface.
•
The entire grid system is reengineered to suite the above needs, as shown in Figure 1. Exemplar grid applications are selected from four fields: scientific research, environment and natural resource sector, manufacturing industry and service industry. The nation-wide distributed grid nodes will be equipped with grid enabling high-performance computers. All applications must be accessible by grid users from all over China as grid services.
Science
Environment
Grid User Interface
Manufacture
Service
Grid Developer Interface
Grid System Software
Node
Node
Node
Node
Node
Internet Figure 1. Architecture Overview of the China Grid The roadmap of the China Grid project is as follows. In middle 2003, all grid nodes and grid applications will be clearly defined, and they will be put to use over the Internet offering point-to-point services. In fall of 2003, the beta version of the grid software will be installed, creating a uniform grid test-bed. In fall of 2004, grid-enabling highperformance computers will be added to form a highperformance (6-10 Tflop/s) grid platform. 2005 will see a
Dawning is the brand of high-performance computing systems developed at ICT’s HPC arm of the National Research Center for Intelligent Computing Systems (NCIC). NCIC started the research and development of high performance computers in 1990. At the first half of 1990s, the main focus of NCIC was supercomputers based on SMP (Dawning 1) and MPP (Dawning 1000) technologies. After 1995, NCIC turned its research direction to cluster-based superservers, to better take advantage of market demands and Moore’s law. Hundreds of Dawning superservers are now used in scientific computing, engineering simulation, network information services, and business applications in China. The current main research thrust of NCIC is to develop grid enabling cluster superservers [6]. 2.1 Dawning 3000
A key component of the first China computing grid is Dawning 3000 superserver, which employs a SMP cluster architecture and a message-passing programming model. The 403-Gflop/s system provides 70 nodes with 280 microprocessors, 168 GB memory, and 3.6 TB disks. It consists of 10 cabinets and a system console. The nodes of Dawning 3000 use Power3-II and PowerPC RS64-III microprocessors. It uses 64 computing nodes for high-performance computing tasks and 6 service nodes for I/O service, login service, network service and database service. Dawning 3000 is equipped with five separate networks: a system area network for high-speed communication; an internal network for system management and TCP/IP communication by the cluster operating system; an external network for user accesses; and two networks for system monitoring and fault diagnosis. The system area network has three options: a Myrinet by Mricom, a NCIC-designed 2-D mesh, or a Gigabit Ethernet. The architectural principle of Dawning 3000 is SUMA, for Scalability, Usability, Manageability, and Availability. The system software of Dawning 3000 includes parallel programming software, a cluster file system, and a cluster management system. Since its announcement in the end of 2000, several Dawning 3000 systems have been installed in users sites and are currently operational. A successful use case is at Beijing Genomics Institute, where Chinese bio-scientists used two Dawning 3000 systems in two cities to successfully generate the draft sequence of rice genome. The research result was published as a cover article in the April 2002 issue of Science [5] [10]. 2.2 Dawning 4000
Dawning 4000 is an on-going research project, as a key component of the China Grid project. The first Dawning
4000 system is expected to be shipped in spring of 2003, and a more advanced version will start to ship by fall of 2004. Dawning 4000 will support both AIX and Linux operating systems. Grid enabling is the key technical direction of Dawning 4000. To achieve this vision, Dawning 4000 adopts a gridenabling cluster architecture. Previous cluster operating system in Dawning 3000 is reconstructed to be service oriented. The new cluster operating system in Dawning 4000 is structured as 4 tiers: cluster middleware, cluster public services, cluster functions, and cluster user interface. The cluster operating system is not a simple collection of management components any more. Horizontally, component functionality is partitioned into nonoverlapping sets. Vertically, logical function is hierarchical constructed, with common functions abstracted for sharing, and modules closely tie to the underlying node platform are extracted out for easy migration. From the user’s viewpoint, application functions are realized through a public infrastructure and collaboration between modules. The architecture of Dawning 4000 is outlined in Figure 2. We inherit the SUMA features from Dawning 3000 as design principles of Dawning 4000, but add 4 more features: Service, Security, Specialization and Intelligence.
The Vega Grid is a research project conducted at Institute of Computing Technology, Chinese Academy of Sciences. It aims to learning fundamental properties of grid computing, and developing key techniques that are essential for building grid systems and applications. The Vega Grid team currently consists of more than 150 people, and is conducting research work in the following areas: •
Dawning 4000 Superservers: Terascale grid enabling clusters on Linux/Intel and AIX/PowerPC platforms.
•
Vega Grid Software Platform: This work includes research on grid system software, grid application development tools, and grid user interface. The objectives are to enable resource sharing, collaboration, service composition, and dynamic deployment, utilizing open standards such as OGSA, Globus and web services.
•
Vega Information Grid: Research on enabling technology for information sharing, information management, and information services in an ASP environment or a wide-area enterprise environment.
•
Vega Knowledge Grid: Research on knowledge sharing, knowledge management, and knowledge services in a wide area Web environment [11].
3.1 The VEGA Design Principles
To Internet Grid Resource Router
3. THE VEGA GRID PROJECT
Application s Servers Nodes
The Vega Grid development is guided by the following four design principles, abbreviated in total as VEGA [8]: •
Versatile Services. The grid should have the ability to support various services. That is, the grid should be constructed as an infrastructure to provide a developing and operational environment supporting various applications, various use patterns and various platforms. All applications and interactions should use the service mechanism.
•
Enabling Intelligence. The grid should have the ability to support intelligent computing. The grid should be more intelligent than the Internet and it can provide the automatic production of information, knowledge and services. The grid itself is not the subject that provides the intelligence but it can assist people to develop intelligent applications.
•
Global Uniformity. From the viewpoint of users, the grid should be a single virtual computer. The grid should supply Single System Image (SSI), such as Single Sign-On and other related technologies.
•
Autonomous Control. The grid should not be governed by a central administration. All components in the grid can freely join or leave at their own will. For members providing the resources, they have the full right to control their own resources exported; and for members using the resources, they have the right to use resources as they like within the purview of their rights.
Internal Networks Storage Servers Nodes
Management and Control Nodes
Figure 2. Dawning 4000 Architecture Overview The Dawning 4000 strives for a balance between a generalpurpose architecture and specialized applications. A key technology to achieve this goal is dynamic deployment of functionality (DDF), whereby systems and applications functions are not pre-installed, but deployed dynamically to match the users’ needs. A new architectural component in Dawning 4000 is the grid resource router. It differs from a network router in that it does not just route messages, but provides a link to other grid nodes and to the users. The main function of the grid resource router is to connect grid resources together at the applications level.
3.2 The Architecture of the Vega Grid [7]
A three-layer architecture of the Vega Grid is illustrated in Figure 3. At the grid hardware layer, we are developing Dawning 4000 superservers, which are clusters with enabling technology to support grid platforms and applications. Other components at the grid hardware layer include a grid client device and a grid resource router. The Vega Grid Client is an easy to use client device for grid users. The Vega Grid Router [4] enables application-level connectivity and allows resources to be efficiently deployed and discovered. The grid software platform layer includes grid system software and middleware, such as Globus, OGSA, web services, and other commercial grid software, as well as technologies developed by the Vega team. The application layer includes various application software servers, such as database servers, web servers, and business application servers. The Vega Grid adds two new components at this layer, one at the client side and one at the server side. The Vega Grid “Browser” is different from a traditional web browser, in that it allows users to write to and to operate the grid. The Vega Grid Server (the GSML server) is a portal to the grid, which provides a logically single entry point for users to interact with the grid, and handles processing tasks that are common to all grid services.
Vega Grid Browser
GSML
GSML Server
GSRP
be constructed as a fully distributed resource network via Resource Routers. The Resource Router is a transfer station for resource discovery requests, and it can collect the information of grid resources and provide a path for resource requests from grid client devices to grid resources. Currently, the Resource Router is implemented in software. Work is underway to implement it as a special hardware to execute resource discovery and resource routing tasks. 3.3 The Vega Grid Operating System
At the grid system software layer, we are using popular technologies such as OGSA/Globus and web services as a basis of our Vega Grid Operating System. At the same time, we are developing complementary technologies to suite the needs of the applications and users in the China Grid project. A research direction here is to provide the ability to access resources spanning multiple administration domains. Our approach is to integrate the global user identification into the traditional operating systems on grid nodes, so that grid users can access and integrate grid resources distributed in multiple domains transparently. The following are two examples in this direction. •
Vega Grid File System. File operations in traditional file systems are restricted within a single domain. That is, a user belonging to one administration domain can not access the file resources in another domain directly. To solve this problem, we are developing a Vega Grid File System to support file operations that can span multiple domains. The main idea of Vega GFS is to integrate the global user identification into existing file systems.
•
Vega Grid Process Management. The approach we use in Vega Grid Process Management is similar to Vega GFS. We also integrate the global user identification into the traditional UNIX process structure. By this method, a process will be owned by a global user in a grid but not by a local user belonging to a certain administration domain. A grid user can dynamically create and access processes among multiple domains in wide area.
Other Servers: Database, Web, Applications
Grid Operating System Layer: OGSA, Globus, Web Service, Vega GOS
Grid Hardware Layer: Dawning Grid Enabling Superservers Resource Router, Grid Client
Figure 3. The Vega Grid Three-Layer Architecture The Vega Grid “Browser” and the Vega Grid Server interact through a new protocol, called the Grid Service Request Protocol (GSRP). Another new feature is the Grid Service Markup Language (GSML), which allows users (not necessarily programmers) to specify grid services and user interface in an easy to use fashion. 3.2 The Vega Grid Resource Router
The Grid Resource Router is the backbone of the Vega Grid hardware. It is a bridge linking all grid client devices and grid resources together. All grid client devices and grid resources can dynamically join or leave the grid by connecting or disconnecting to one or more Resource Routers. As more and more Resource Routers are linked together, the grid can expand to a large scale. The grid can
3.4 Vega GSML and Related Tools
The Vega Grid user environment is a set of tools and protocols that enable end users to use grid resources conveniently. We are developing a suite of software, called the GSML suite, to approach this goal. The GSML software suite consists of the following protocols and tools, as illustrated in Figure 4. •
A language called GSML, for users to program a grid and to access a grid.
•
A software grid “browser” (also called GSML browser) at the client side, which renders GSML pages and provides an interface for users to access a grid. This grid browser is different from a Web browser in
that it allows the users not only to read contents from a grid, but also to write to and to operate a grid, by sending (lightweight) service requests to the grid side. •
A software grid server (also known as GSML server) that receives service requests from the client side, processes the requests, and sends results back to the client side.
•
A protocol for the interactions between the GSML server and the GSML browser, called Grid Service Request Protocol (GSRP).
•
A software tool called grid Resource Mapper, which converts native grid resources into virtual resources.
•
A software tool called grid Resource Composer, by which a user can browse/select virtual resources and integrate them into a GSML page.
Resource Mapper
Resource Composer
GSML Page
Grid Resources GSRP
Grid Browser
Grid Community GSML Server
Figure 4. The structure of Vega Grid user environment When designing the Vega Grid user environment, we focus on two important issues: how to empower users with the ability to program the grid and at the same time, how to keep the complexity manageable. The objective is to have an interface that has enough power while still usable, so that an end user with simple training can use it. For the programmability issue, we propose the Grid Service Markup Language (GSML), which is based on XML and includes a set of tags to describe various grid resources such as contents, services, databases, etc. The GSML supplies the programming ability to end users, who can construct a customized resource view easily and quickly. For the usability issue, we adopt a three-point approach. First, we restrict the power of GSML to less than that of finite state machines. Second, we design the GSML suite to be an amplifier, so that a simple user request can be translated into a lot of activities at the grid side. Third, we
map the global grid resources to a user-specific view. The last two points are helped by a new concept called grid community. Following the wisdom in computer architecture and operating systems development, we view the grid resources to form something similar to the address space in a computer system, including physical addresses, virtual addresses and effective addresses. Analogously, we can use a three-level scheme to view grid resources: the first one is the native grid resource level (physical resources), which includes all resources in the grid; the second one is the grid community level (virtual resources), which is a subset of grid resources; the third one is the GSML page level (effective resources), which is a user-specific resource collection. The role of the Resource Mapper is to convert the distributed heterogeneous resource to virtual resources in a “centralized” community. After this three-level conversion, the content of a GSML page can be a small resource collection and user-specific, which enable users to make a customized resource view easily. The grid community concept offers two additional advantages. The first one is that a grid community can serve as a container of common functions of the community of users and resources, such as access control, context, and trust. By just specifying a community name in the head part of a GSML page, users can implicitly take advantage of all common benefits offered by the community, taking it for granted that the GSML software suite will take care of all the ensuing operational details. The second advantage of grid community is to simplify users’ needs of technical skills. We can classify users into three roles: executive, the executive’s secretary, and technical staff. We envision that the technical staff will use the Resource Mapper to convert physical resources into the community level, which will be in such a form that the secretary could understand. The secretary can create a customized grid service in the form of a GSML page, using the GSML Composer tool. The executive can then read, write, and operate the grid with the personalized GSML page through the grid browser. Our eventual goal is that, as the GSML suite matures, any high-school graduate (such as a farmer) will be able to program and to use grid. 4. CONCLUSIONS
Based on our research experiences and user interactions in the past five years, we can offer several concluding remarks: •
Grid technology is not just a high-performance technical computing area any more. It is a paradigm shift affecting all application fields. A case in point is that the China Grid project has attracted research proposals from more than 20 applications fields in China, ranging from pollution control, forestry ecology, national land survey, aero and space industries, city-wide SME supply chain management, media publishing, virtual observatory, to drug
discovery and bioinformatics research. Consequently, the grid should not be just a computational grid, but a service grid enabling resource sharing and collaboration utilizing various computing, storage, communication, data, information, software and transactional resources. •
Since grid is a paradigm shift, it provides a unique opportunity for all computer professionals, especially those in the Asian-Pacific region, to innovate new techniques and to explore new business models. A key theme is how to make existing techniques and business models grid enabling or grid enabled. At ICT, we are investigating technologies such as grid enabling clusters, grid resource routers, grid file systems, and grid client devices.
World Wide Web, in that all peoples are involved to create a beautiful world. ACKNOWLEDGMENTS
We would like to thank Professors Qian Depei, Xie Xiaonghui, Xiao Nong, and Yang Guangwen for useful discussions and comments. We are indebt to the Dawning and the Vega Grid research teams for their contributions. REFERENCES
1. I. Foster, C. Kesselman, “Globus: A Metacomputing Infrastructure Toolkit”, International Journal of Supercomputer Applications, 11(2), 1997, pp. 115-128. 2. I. Foster, C. Kesselman, J. Nick, S. Tuecke, “Grid Services for Distributed Systems Integration”, IEEE Computer, 35 (6), 2002, pp. 37-46.
Many people perceive grid as a new infrastructure. However, we must study new interfaces that enable people to program and use this infrastructure effectively. In this direction, the Vega Grid team at ICT is developing the GSML software suite and the grid community technique.
3. H. Kreger, IBM Software Group, Web Services Conceptual Architecture (WSCA 1.0).
For grid technology to eventually succeed as an industry in the market, there must be one great global grid (or The Grid, as Ian Foster put it). Multiple isolated grids are a contradiction of terms. To achieve this global grid vision, we must have an international cooperative effort in developing an open, standard, and modular grid architecture. The OGSA research and the Global Grid Forum have made long strides in developing grid architecture with open standards. Care must be taken to make the architecture modular as well, so that individual researchers will have the opportunity to implement modules with the most effective techniques, and users will have the freedom of selecting the best combination.
5. D. Normile, “Beijing Genomics Institute: From Standing Start to Sequencing Superpower”, Science, 296(5565), 2002, pp. 36-38.
The last point needs some elaboration. The China Grid project, as well as the Vega Grid team at ICT, consider ourselves a part of the international grid research community, including both academic and industrial researchers. As such, we are actively joining the Global Grid Forum activities and seeking collaboration with researchers from all over the world, to jointly develop the global grid together. Our current international partners include the Globus team, IBM, and Platform. We strongly believe that for grid technology to succeed, it must follow the examples set by its predecessors, the Internet and the
9. Z. Xu, N. Sun, D. Meng, W. Li, “Cluster and Grid Superservers: The Dawning Experience in China”, Proc. of the 3rd IEEE Int’l Conf. on Cluster Computing, 2001, pp. 351-358.
•
•
4. W. Li, Z. Xu, F. Dong, J. Zhang, “Grid Resource Discovery Based on a Routing-Transferring Model”, The 3rd International Workshop on Grid Computing, November, 2002.
6. N.H. Sun, T.Y. Liu, “Grid Enabling Clusters”, Journal of Computer Research and Development, 39 (8), 2002, pp. 917-922. 7. Z. Xu, W. Li, “The Research on Vega Grid Architecture”, Journal of Computer Research and Development, 39 (8), 2002, pp. 923-929. 8. Z. Xu, W. Li, H. Fu, Z. Zeng, “The Vega Grid and Grid-Based Education”, The 1st International Conference on Web-Based Learning, 2002, pp. 228240.
10. J. Yu et al, “A Draft Sequence of the Rice Genome”, Science, 296(5565), 2002, pp. 79-92. 11. H. Zhuge, “A Knowledge Grid Model and Platform for Global Knowledge Sharing”, Expert Systems with Applications, 22(4), 2002, pp. 313-320.
