Reconfiguration of Information Management Framework based on ...

Reconfiguration of Information Management Framework based on Adaptive Grid Computing Eun-Ha Song1, Sung-Kook Han1, Laurence T. Yang2, Young-Sik Jeong11 1

Department of Computer Engineering, Wonkwang University 344-2 Shinyong-Dong, Iksan, 570-749, Korea {ehsong, skhan, ysjeong}@wku.ac.kr 2 Department of Computer Science, St. Francis Xavier University Antigonish, NS, B2G 2W5, Canada [email protected]

Abstract. In this paper, GridIMF provides the users with consistency while adapting to variability grid information. GridIMF designs hierarchical 3-tier information management model in accordance with participating intention, roles and operating strategies of grid information. In order to manage grid information in effective ways, GridIMF provides grid service while dividing to GVMS and GRMS. GVMS suggests optimal virtual organization selection mechanism for improving performance of specific applications and LRM auto-recovery strategy that treat faults of virtual organization. GRMS supports adaptive performance-based task allocation method for load balancing and fault tolerance. State monitoring and visualization view provides adaptability of managing grid information. Application proxy removes inter-dependency between service objects of GridIMF and application objects. For analyzing GridIMF executability, it adapts two fractal image processing those characteristics are different.

1 Introduction Grid resources are the key elements to support performance of grid application. If computers are the grid resources, it can be specifications of hardware and software in a limited definition or it can be a certain organization for specialized goals in a broad definition. Grid resources potentially have policies about another fields, and they are controlled by another institutions. Therefore, grid computing environment requires reconfiguration in order to support utilizing diverse resources all over the world, and achieve the objects of computational grid. In grid computing environment, it is needed to design a framework that can control upper requirements including representation of grid resources and control of grid operation [1][2][3][4]. In this paper, GridIMF(Grid Information Management Framework) is established with accepting characteristics of grid resources and supporting grid services at the same time. In this point, grid resources mean a tool to perform an operation. Meanwhile, metadata include computing information (CPU, memory and CPU-Load) and ∗

Corresponding Author : Young-Sik Jeong

networking information (bandwidth, wait time). This paper defines virtual organization, resources, and metadata as grid resources. That is, grid resources have integral meaning regardless of scale of information or its characteristics [5][6]. According to the role of grid information, GridIMF designs hierarchical 3-tier information management model and it designs 3-layer framework model in accordance with managing of grid information. A framework model provides grid services with dividing GVMS(Grid Virtual Organization Management System) for managing virtual organization and GRMS(Grid Resource Management System) for managing metadata. This framework shows adaptability and validity of supported service with two differential fractal image processing.

2 Related Works As the grid logically couples multiple resources owned by different individuals or organizations, the choice of the right model for resource management architecture plays a major role in its eventual success. There are models of grid resource management such as Hierarchical, Abstract Owner, and Economy Model, and Table 1 shows a representative system that follows aforementioned models [7]. Table 1. Three Models for a Grid Resource Management Architecture. Model Hierarchical Model Abstract Owner Model Economy/Market Model

System Globus [8], Legion, Ninf, NetSolve, GridIMF Nimrod/G, JaWS, Myriposa, JavaMarket

GridIMF designed by this paper applies Hierarchical Model that a lot of current grid systems follow. GridIMF includes passive elements of Hierarchical Model such as resources, tasks, jobs, schedules, and GRMS suggests active elements of Hierarchical Model such as scheduler, information service, and domain control agent. GridIMF admits managing functions about information service and resource management that the representative model for Hierarchical Model, Globus toolkit, includes. But it is inefficient in the matter of management for metadata as it has the uniformed change and delivery cycle which ignore the characteristic of data. Globus Toolkit can cause the degradation of grid performance. GridIMF provides GVMS and GRMS supporting Globus toolkit’s MDS and GRAM with considering management of metadata. GVMS applies the optimum virtual organization selective mechanism which maximizes the use of resource in the user point of view and optimizes it in the application point of view; and suggests auto-recovery strategy which confronts dynamically to the VO fault. GRMS receives the metadata information within local resource periodically through virtual observer and confronts the load balancing and local resource fault which was resulted in reducing total amount time of computing with adaptive performance-based task allocation which runs scheduler. GridIMF understands the Grid information in real-time and adds the monitoring technique to enable the effective sharing and putting practice. Monitoring technique can be adapted to the variableness of Grid information and the various forms of virtualiza-

tion view have easy way of management [9]. Table 2 shows the comparison of Globus toolkit and supportive system which was presented in this system. Table 2. Globus vs. GridIMF System Globus Activity Functions Scalability of grid ● Dynamic VO configuration ▲ Availability for service ● Deletion of replicate operation at ✕ fault Information state monitoring

●

Information state visualization Load balancing strategy Fault tolerance strategy Independency among application User access interface

✕ ▲ ▲ ● ●

GridIMF

Remark

Hierarchical 3-tier information management model Optimum virtual organizations selection mechanism and auto● recovery strategy PDH Library, ● (LRM/RP)Virtual observer ● Variable visualization view ● Performance-based Task Allocation Method ● ● Application Proxy ▲ Java Applet (●: support, ▲: a partial support, ✕: nonsupport) ● ● ●

3 Grid Information Management Framework 3. 1 GridIMF Architecture GridIMF aims the high performance and the distributed computing. Basically in the infrastructure, the dynamic and effective maintenance, management, and usage of metadata which is the detailed element of grid information should be available. In this paper, as shown in figure 1, 3-layer framework model is constructed and the existing computing resources which are preserved in one site can be formed into the GridIMF formation.

Fig. 1. 3-layer framework model

Infra layer includes the base task to connect the different characteristic constituent into the integrated resource. Middleware layer has a role to show the available information to the user into one system and adds the function which was suggested in this

paper to the general middleware function. The management of metadata which was physically divided individually is divided into GRMS and the management of virtual organization is divided into GVMS. In application layer, the grid applications which need much calculation can approach to the grid information which is connected through infra layer and application and dependent middleware layer service are provided through application proxy. 3.2 Function Model of the GridIMF Components GridIMF divides the components of grid into RR(Resource Requester), RP(Resource Provider), LRM(Local Resource Manager) and GIM(Global Information Manager) depending on the intention and purpose of participation. Fig. 2 shows the structure of architecture layer and control stream of suggested GridIMF components.

Fig. 2. Structure of layer and control stream of GridIMF components

RR is the grid user and requests job submission to GIM. RR can request several jobs at the same time, only receives the corresponding result of requested job and does not participate in computation. RR plays a role of interface delivering information of independent application. GIM is the top level manager and is a kind of resource brokering proxy connecting between resource and user who wants to use grid in remote. GIM manages the connection by providing common interface of components and executes job submission and control management as proxy. GIM is a super scheduler which brings the list of LRM satisfying based on job specification of specific application and permits the connection. LRM gets the delegation collecting resource and allocating task for requested job. LRM selects RP through resource discovery, intermediate the connection between RP and RR and makes core scheduling between remote resources. RP is a group which permits the execution for a part of task instead of huge scale of job and collects the information of metadata. RP delivers the executable resource, performs the operation and transmits the result.

4 Grid Virtual Organization Management System 4.1 Task Brokering Rule Virtual organization has different policy, purpose and size. Grid application is finished its task by virtual organization with the scheduling. Therefore, the performance of application is the selection of virtual organization appropriate to the task. GridIMF defines 4 kinds of factors for selecting optimum virtual organization: ▪ Possibility of application execution: One application is assumed to be executed by one virtual organization. One LRM is possible to execute applications independently. GIM provides a list of application which is already executed and should be executed. LRM selects application which can be executed. The first stage selection depends on LRM which knows the capability and size of its virtual organization. ▪ Idle State: GIM senses the wait state by LRM virtual observer that is going to execute the requested task and then decides the possibility of execution. ▪ Application Power: The operation result of grid application is recorded as log file. The attribute of log file is the application name, execution time, LRM information, number of LRM and RP, total amount of static CPU speed, etc. ▪ LRM Performance Index: Performance index can be obtained by the analysis of application log file. It selects a value close to the expected value in proportion to the average speed of CPU and number of nodes for executing requested application. 4.2 LRM State Control and Auto-Recovery Strategy GIM is the manager of dynamic LRM so that transmission of control message is frequent related to LRM. GridIMF makes LRM virtual observer that gathers information of LRM and plays communication broker, and separates operation and control. LRM virtual observer is the monitor which makes scheduling the task and manages RPs inside virtual organization. LRM auto-recovery strategy senses the LRM fault and secures the availability of service by obtaining the corresponding resource from another LRM. RR can be possible to use the grid information and to change the specification of executing application until the connection of GridIMF is disconnected. The fault information of LRM is accomplished by LRM virtual observer. When LRM fault is found, it looks for a new LRM in idle state. RR sends address, remote object reference and task table for the reference and operation is automatically executed by a new LRM. When the search for LRM is failed, RR is in wait state and added into the waiting list. The management of RR task table and remote object reference do not have replicated operation of LRM which generated fault. 4.3 GVMS Monitoring Information Visualization GVMS provides the virtualization view of various forms to manager and system planner through real-time monitor. Monitoring and virtualization data is limited to LRM and RP in the GIM side of hierarchical information management model and

divided into Virtual Organization Monitor. It does not include metadata information of RP. The target of monitoring is RR, LRM and RP which are performing and holding calculation. Visualization factor recorded the relative group which was traced by virtual observer in LRM Entry and RR Entry and transferred the revised information to the view. GVMS monitoring information visualization is provided with button from GIM main frame with one frame and two views as shown in Fig. 3.

Fig. 3. Information Visualization View of GVMS

GIM state transition frame is a demon frame which runs during main frame loading. LRM/RR Join View provides the relationship between RR and LRM with table component, and the detailed information of selected line is output into text component. LRM/RP Hierarchical View provides the node structure of GridIMF with hierarchically. Fig. 3 is composed with two virtual organizations and seven RPs, but the dotted area performs RP as well as two applications.

5. Grid Resource Management System 5.1 Task Allocation Algorithms GRMS makes the counter measure and suggests DPTA(Dynamic Performance-based Task Allocation) and APTA(Adaptive Performance-based Task Allocation) for minimizing the cost of resource. Table 3 is the suggested task allocation method. Table 3. Task Allocation Method of GridIMF Task Allocation Method

Performance Evaluation & Reallocation Factor

Dynamic Performancebased Task Allocation

▪ CPU Speed ▪ Job History

Adaptive Performancebased Task Allocation

▪ CPU Speed ▪ CPU Usage ▪ Proportional Factor

Remark ▪ Fault Detection ▪ Append Processor ▪ State Monitoring ▪ Fault Detection ▪ Append Processor ▪ Real-time State Monitoring

DPTA is a method allocating and reallocating task according to the ratio of performance by considering RP performance. Performance index use CPU speed which are the static resource factor of RP and job history. Assuming total job amount for the application is TotJobSize and CPU speed value of RPs is {sRP0 , sRP1 ,L, sRPNumRP−1} , the job size of random sRPi is allocated as follows. JobRPi =

sRPi NumRP −1

∑ sRPk

× TotJobSize,

i = 0 ~ NumRP − 1

k =0

The factor of job history is the factor measuring expectation of each RP performance and dynamically expects the job size per performance. The size of reallocated job for RP is as follows. Re AllocJobRP i = pRateIncom RPi × ( JobIncomRP j − ComJobInco mRP j )

sComRP i

pRateIncom RP j =

sComRP i + sIncomRP j ×

ComJobRP j sComRP j

where, sComRP i : Finished RP performance for the allocated job

sIncomRP j : Unfinished RP performance for the allocated job JobsIncomR Pj : Unfinished size of initially allocated RP job for allocated job

ComJobsInc omRP j : Size of job performed by RP which did not finish the allocated job pRateIncom RP j : RP performance index occurring performance change

APTA is a method which monitors the static and dynamic performance of RP and allocates the job adaptively. This establishes a standard based on the performance executing real allocated task not a physical performance for the RP performance. RP performance value applies proportional factor and CPU speed. Proportional Factor(PF) is a standard value as CPU usage rate used by RP user or internal system differs in the degree of total execution time depending on the range. Table 4 is the proportional factor in different section. Table 4. Proportional Factor in different section Section A C E

User CPU Usage 0~10% 20~30% 40~60%

PF 2.10 1.67 1.09

Section B D F

User CPU Usage 10~20% 30~40% 60% over

PF 1.91 1.40 1.00

Assuming CPU speed value of RPs is {sRP0 , sRP1 ,L, sRPNumRP −1} , the expectation of total CPU usage is CPUUsageAv g at the time of reallocation and expectation of job processor usage rate is JobUsageAvg , the performance value of random RPi is as follows. RPi Performance = (100 − (CPUUsageAvg − JobUsageAvg )) × PF × sRPi × 0.01

APTA uses RPPerformance value not the job history information and its reallocation is similar to DPTA. Reallocated job size of RP is as follows. Re AllocJobRPi = pRateIncomRPi × ( JobIncomRPj − ComJobIncomRPj )

pRateIncomRPi =

pComRPi ComJobIncomRPj pComRPi + pIncomRPj × JobIncomRPj

where, pComRPi : value of RP which finished the allocated job pIncomRP j : value of RP which did not finish the allocated job

Fig. 4 is the comparison of task allocation method that user gives random CPU usage rate to 4 RPs with different specification. For the case if user CPU usage rate is below 10%, there is no difference in task allocation methods. This implies that performance changes almost did not in RPs and it did not influence on the task performance rate. On the contrary, the case of user CPU usage rate increasing to 20~30% and 30~40%, the total job performance time of APTA considering total CUP usage rate is low. That is, user CPU usage rate of RP influenced on job performance rate of total CPU usage rate.

Fig. 4. Comparison of task allocation method according to the change of user CPU usage rate

5.2 GRMS Monitoring Information Visualization The monitoring of RPs has effect on LRM scheduling and task allocation method which performs real operation. The state value of RP is measured and improved the operation usage and controlled the defect. After revising the metadata information, the state information is received through RP virtual observer and transferred to RP Entry and provided the user selection event with a view. Fig. 5 provides the state information and analysis result of RP with one frame and five views. LRM provides task progress which is a performing task with progress bar component, and provides RP list which is connected to LRM with combo box. RP information view represents the selected metadata information of RP with progress bar and text component and applies to performance test of RP with analyzing sampled hardware information. RP Real-time Table View revised the RP information in a certain period and is represented in Table. RP state graph View represents the RP CPU usage and user CPU usage in the real-time graph form. Connection Graph View observes task control situation of RP which was connected to LRM. RP Network Information View represents the ratio between whole period of task control and network transmitting period of RP in the form of bar graph.

Fig. 5. Information Visualization View of GRMS

6 GridIMF implementation and performance result 6.1 Application Architecture Application is implemented with the verification of strategy suggested in GridIMF. Grid user has different request depending on application so that the application architecture is structurally divided only with minimum correlation. For the mechanism minimizing interdependence between different application and GridIMF, Application Proxy is designed as shown in Fig. 6. Application Proxy is a standardized common API and includes core function for interacting between application and GridIMF. Application proxy is an Application Name of plug-in for accessing specific application and the object is generated dynamically by Application Templet.

Fig. 6. Application Architecture of GridIMF

6.2 Implementation of Application and Result of Performance The available application to GridIMF is independent to each task and it is supposed to require a lot of calculation compared to small amount of data. In this paper, the fractal image processing is shown as Fig 7. Application performs image generation by complicated calculation and performance process is certified visually. Fig. 3 (a) is the Mandelbrot fractal image processing of LSM coloring technique. Scheduling is APTA and two new RPs are joined in the calculation orderly in the middle of task performance by two RPs. Fig. 3 (c) is the result of the first calculation and there is a difference in coloring technique for re-assigning occurrence according to RP add and performance change. Resource usage and requirement change should be available before the end of GridIMF by grid user. So, Fig. 3 (c) can perform the re-requested job with changing of application list when the specific domain is established by drag. Fig. 3 (e) is Mandelbrot Fractal Image Processing which performs in dividing task according to the number of divergence. 300 times of divergency is supposed and the number of divergency is defined by task and it is distributed and performed to RP as performance ratio. Panel is divided into canvas which visualizes task management process and task result.

Fig. 7. Grid application adaptation of two fractal image processing with performance analyzing of GridIMF

7 Conclusions In this paper, grid application was applied with constructing GridIMF to adapt grid Information and to provide the consistency to user. Grid information is a generic name of virtual organization, resource, and metadata. GridIMF is composed of 3layer framework model such as infra layer for grid information, middleware layer for management, and application layer for application. Especially, middleware layer was divided into hierarchical 3-tier information management model according to participation intention, role, and management policy of grid information; and management domain was divided into GVMS and GRMS. GVMS gives authority to use remote resource through resource connection rule and task brokering with the optimal virtual

organization selection mechanism by performance ratio. Even when the virtual organization defect is occurred, LRM auto-recovery strategy is suggested to change the user requirement and to get rid of operation replication. GRMS supported the load balancing and fault tolerance with the adaptive performance-based task allocation which can control the grid information state change, participation, session, and defect. The information state monitoring and visualization of GridIMF supports the reliability of grid information collection. Application proxy was established for providing service which agreed to special grid application with the minimal code modification and the minimal relation with grid infrastructure. The service validity which was suggested in this paper was verified through grid application adaptation of two fractal image processing with performance analyzing of GridIMF and grid application. The main contributions of this paper are as followed; scalability (hierarchical 3tier information management structure), adaptive (dynamic virtual organization, task allocation), availability of service, deletion of replication operation, information state monitoring/ visualization and independent of applications. For further study, in the view of GridIMF approach which was suggested in this paper, skill add of user approach interface following detailed task process for usercentered task performance and dependability of detailed task and adaptation of specific application are needed. Acknowledgement. This paper was supported by Wonkwang University in 2006.

References 1. I. Foster, C. Kesselman, "The Grid: Blueprint for a New Computing Infrastructure", Morgan Kaufmannn, 1999. 2. I. Foster, C. Kesselman, S. Tueche, "The Anatomy of the Grid: Enabling Scalable Virtual Organizations", International Journal Supercomputer Applications, Vol. 15, No. 3, 2001. 3. I. Foster, "The Grid: A New Infrastructure for 21st Century Science," Physics Today.org, 2002. 4. A. Takefusa, H. Casanova, and S. Matsuoka, F. Berman, "A Study of Deadline Scheduling for Client-Server Systems on the Computational Grid,” High Performance Distributed Computing, 2001. Proceedings 10th IEEE International Symposium, Aug. 2001 5. Dr. Bernhard R. Katzy, "Design and Implementation of Virtual Organizations," University St. Gallen and Erasmus University Rotterdam. 6. Rashmi Bajaj and Dharma P. Agrawal, Fellow, "Improving Scheduling of Tasks in a Heterogeneous Environment," IEEE Trans. Parallel and Distributed Systems, Vol. 15, No. 2, pp. 107-118, 2004. 7. Rajkumar Buyya, Steve Chapin, and David DiNucci, "Architectural Models for Resource Management in the Grid," Grid Computing GIRD 2000. First IEEE/ACM International Workshop Bangalore, India, pp. 20-33, 2000. 8. I. Foster, C. Kesselman, "Globus: A Metacomputing Infrastructure Toolkit, " International Journal supercomputer Applications, Vol. 11, No. 2, pp. 115-128, 1997. 9. D. Angulo, I. Foster, C. Liu, and L. Yang., "Design and Evaluation of a Resource Selection Framework for Grid Applications," Proceedings of IEEE International Symposium on High Performance Distributed Computing (HPDC-11), Edinburgh, Scotland, July 2002.