Workflow Management for High Availability of ...

European Journal of Scientific Research ISSN 1450-216X Vol.81 No.3 (2012), pp.378-385 © EuroJournals Publishing, Inc. 2012 http://www.europeanjournalofscientificresearch.com

Workflow Management for High Availability of Resources in Grid Environment S. Baghavathi Priya Corresponding Author, Research Scholar Jawaharalal Nehru Technological University Hyderabad Andhra Pradesh, Associate Prof./IT. Rajalakshmi Engineering College Chennai, Tamil Nadu. India E-mail: [email protected] T. Ravichandran Principal, Hindustan Institute of Technology Coimbatore, Tamil Nadu, India Abstract The objective of the work is to propose workflow management for high availability of resources in grid computing. Grid is providing infrastructure for grid workflow for managing grid applications. The grid infrastructure focuses on large-scale resource sharing, innovative applications, and high performance orientation. Grid computing enables the sharing, selection and aggregation of distributed heterogeneous resources that are under the control of different grid sites. Workflow management is emerging as one of the most important grid services. Grid workflow systems have been proposed for defining, managing and executing scientific workflows. The grid environment provides a rich foundation for sharing basic resources and capabilities. The availability of workflow resources that are both high quality and cost free is important. There are several significant issues that have been raised for workflow enactment in the context of grid systems. Of course, for those interested in scientific applications, resource availability is always an issue. An efficient algorithm is required for executing complex workflow. In this paper, we propose resource availability are to be utilized successfully for the completion of workflows in grid computing systems. An Availability based Workflow Management (AWM) is proposed to regulate the number of workflows to utilize the available resources in different grid environment. Keywords: Grid computing, Grid workflows, Workflows resources, Availability, Resource allocation.

1. Introduction Grids have emerged as a global cyber infrastructure for the next generation of e-science applications, by integrating large-scale distributed and heterogeneous resources. Grid computing has the potential to provide users on demand access to large amounts of computing power, just as power grids provide users with consistent, pervasive, dependable and transparent access to electricity, irrespective of its source.

Workflow Management for High Availability of Resources in Grid Environment

379

Grid computing is a great advancement in the domain of distributed systems. Grid computing aims to “enable resource sharing and coordinated problem solving in dynamic, multiinstitutional virtual organizations”. A workflow consists of a sequence of concatenated steps. Workflow may be seen as any abstraction of real work. A workflow is a collection of scientific tasks according to their requirements. Workflow management has emerged as one of the most important grid services in past few years. A workflow management system[11] is generally employed to define, manage and execute these workflow applications on grid resources. Resource allocation is a critical and very delicate aspect of the grid-based workflow enactment. Grid resource availability is defined as the probability of available resources to be utilized successfully for completing the workflows in the grid computing system. Because of the heterogeneity of grid computing, an efficient algorithm is required to regulate the number of workflows for utilizing the available resources in different computing nodes. The sharing is not primarily a file exchange but rather a direct access to computing resources, software, storage devices and other resources, with necessary, highly controlled sharing rule which defines clearly and carefully what is shared, who is allowed to share and the conditions under which sharing occurs[15,16]. On the perspective of the computing model, the grid always use a local resource manager to manage the computing resources for a grid site, while the users submit jobs to request some resources for some time. More precisely, the dependability represents a set of attributes namely : availability, reliability, robustness, safety, integrity and maintainability[18]. Availability which is the time proportion a system is in a functioning condition. More precisely, it is the probability that the system is in the correct state a given time. Generally , grid resource management is defined as the process of identifying requirements, matching resources to applications, allocating those resources, and scheduling and monitoring grid resources over time in order to run grid applications as efficiently as possible. As the complexity of grid applications increases, it becomes ever more important to provide a means to manage application composition and the generation of executable application workflows [8]. The rest of the paper is organized as follows; Section 2 illustrates the related work on resource availability and new approach. Section 3 describes proposed architecture for grid workflow system. Section 4 describes availability based workflow management algorithm. The experimental results are discussed in section 5. Section 6 concludes the paper and presents the future work.

2. Related Work and our New Approach We first review related work on workflow management in computational Grid. Then, we introduce efficient algorithm for utilizing available resources in different computing nodes. 2.1. Related Previous Work An online scheduling approach was proposed for multiple mixed-parallel workflows in grid environments [1]. Fairness Dynamic did not work with mixed-parallel workflows composed of dataparallel tasks. Concurrent Transaction logic (CTR) method was presented [2] for grid process modeling for the modeling and controlling execution of grid workflow. A method for efficient scheduling to obtain optimum job throughput in a distributed campus grid environment was presented [3] in which the results were compared with homogeneous compute cluster. Grid based resource management was focused [4]. Some faults may be managed directly by the underlying resource management systems – however, this cannot always be guaranteed. The results of several scenarios that demonstrate the functionality of the Fault Tolerance and Recovery component was presented [5]. Workflow scheduling on the Grid becomes more challenging when multiple scheduling criteria are considered. The proposed taxonomies, [6] identifying the most common use cases and the areas that have not been sufficiently explored yet. A novel Min-Min-Average (MMA) algorithm was proposed for efficiently scheduling transaction-intensive grid workflows involving considerable communication overheads. The MMA algorithm [7] is based on the popular Min-Min algorithm but uses a different strategy for transaction-

380

S. Baghavathi Priya and T. Ravichandran

intensive grid workflows with the capability of adapting to the change of network transmission speed automatically. The main benefits brought up by the system and the difficulties that have been faced by the system developers and the users and managers of the Our Grid Community was discussed [9]. A fuzzy timing technique is applied to address new challenges of workflow management in a crossdomain and highly dynamic grid environment [12]. The taxonomy not only highlights the design and engineering similarities and differences of state-of-the-art in Grid workflow systems, but also identifies the areas that need further research [13]. A multi-tiered workflow management system was described. The system couples a performance modeling tool with local and global scheduling algorithms that aim to meet user-specified deadlines while resolving user and resource conflicts [14]. 2.2. Our New Approach This section presents the Availability based Workflow Management (AWM) approach proposed for management of workflow applications based on availability of resources. The architecture is shown in the fig 1 is used for grid workflow management. The grid environment is composed of 3 tier architecture. In the upper layer of the architecture the grid user submits the job, the performance monitoring and management is done in the middle layer and the lower layer consists of a set of physical resources. The users have complex tasks and want to take advantage of the resource-rich environment provided by the grid to solve their problems subject to a set of constraints such as dead lines, cost, and quality of the solution. A complex task consists of multiple activities. Activities are units of work to be performed by the agents, humans, computers, sensors and other man-made devices involved in the workflow enactment. A process description also called a workflow schema is a structure describing the activities to be executed and the order of their execution. A workflow has three dimensions; 1.The process: the process dimension refers to the creation and eventual modification of the process description. 2. The case: The case dimension refers to a particular instance of the workflow when the attributes required by the process enactment are bound to specific values. 3. the resources : The resource dimension refers to discovery and allocation of resources needed for the enactment of a case. Workflow enactment is the process of carrying out the activities prescribed by the process description for a particular case. In the case of workflow enactment, there are two aspects to this: efficiency and robustness. In terms of efficiency, the critical issue is the ability to quickly bind workflow tasks to the appropriate grid resources. Robustness is another issue. A related issue is the monitoring of the workflow. Another important thing is grid service-workflow management. The main functionalities of grid workflow management include workflow construction, simulation, scheduling, execution, monitoring, conflict solving and so on. However, resource availability, and, most importantly, its impact on the performance of largescale computing environments are analyzed. The evaluation of workflow system performances depends on many factors, amongst which the system’s architecture, the workload, and also the system’s and user’s objectives. For instance, grid users may have as objective to complete the tasks with minimum completion time. Another possibility is to maximize the utilization of the resources. This work is based on grid resource availability. Grid resource management is based on two grid services: information and performance services. Various types of tasks that can be performed within a workflow. Grid workflow can be seen as a collection of tasks that are processed on distributed resources in a well-defined order to accomplish a specific goal.

3. Grid Resource Management 3.1. The Grid Workflow Architecture The Fig.1 shows the architecture and functionalities supported by various units of the grid workflow system. Users interact with workflow modelling tools to generate a workflow specification, which is submitted to a run-time service called grid workflow engine. Major functions provided by the grid

Workflow Management for High Availability of Resources in Grid Environment

381

workflow engine are scheduling, performance and monitor management. Grid workflow engine provides operational functions to support the execution of the workflows based on the process definitions. Performance management evaluates the grid service according to execution condition of the grid service and facilities the dynamic scheduling of workflow engine for optimal performance. Monitoring management collects the execution information of the instances and resources to provide the grid users. The workflow engine can dynamically schedule the workflow using the performance and resource information. Grid environment consists of grid information services, resource services and application services. Figure 1: The Grid Workflow Architecture

Grid Information service : To enable efficient and appropriate uses of the resources from both the systems and applications perspectives, it is important to provide means to keep track of the availability and attributes of millions of resources. In grid, this functionality is provided by grid information service. In an information service computing resources are characterized by sets of attributes, as for example the type of the operating system, network address, CPU speed or storage capacity. Resource service: A fundamental function of resource service is the search for resources with specific combinations of attribute values. Due to a large number of resources, indexing of the attributes becomes necessary. The responsibility of resource service [10] is to select the optimal resources according to the requirements after analyzing the requirements submitted by the user to execute the task. Application service : The responsibility of application service is to collect the requirements of the task groups. The analysis of these requirements will be collected by resource service for searching the required resources.

382

S. Baghavathi Priya and T. Ravichandran

4. Availability Based Workflow Management Algorithm Figugre 2: Task assignment to resource

The Fig 2. shows a task assignment to resource. This section presents the Availability based Workflow Management (AWM)approach proposed in this paper for management of workflow applications based on availability of resources. Fig. 1 shows the structure of AWM.  The Estimated Resource Available Time ERAT(ti, Rj) is defined as the estimated resource available time of task ti.on task group TGk.  The Estimated Data transfer unit Available Time EDAT(ti, Dx) is defined as the estimated data transfer unit available time of task ti.on task group TGk.  The Mean value MERAT of estimated resource available time is defined as the average value of estimated resource available time.  The Mean value MEDAT of estimated data transfer unit available time is defined as the average value of estimated data transfer unit available time.  The Minimum Completion Time MET(ti, Rj ) is defined as minimum successful time of completion of a task ti. on the resource(s) Rj. Estimated resource available time Er(at) = tr(a) – tr(u) (1) Estimated data transfer unit available time Ed(at) = td(a) – td(u) (2) Step

Algorithm : Availability based Workflow Management Input: T : A list of workflow tasks. R : A group of resources. Output: C : A successful completion of tasks.

begin while(T!=Null and R!=Null) do Select ready task ti є T with the highest priority task. Arrange ready tasks to task groups TGk based on required resources. for each task group TGk є TG Choose a set of capable resources for executing the tasks. Assign tasks ti to the resource(s) Rj . T = T – { ti } R = R – { Rj } Add the new tasks to the task groups from the task list. for each task ti in the task group TGk Calculate MERAT and Calculate MEDAT Assign tasks ti to the resource group Rj that gives Minimum Completion Time MCT((ti, Rj ) end while end

Workflow Management for High Availability of Resources in Grid Environment

383

A task may use a variable number of grid resources which may be added or with drawn according to the task requirements and to the current system state. Resource units indicate how much of a resource’s available time, according to the resource calendar, is being used to work on a particular task. Resources are the nodes that are applied to completing a task. Duration = work / No. of resources. Resource availability is usually defined as the expected fraction of time during which a resource or system is functioning acceptably. It is the ratio of up time to the sum of up time plus down time. The down time is the product of the failure intensity and the mean time to repair(MTTR).Availability integrates both reliability and maintainability parameters and depends on the number of failures that occur and on how quickly any faults are rectified. The long-run or steadystate availability is defined as the proportion of the time during which the resource is available for use. It can be expressed as : Availability 

Up - time Up - time  down - time

The denominator is equal to the total time for which the resource is required to function and the up-time is the actual period for which the resource is available for use. The percentage of time the resource is under operation is called the steady-state availability. It characterizes the mean behavior of the resource. The availability function A(t) is defined as the probability that the resource is operating at time t. The steady-state or long-term availability of a single resource is A

μ λμ

5. Results and Discussions Gridsim Toolkit 4.0 which allows modeling and simulation entries in grid system. Figure 3: Minimum Completion Time

Task completion time(sec)

Minimum Completion Time 18 16 14 12 10 8 6 4 2 0

TG1 TG2 TG3 TG4 TG5

R1

R2

R3

R4

R5

Resource Group

In this section, the proposed AWM algorithm is analyzed. The simulation is based on the grid simulation tool kit [17] Gridsim Toolkit 4.0 which allows modeling and simulation entries in grid system. The heterogeneous environment is build by various resource specifications. The resource in the grid environment differs in type of operating system, network address, CPU speed or storage capacity.

384

S. Baghavathi Priya and T. Ravichandran

The main aim for submitting a task is to minimize the completion time. The Fig 3 shows the minimum completion time is calculated for various task groups which are assigned to different resource groups. Figure 4: Estimated Resource Available Time

Task Completion Time(sec)

Estimated Resource Available Time 18 16 14 12 10 8 6 4 2 0

Min-Min MMA AWM

R1

R2

R3

R4

R5

Resource Group

In fig 4, the estimated resource available time for different algorithm is compared, in all the cases the resource available time for the AWM algorithm is maximum and its compared with Min-Min algorithm and Min-Min Average(MMA) algorithm.

6. Conclusions and Future Work In this paper, we have introduced a new algorithm called the AWM (Availability based Workflow Management)algorithm for regulating the number of workflows to utilize the available resources in different grid environment This algorithm is based on the available resources to be utilized successfully for completing the workflows in the grid computing system. There are two major contributions in this paper: one is the minimum completion time is calculated for various task groups which are assigned to different resource groups and the other is the comparison of resource available time for different algorithms. The simulation has demonstrated that our AWM algorithms has maximum resource available time among all resources and schedule the next task to that resource which gives minimum completion time. In the future, we plan to realize this algorithm in some real-world e-business applications.

References [1] [2] [3] [4]

Chih-Chiang Hsu, Kuo-Chan Huang, Feng-Jian Wang “Online scheduling of workflow applications in grid environments”, International Journal of Furure Generation of Computer systems, Vol. 27, 2011, 860-870. Zhilin Feng, Yanming Ye, Xiaoming Liu, “A Formal Modeling Method for Grid Workflow Based on Concurrent Transaction Logic”, in Journal of Convergence Information Technology, Volume 6, Number 2. February 2011. Srirangam V Addepallil, Per Andersen and George L Barnes, ”Efficient Resource Matching in Heterogeneous Grid Using Resource Vector”, in International Journal of Computer Science and Information Technology, Volume 2, Number 3, June 2010. Marco Lackovic, Domenico Talia, Rafael Tolosana-Calasanz, Jos´eA.Ba˜nares, Omer F. Rana, “A Taxonomy For the Analysis of ScientificWorkflow Faults”, in 2010 13th IEEE International Conference on Computational Science and Engineering.

Workflow Management for High Availability of Resources in Grid Environment [5] [6] [7] [8] [9] [10] [11]

[12] [13] [14] [15] [16] [17] [18]

385

Elvin Sindrilaru, Alexandru Costan, Valentin Cristea, ”Fault Tolerance and Recovery in Grid Workflow Management Systems “, in 2010 International Conference on Complex, Intelligent and Software Intensive Systems. Marek Wieczorek, Andreas Hoheisel, Radu Prodan, ” Towards a general model of the multicriteria workflow scheduling on the grid”, in FutureGenerationComputerSystems25(2009)237256. Ke Liu, Jinjun Chen, Hai Jin and Yun Yang, “A Min-Min Average Algorithm for Scheduling Transaction-Intensive Grid Workflows”, in 7th Australasian Symposium on Grid Computing and e-Research (AusGrid 2009), Wellington, New Zealand. Miroslav Ruda, ” Grid Simulator with Production Scheduling Algorithms”, in pisa, 2008. Francisco Brasileiro, Eliane Ara´ujo, William Voorsluys, Milena Oliveira, Fl´avio Figueiredo, “Bridging the High Performance Computing Gap: the Our Grid Experience”, in Seventh IEEE International Symposium on Cluster Computing and the Grid (CCGrid'07) 0-7695-2833-3/07. Rajiv Ranjan, Aaron Harwood and Rajkumar Buyya, ” A Study on Peer-to-Peer Based Discovery of Grid Resource Information” in December 1, 2006. Junwei Cao, Stephen A. Jarvis, Subhash Saini and Graham R. Nudd, “GridFlow: Workflow Management for Grid Computing”, Proceedings of the 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRIDí03) 0--7695-1919-9/03 © 2003 IEEE. Junwei Cao, Stephen A. Jarvis, Subhash Saini and Graham R. Nudd, ” Grid Flow: Workflow Management for Grid Computing”, in Proceedings of the 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRIDí03) 0--7695-1919-9/03. Jia Yu and Rajkumar Buyya, “ A Taxonomy of Workflow Management Systems for Grid Computing“. D. P. Spooner, J. Cao, S. R. Jarvis, L. He and G. R. Nudd, “Performance-aware Workflow Management for Grid Computing”, in The Computer Journal. Junwei Cao, Stephen A. Jarvis, Subhash Saini and Graham R. Nudd, “Grid Flow: Workflow Management for Grid Computing “. Dan C. Marinescu, “A Grid Workflow Management Architecture”. Rajkumar Buyya and Manzur Murshed, "GridSim: a toolkit for the modeling and simulation of distributed resource management and scheduling for Grid computing", Concurrency and Computation: Practice and Experience, Vol. 14, pp. 1175-1220. Geoffrey Fox, Dennis Gannon, “Workflow in Grid Systems”.