New Algorithm to Resource Selection in ...

International Journal of Intelligent Information Technology Application, 2009, 2(5):204-208

New Algorithm to Resource Selection in Computational Grid Using Learning Automata Ali Sarhadi Islamic Azad University – malayer branch malayer, Iran Sarhadi_ali@ iau-malayer.ac.ir Mohammad Reza Meybodi Amirkabir University of technology Computer Engineering Department, Tehran, Iran [email protected]

Abstract— one of the main challenges in Grid computing is efficient selection of resources (CPU-hours, network bandwidth, etc) for jobs in resource discovery phase. Due to the lack of centralized control and the dynamic/stochastic nature of resource availability, any successful selection mechanism should be highly distributed and robust to the changes in the Grid environment. Moreover, it is desirable to have a selection mechanism that does not rely on the availability of coherent global information. In this paper we study a minimalist decentralized algorithm for resource selection in a simplified Grid-like environment that meets the above requirements. We consider a system consisting of large number of heterogeneous learning automata connected to tasks that select best resources for their computational needs. There is no communication between the learning automata: the only information that learning automata’s received is the (expected) completion time of a job it submitted to a particular resource and which serves as a reinforcement signal for the learning automata. The results of our experiments suggest that reinforcement learning can be used to improve the quality of resource selection in large scale heterogeneous system. Index Terms—resource selection, learning reinforcement learning, grid computing

automata,

I. INTRODUCTION Computational Grid is a new paradigm in distributed computing which aims to realize a large-scale high performance computing environment over geographically distributed resources. Computational Grid enables the sharing, selection, and aggregation of highly heterogeneous resources for solving large scale problems in science, engineering and commerce. Numerous efforts have been exerted focusing on various aspects of grid computing including resource specifications, information services, allocation, and security issues. A crucial issue to meeting the

computational requirements on the grid is the resource discovery. However, the discovery and configuration of suitable resources for applications in heterogeneous environment remain challenging problems. Like others [1-5], we postulate the existence of a Resource Selector Service (RSS) responsible for selecting Grid resources appropriate for a particular problem run based on that run’s characteristics; organizing those resources into a virtual machine with an appropriate topology; and potentially also assisting with the mapping of the application workload to virtual machine resources. These three steps- selection, configuration, and mapping- can be interrelated, as it is only after a mapping has been determined that the selector can determine whether one selection is better than another. Many projects have addressed the resource selection problem. Systems such as NQE [6], PBS [7], LSF [8], ISOFT [9], and Load Leveler [10] process user-submitted jobs by finding resources that have been identified either explicitly through a job control language or implicitly by submitting the job to a particular queue that is associated with a set of resources. This manually configured queue hinders the dynamic resource discovery. Globus [11] and Legion [12], on the other hand, present resource management architectures that support resource discovery, dynamical resource status monitor, resource allocation, and job control. These architectures make it easy to create a highlevel scheduler. Legion also provides a simple, generic default scheduler. But Dail et al. [13] show that this default scheduler can easily be outperformed by a scheduler with special knowledge of the application. Resources on the grid are typically shared and undedicated so that the contention made by various applications. Although there has been considerable attention given to the resource selection problem in the Grid, very few researchers have addressed the problem from the perspective

1999-2459 /09/$25.00 ©2009 Engineering Technology Press

204

of learning and adaptation. Meanwhile, the learning automata systems (LAS) and distributed AI communities have shown that groups of learning automata can successfully solve different load balancing and resource selection problems [14,15]. The goal of this paper is to apply learning automata techniques to the problem of resource selection in the Grid. The LA approach is well suited for describing the Grid, because the distributed, autonomous nature of learning (Grid users and resources) reflects the federated nature of the Grid. Learning nature allows the learning automata systems adapt to changes, such as the changing resource capacities, resource failure.

highest probability of being rewarded) through repeated interaction on the system. If the learning algorithm is chosen properly, then the iterative process of interacting on the environment can be made to result in selection of the optimal action. Figure 1 illustrates how a stochastic automaton works in feedback connection with a random environment. Learning Automata can be classified into two main families: fixed structure learning automata and variable structure learning automata (VSLA) [16]. In the following, the variable structure learning automata which will be used in this paper is described.

II. GRID RESOURCE DISCOVERY ISSUES A core Grid functionality that could be effectively redesigned is resource discovery. Resource discovery is a key issue in Grid environments, since applications are usually constructed by composing hardware and software resources that need to be discovered and selected. In the OGSA framework each resource is represented as a Grid service, therefore Resource discovery mainly deals with the problem of locating and querying information about useful Grid services. In Globus Toolkit the current implementation of the OGSA - information about resources is provided by Index Services. An Index Service is a Grid service that holds information about a set of Grid services registered to it. A primary function of the Index Service is to provide an interface for querying aggregate views of service data collected from registered services. There is typically one Index Service per virtual organization (VO). Index Services can be organized in hierarchical tree structures, in which higher-level Index Services hold information about all the underlying resources. However, for scalability reasons, a multi-level hierarchy of Index Services is not appropriate as a general infrastructure for resource discovery in large scale Grids.

Figure 2. the interaction between learning automata and environment

A VSLA is a quintuple < α, β, p, T(α,β,p) >, where α, β, p are an action set with s actions, an environment response set and the probability set p containing s probabilities, each being the probability of performing every action in the current internal automaton state, respectively. If the response of the environment takes binary values learning automata model is P-model and if it takes finite output set with more than two elements that take values in the interval [0,1], such a model is referred to as Q-model, and when the output of the environment is a continuous variable in the interval [0,1], it is refer to as S-model. The function of T is the reinforcement algorithm, which modifies the action probability vector p with respect to the performed action and received response. Assume β ∈ [0,1] . A general linear schema for updating action probabilities can be represented as follows. Let action i be performed then: A) Desire response pi ( n + 1) = pi ( n) + a[1 − pi ( n)] p j ( n + 1) = (1 − a ) p j ( n) ∀j j ≠ i B) Undesired response pi ( n + 1) = (1 − b ) pi ( n )

Figure 1. Grid information service for resource discovery

III. LEARNING AUTOMATA Learning Automata are adaptive decision-making devices operating on unknown random environments. A Learning Automaton has a finite set of actions and each action has a certain probability (unknown to the automaton) of getting rewarded by the environment of the automaton. The aim is to learn to choose the optimal action (i.e. the action with the

p j ( n + 1) =

b + (1 − b ) p j ( n ) ∀ j j ≠ i r −1

Where a and b are reward and penalty parameters. When a=b, the automaton is called LRP. If b=0 the automaton is called LRI and if 0