DYNAMIC LOAD BALANCING BY MIXING ...

Journal of Advance Research in Business, Management and Accounting

DYNAMIC LOAD BALANCING BY MIXING CENTRALIZE AND DECENTRALIZED APPROACH IN CLUSTER OF WIRELESS SENSOR NETWORK Dr. Sharada Patil Associate Professor Sinhagad Institute of Business Administration And Research, Savitribai Phule Pune University [email protected] Abstract—Now a day’s wireless sensor networks have becomes very useful to monitor environments in the field of military applications as well as civil applications. Load balancing can be used to extend the lifetime of a sensor network by reducing energy consumption. Load balancing using clustering can also increase network scalability. Wireless sensor network with the nodes with different energy levels can prolong the network lifetime of the network and also its reliability. This paper also discusses the improvement to be made for future proposed load balancing schemes. In this paper also proposes dynamic load balancing algorithm which can be follows centralized and decentralized approach for cluster of wireless sensor network. Index Terms— Cluster, Wireless sensor network, Dynamic load balancing Algorithm. I. INTRODUCTION Parallelism has been in use for many years, mainly in high-performance computing. Interest in parallelism has grown gradually due to the physical constraints preventing frequency scaling. As power consumption (and consequent heat generation) by computers has become a concern in recent years, parallel computing has become the dominant paradigm in computer architecture, mainly in the form of multi core processors. The task level parallelism or instruction level parallelism is called as parallel processing. ([12][2] Parallel processing is mainly divided into two types viz. Homogeneous parallel processing or heterogeneous parallel processing. Homogeneous parallel computer means collection of identical processors connected with each other, where as

Vol. 2 Issue 4 April 2016 Paper 8

heterogeneous parallel processing means a collection of dissimilar processors (whose machine language is different from each other) connected with each other. There are many forms of parallel computers. [12][2] Parallel computers can be roughly classified according to the level at which the hardware supports parallelism. This classification is broadly analogous to the distance between basic computing nodes. These are not mutually exclusive: for example, clusters of symmetric multiprocessors are relatively common. [8][4] 1.1

COMPUTER CLUSTERS

A computer cluster is a collection of loosely coupled computers, which work together such that outer world feels that it's a single computer. These loosely coupled computers are called as components of the cluster. These components of a cluster are commonly, but not always, connected to each other through fast local area networks. Generally clusters are deployed to improve performance as compared to a single computer. It is more cost-effective than single computer of comparable speed as well as availability. The individual computers can be PCs or workstations. Ideally, a cluster works as an integrated computing resource and has a single system which distributes task amongst all its nodes. Hence, the users can see only a single system. Some of these influencing factors are the parallel workload, presence of any sequential and/or interactive jobs, native operating system, node hardware, network interface, network, and communication software. (Paul Werstein, Hailing Situ, and Zhiyi Huang, - 06) [8] There exists a popular software which evenly distributes workload amongst the clusters called as

72


load balancing algorithm. The main objective of the load balancing algorithm is to speed up the system and enhance super computing power within the clustered system. There are two significant ways of performing load balancing viz. Static policy and dynamic policy. ( Janhavi B,Sunil Surve ,Sapna Prabhu-2010 ) (64) A Beowulf cluster is a computer cluster in which normally identical, commodity-grade computers are networked into a small local area network (LAN) with libraries and programs installed, which allows the processor to be shared amongst them. As a result can be designed a highperformance parallel computing cluster from inexpensive personal computer hardware. [15] There is no software that defines a cluster as a Beowulf. Beowulf clusters normally run on a Unix-like operating system, for example BSD, Linux, or Solaris, normally built from free and open source software. Commonly used parallel processing libraries include Message Passing Interface (MPI) and Parallel Virtual Machine (PVM). Both of these permit the programmer to divide a task among a group of networked computers and collect the results of processing. Examples of MPI software include OpenMPI or MPICH. Few additional MPI implementations are also available. 1 [15] 1.2 LOAD BALANCING ALGORITHMS AND ITS

1.3 DYNAMIC LOAD BALANCING ALGORITHM Dynamic load balancing policies depend on the current status of the system. In dynamic policies, the workload is distributed among the processors at run time. New processes are assigned to the processors based on the runtime information that is collected from each node. If each node in the system becomes overloaded, the task that causes this overloading should be transferred to an under loaded node and run there. Although the dynamic policies have many benefits and can adopt with the current state of the system, sometimes they will incur extra overhead on the system because of the process migration and reservation resources of the system for collecting the information about the current status of the system. After collecting, this information should be available to other nodes to make an optimal decision. If the load balancing scheme is a centralized scheme, all nodes transfer their information to a cluster head or a decision making center that can be an agent, whereas in distributed schemes, this information should be available to all nodes. (Parimah Mohammadpour, Mohsen Sharifi, Ali Paikan-2008 )[8]. The information exchange, policy can obey one of the following policies: A. Periodic policy

TYPES

The load balancing algorithm is the parallel program, which distributes even workload among the compute nodes of the clusters to provide very good throughput, which increase the speed of cluster system using message passing interface. Definition: The presence of a number of processors in multiprocessor systems shows the necessity of the uniform distribution of workload among these processors, since, studies have shown that in these systems the probability of a processor being idle in the system and other processors having a queue of tasks at hand is very high. The workload of the system is distributed appropriately among the processors so that it can increase the overall speed of the system. This is called as “Load balancing algorithm”. The load balancing algorithm is divided into two categories: 1. Static Load Balancing Algorithm 2. Dynamic Load Balancing Algorithm [8]


All nodes broadcast their current state information at certain time intervals. Setting time for intervals is very important. If this time is very small, it causes high overhead on the system and if it is very large, it decreases the accuracy of the system and some valuable data can be missing. Here, communication overhead is increased. B. Demand-driven policy One node collects the current state of other nodes only when it becomes overloaded or under load, i.e., if the demand of load collection comes then only load is collected according to demand of the processor This demand is based on state or some other criteria. Here communication overhead is decreased. C. State-change-driven policy Nodes publish information about their state whenever their state changes by a certain amount. If the overhead of dynamic load balancing policy

73


can be limited to a certain acceptable range, in most cases, dynamic load balancing policies outperform the static load balancing policies. However, the states of the processor depend on specific algorithms. Here communication overhead is increased. (Parimah Mohammadpour,Mohsen Sharifi, Ali Paikan-2008).[8]

In pre-emptive load balancing approach, processes in any state of heavily loaded nodes is migrated to lightly loaded node where the process state could be new born or running or waiting. (92)

D. Load distribution policy

Load balancing in the application level concentrates on minimizing the completion time and maximizing the speed of an application, whereas at system level, it concentrates on maximizing the throughput or utilization rate of the nodes. Load balancing in system level is also known as distributed scheduling. In application level load balancing, application programs are converted into parallel processes or threads [6][4]

This policy depends on heterogeneous node in clusters or homogeneous node of the system. In heterogeneous system, the load cannot be distributed among the nodes, but into homogeneous system, it needs the distribution of processes and as a result to decide load distribution policy. [1][2][4] E. Centralized and decentralized scheme In Centralized schemes, a central manager collects the information of each node and performs the decision making based on overall knowledge of the system. In a distributed load balancing scheme, each node makes decisions based on its local knowledge. A neighbor based scheme is a distributed scheme that makes efficient load decisions without having any overall knowledge about the system. In Decentralized scheme, each node can participate in the load balancing decision, but the distribution of load is dependent upon complete knowledge i.e. load of each node. Here, each node uses the same load distribution policy. Hence accordingly it distributes load using process transfer policy. (Parimah Mohammadpour, Mohsen Sharifi, Ali Paikan-2008 ) ( Janhavi B,Sunil Surve ,Sapna Prabhu-2010 ) (114)(64) F. Preemptive and non pre-emptive load balancing Load imbalance can occur due to any of the following causes: 1. Application imbalance 2. Workload imbalance 3. Heterogeneity of hardware resources In non pre-emptive load balancing approach, only those processes of heavily loaded node are migrated to lightly loaded node whose execution has not started yet. In other words, only new born processes can be migrated, running processes cannot be migrated in this system.


G. Application level and system level Load Balancing

LIMITATIONS OF EXISTING LOAD BALANCING ALGORITHM 2

Till now, it is known that load balancing algorithms are divided into static and dynamic. Limitations of existing static load balancing algorithm are given as below: 1. Static algorithm distributes processes among the cluster node before its actual execution. Hence heavily loaded node is always, remaining heavily loaded in FIFO distribution. 2. Speed–weighted random splitting policy sometimes gives better results, but many a times it also gives bad result because of an uneven execution policy of the processes to CPU nodes within the clusters. 3. Load-dependent static policy gives better results, but it will not solve the problem of heavy load processor 4. Generally static load balancing is done by the server node due to which maximum time on server is wasted in load balancing. As mentioned before, the dynamic load balancing can solve the problem of load balancing by migrating processes form heavily loaded processor to lightly loaded processor. Process migration needs to transfer all the processes with all required processing elements (i.e. Process control block, data, etc.) 1. Process distribution policy needs all knowledge about the network because each node needs to communicate with each other and it is time consuming.

74


2. The server node only performs load balancing operation in a centralized approach. 3. In decentralized approach communication overhead is high. Many authors have discussed about the benefits and drawbacks of process migration for the algorithm distribution rule. Some authors have concluded that migration cannot provide better performance than other alternatives like remote execution due to its costs, while others have argued that even when the costs of migration is high, it can significantly improve systems’ performance, especially when the systems are highly variable. The process migration cost is dependent upon various factors like total burst time and remaining burst time of the process, the size of the ready queue of the operating system, total memory requirement of the process, total input/output operations required for that process and total communication overhead required for that process.

3 Idea of Proposed Load Balancing Model The dynamic load balancing algorithm mainly removes the bottlenecks presented by the static coscheduling approach thus making the cluster coscheduling scalable. But it presents a larger communication overhead as compared to static load balancing algorithm because the dynamic load balancing algorithm performs process migration. In a centralized load balancing algorithm, load is distributed uniformly among the processors. The only disadvantage is maximum time of the central processor is wasted in load balancing rather than process execution. Hence the performance of the server decreases.A decentralized load balancing algorithm decision of load distribution is taken from the entire node hence, each node has loads of other nodes and communication overhead increases tremendously. The main disadvantages of this type of balancing is all nodes need information about all nodes hence too much communication overhead is required.In order to balance the load uniformly over a cluster system, one has to choose a mix of centralized and decentralized approach. This research study is aimed at designing a new load balancing algorithm totally dynamic in nature, focusing on reducing two important parameters taken into consideration. This communication overhead and load balancing


time depends upon the approach selected in the algorithm. The main objectives of the proposed load balancing algorithm should be explained below: 1. To reduce communication overhead within the clustered system. 2. To minimize the time of load balancing. The load balancing algorithm for the clusters can be made more robust by scheduling all jobs, irrespective of any constraints so as to balance the load perfectly. This work is extended to develop a new algorithm that modifies dynamic, decentralized approach so as to reduce the communication overhead as well as reduce migration time and also make it scalable. Till now some algorithm follows centralized approach or decentralized approach for load balancing. But in our proposed load balancing algorithm should mix the two approaches centralized as well as decentralized approach.Time and speed are the key factors of today’s modern age. Hence, everyone wants to extract supercomputing power from cluster system. This becomes possible using load balancing algorithms on the cluster. Now a day’s, clusters are used in almost all the fields. They range from safetycritical systems such as nuclear reactors and automotive, controller, network management, navigation systems, stock trading, and command and control systems to entertainment software such as games and graphics animation. This may include the moves in a chess game, sequence of web pages navigated, sequence of items bought, sequence of events happening before an equipment breakdown, sequences of medications taken to sequences of symptoms developed etc. The main aim of this research study is to design new load balancing algorithm such that it should increase the overall speed and performance of the cluster system. Design process and detailed three proposed algorithm.During the research work, new dynamic load balancing algorithms is proposed, which are explained as follows; 1. Authority Ring with Periodically Load Collection of Load Balancing (ARPLCLB) Algorithm.[23] The theme of algorithm was presented, discussed in the international conferences as well as published in their proceedings and added into the IEEE explorer digital library too. The algorithms

75


have been published during the research in reputed international journals. The features of these algorithms are as follows:

approach master node is fixed usually it is big processor. 7. The master node decision is self declared decision of any idle or lowly loaded node when the system is imbalanced.

1. The algorithm mix two approaches centralized and decentralized approach 2. In a centralized approach only central node is responsible for load balancing called master node. Master node can create an order packet for all other nodes.

4 Dynamic

3. Any idle node or lowly loaded node can become a master if and only if the system is imbalanced and it has authority packet. In centralized approach when order arises, it always choose new process or minimum executed process for process migration

This algorithm never finds out the distance between two nodes because it is developed in such clusters which are very close to each other and connected through a wire which is known as LAN.

4. When all the nodes are highly overloaded, then it stops authority ring and start delay so that it minimizes the communication overhead for some few numbers of seconds and all the nodes can give their all attention only to the execution of processes.

If there is a need to use the same algorithm for wireless sensor network, then the distance between the two nodes need to be considered. Hence, master node needs to collect the distance of each node at run time. This again needs extra communication overhead. This extra communication overhead is explained in following figure 1.

Algorithm network:

5. Decentralized approach was used at the time of load collection, and during the process migration state logic and process reselection criteria are used by all compute nodes.

Whenever any node becomes a master node then it sends zero length packets to each node to find the distance between two nodes. Once it calculates the distance, then it needs to start the authority ring between the compute nodes of the cluster.

6. According to this algorithm master, a node can be any idle node, but in centralized

Collection of Load by demand need to find distance between the nodes

Load Balancing for wireless sensor

CPU ID 1 with Idle states CPU ID 4 with heavy state

CPU ID 5 new Master

CPU ID 0 Old Master

CPU ID 4 with heavy state CPU ID 1 with Idle states


76


Figure 1 : Cluster of Wireless sensor Network This algorithm has maximum communication grid system peer to peer network system with any overhead due to this. Hence, after distance parallel computing system. collection, it collects load information. This References distance needed to be considered at the time of load balancing, hence WSN cluster needs to design [1] Bernd F reisleben Dieter Hartmann Thilo Kielmann [1997] “Parallel Raytracing A Case Study on separate load balancing algorithm.

Partitioning and Scheduling on Workstation Clusters” 1997 Thirtieth Annual Hawwaii International Conference on System Sciences.

5 CONCLUSION Faster computers are used to solve larger problems and also to find solutions more quickly, with greater accuracy, and at a lower cost. The limitation of the entire old load balancing algorithm is that it required maximum communication overhead, but at present multi-core processors are commonly used as well as some processors are dedicated to the network communication purpose, then this algorithm can play important role in the parallel processing system like cluster system.. In order to balance the load uniformly over a cluster system, centralized and decentralized approaches are mixed together and this research proposes three parallel algorithms. These algorithms are mainly designed for homogeneous nodes of the clusters. Usually on heterogeneous cluster process migration concept is not allowed because the machine language of each processor is different. These algorithms can be implemented using message passing interface. As stated in Appendix 4, there are many types of MPI instructions used for the communication of compute nodes. But out of them, it only uses MPI blocking instructions because they can pass messages in the minimum amount of time with accuracy. During the research study it has proved that latency time and bandwidth by blocking MPI instructions are very less. These blocking MPI instructions require proper synchronization between the concurrent processes. If proper synchronizations are not between the parallel processes - that means sender process and receiving process, then that means there is a deadlock between the processes. Hence, many authors /researcher argued that this needs more time for process migration, such that same time can be sufficient to execute that process in source node itself. But the third load balancing algorithm also considers process migration factor to transfer the process, hence very less processes are selected for migration. The above algorithm can be used in


[2]

Blaise Barney, (1994) Livermore Computing, MPI Web pages at Argonne National Laboratory http://www-unix.mcs.anl.gov/mpi "Using MPI", Gropp, Lusk and Skjellum. MIT Press

[3]

Erik D. Demaine, Ian Foster,Carl Kesselman, and Marc Snir [2001] “Generalized Communicators in the Message Passing Interface” 2001 IEEE transactions on parallel and distributed systems pages from 610 to 616.

[4]

Hau Yee Sit Kei Shiu Ho Hong Va Leong Robert W. P.Luk Lai Kuen Ho [2004] “An Adaptive Clustering Approach to Dynamic Load balancing” 2004 IEEE 7th International Symposium on Parallel Architectures, Algorithms and Networks (ISPAN’04)

[5]

Janhavi B,Sunil Surve ,Sapna Prabhu- 2010 “Comparison of load balancing algorithms in a Grid” 2010 International Conference on Data Storage and Data Engineering Pages from 20 to 23.

[6]

M. Snir, SW. Otto, S. Huss-Lederman, D.W. Walker and J. Dongarra,(1996) MPI: The Complete Reference (MIT Press, Cambridge, MA, 1995). 828 W. Gropp et al./Parallel Computing 22 (1996) 789828.

[7]

Marta Beltr´an and Antonio Guzm´an [2008] “Designing load balancing algorithms capable of dealing with workload variability” 2008 International Symposium on Parallel and Distributed Computing Pages from 107 to 114.

[8]

Parimah Mohammadpour, Mohsen Sharifi, Ali Paikan,[2008] “A Self-Training Algorithm for Load Balancing in Cluster Computing”, 2008 IEEE Fourth International Conference on Networked Computing and Advanced Information Management , Pages from 104 to 110.

[9]

Paul Werstein, Hailing Situ and Zhiyi Huang [2006] “Load Balancing in a Cluster Computer” Proceedings of the Seventh International Conference on Parallel and Distributed Computing, Applications and Technologies.

[10] Sharada Patil, Dr Arpita Gopal,[2012], Ms Pratibha Mandave “Parallel programming through Message Passing Interface to improving performance of clusters ” – International Docteral Conference (ISSN 0974-0597) SIOM, Wadgoan Budruk in Feb 2013.

77


[11] Sharada Patil,Arpita Gopal “Comparison of Cluster Scheduling Mechanism using Workload and System Parameters” 2011 ISSN 0974-0767 International journal of Computer Science and Application. [12] Sharada Patil,Arpita Gopal “STUDY OF DYNAMIC LOAD BALANCING ALGORITHMS FOR LINUX CLUSTERED SYSTEM USING SIMULATOR” 2011 ISSN 0974-3588 International journal of Computer Applications in Engineering Technology and Sciences. [13] Sharada Patil,Dr Arpita Gopal, [2011] “Study of Load Balancing ALgorithms” – National Conference on biztech 2011, Selected as a best paper in the conference, got first rank to the paper,DICER, Narhe ,Pune in year March 2011. [14] Sharada Patil,Dr Arpita Gopal, [2013] “Cluster Performance Evaluation using Load Balancing Algorithm” – INTERNATIONAL CONFERENCE ON INFORMATION COMMUNICATION AND EMBEDDED SYSTEMS ICICES 2013,978-1-46735788-3/13/$31.00©2013IEEE (ISBN 978-1-46735786-9) Chennai, India in year Feb 2013. [15] Sharada Patil,Dr Arpita Gopal,[2012] “Need Of New Load Balancing Algorithms For Linux Clustered System” – International Conference on Computational techniques And Artificial intelligence (ICCTAI’2012) (ISBN 978-81-922428-5-9) Penang Maleshia in year Jan 2012. [16] Sharada Patil,Dr Arpita Gopal,[2013] “Enhancing Performance of Business By Using Exctracted Supercomputing Power From Linux Cluster’s ” – International Conference on FDI 2013 (ISSN 0974-


0597) SIOM, Wadgoan Budruk in Jan 2013 [17] Sun Nian, Liang Guangmin [2010] “Dynamic Load Balancing Algorithm for MPI Parallel Computing” 2010 Pages 95 to 99 [18] William Gropp, Rusty Lusk, Rob Ross, and Rajiv Thakur [2005] “MPI Tutorials “ Retrieved from www.mcs.anl.gov/research/projects/mpi/tutorial Livermore Computing specific information: [19] Yanyong Zhang, Anand Sivasubramaniam, JoseÂ Moreira, and Hubertus Franke [2001] “Impact of Workload and System Parameters on Next Generation Cluster Scheduling Mechanisms” 2001 IEEE transactions on parallel and distributed systems Pages from 967 to 985. [20] Yongzhi Zhu Jing Guo Yanling Wang [2009] “Study on Dynamic Load Balancing Algorithm Based on MPICH” 2009 MPI_COMM_RANK: World Congress on Software Engineering. Pages from 103 to 107. [21] Michel Daydé, Jack Dongarra (2005) “High Performance Computing for Computational Science VECPAR 2004 “ISBN 3-540-25424-2 pages 120121 [22] G. Bums and R. Daoud, MPI Cubix (1994) Collective POSIX I/O operations for MPI, Tech. Rept. OSCTR- 1995- 10, Ohio Supercomputer Center, 1995. [23] Sharada Santosh Patil, Arpita N. Gopal. Authority Ring Periodically Load Collection for Load Balancing of Cluster System.American Journal of Networks and Communications. Vol. 2, No. 5, 2013, pp. 133-139. doi: 10.11648/j.a jnc.20130205.13

78