Load Balancing in Peer-to-Peer Data Networks

Load Balancing in Peer-to-Peer Data Networks David Nov´ak Masaryk University, Brno, Czech Republic [email protected]

Abstract. One of the issues considered in all Peer-to-Peer Data Networks, or Structured Overlays, is keeping a fair load distribution among the nodes participating in the network. Whilst this issue is well defined and basically solved for systems with relatively simple search paradigms, none of the existing solutions is appropriate nor applicable for similaritysearch networks. We suggest a new general solution of the load-balancing problem in P2P Data Networks, which is especially suitable for systems with time-consuming search operations. The framework proposed analyzes the source of the load precisely to choose right balancing action.

1

Introduction

Although the most widespread applications of the peer-to-peer paradigm are still the file-sharing systems, the Peer-to-Peer Data Networks (PDNs) form another important family of P2P applications. These networks, also referred to as Structured Overlays, redistribute the data or metadata between the peers and thus well combine the properties of a distributed storage with the functionality of a parallel search engine. The fact that these systems are self-organizing and dynamic by nature makes them tempting for many applications and environments. The search paradigm of the PDN has evolved over time. The Distributed Hash Tables (DHTs) with a simple key-object location have been followed by structures designed for (multi-dimensional) interval queries and, recently, also for very complex similarity queries. One of the issues that has to be considered in all types of PDN is balancing of the load among the participating peers. Talking about load-balancing, the questions arise, “What does the load mean?” and “What instruments do we have to keep a fair load distribution?” Sections 2 and 3 seek for answers to these questions. Especially, the needs of the similarity PDN are analyzed and the finding is that the current load-balancing techniques are not suitable for the similarity networks. Section 4 proposes a new, very general solution of the load-balancing problem in PDNs.

2

Current State

The Distributed Hash Tables, as the first and simplest PDNs, can solve the load balancing by hashing the key-space to the navigation-space using a pseudorandom hash function with a uniform distribution. Since such a function destroys

the locality property of the data, this approach is hardly applicable to more complex search paradigms. The majority of recent works in the area of load balancing in PDN [1–5] is motivated by the need of efficient processing of interval queries. The loadfunction balanced by these techniques is either the amount of data stored by individual nodes or the number of accesses per node. This is legitimate since the data I/O-costs are the most important aspect to be considered for the interval queries. These balancing strategies expect linearly-sorted and range-partitioned data with the option of shifting part of the data from a node to its neighbor and with the option of splitting a node into two half-loaded nodes. Another way to balance the access-load is replication of the high-loaded nodes to less loaded hosts. This concept is adopted, e.g., by the HotRoD system [6] – a DHT-based architecture for interval queries.

3

Peer-to-Peer Data Networks for Similarity Search

This work is primarily motivated by the needs of the systems [7–9] based on a very general similarity paradigm – searching in metric spaces [10]. This approach treats the dataset as a metric space – a set of objects together with a binary distance function. The search queries are specified by a query object and a constraint on the data to be retrieved. The evaluation of the metric distance function is typically computationally intensive and thus the processing of a similarity query is very time-consuming and may vary significantly for different queries that access a certain node. Therefore, the load of a node can be measured neither as the data-volume nor as the number of accesses, but should be measured as the “computational load” of given node. The goal of the load balancing is to reduce the response time of a single search query and to increase the throughput of the system. Improving these two aspects requires deeper analysis of the source of the load, since it may be caused either by time-consuming processing of one query or by high query frequency. In some similarity PDNs, e.g., in the systems based on the metric trees [7], the data are not range-partitioned and thus cannot be shifted from an overloaded node to its neighbor. Also a node cannot be always split into two half-loaded nodes. Unfortunately, these operations are essential for all current load-balancing techniques. The motivation of this work can be summarized as follows. – The load definitions used by existing balancing techniques do not correspond to the needs of the similarity PDNs. – These techniques also expect operations not available in all structures. The contribution of this work is a proposition of a novel general framework for load balancing in PDNs. – The proposed framework uses a very general model of the PDN to be balanced, which is suitable for both similarity and other P2P systems.

– The framework focuses especially on systems with time-consuming search operations and analyzes deeply the source of the load. – It uses both replication and data-volume adjustment depending on the precise source of the overload.

4

A General Load-Balancing Framework for PDN

In this section, we describe a new framework for balancing load in PDNs. Although it has been designed with a focus on the systems with time-consuming query-processing, it is applicable on any PDN. 4.1

Two-Layer Model

An overload of a node may be caused by a large data volume stored at the node. One of the operations suitable in such a situation is shifting part of the data to a less loaded neighboring node. Unfortunately, this operation is not available in all structures since the data are not necessarily totally sorted and range-partitioned. A general solution of this problem is supporting existence of several “logical nodes” at one physical host. This provides an option of splitting the overloaded node and moving a new node to a less loaded host. Another motivation for using this model is supporting systems with multiple-overlays architecture, for which several nodes per host is a basic requirement. The descriptions below summarize the notation used in the following text. Nodes are the logical units of the structure being balanced and form the logical layer. Each node consists of the storage (data with a search mechanism) and the routing structure (links to other nodes and the navigation strategy). Hosts refer to the physical computers. The logical nodes are mapped to the set of physical hosts. In principle, a peer is created by realization of a node at a physical host. The two-layer architecture is sketched out in Figure 1. The load-balancing operations may modify the logical layer, whereas the set of available hosts is managed from the outside of the system. 4.2

System Operations

The usage of the balancing framework is fully transparent for the outside environment – all operations of the particular PDN such as Insert or Search can be posed to the system. In general, when a node receives a search request, it can either forward it to other nodes or evaluate it on its local data or both. In the framework, the forwarding has always priority over all local-data search requests at particular host. The load-balancing mechanism treats the set of hosts as fixed and exploits the following balancing operations that affect the logical layer of the system. The Split operation splits the data of a node equally (if possible) and place the

Node 7 Node 3

Node 3

Node 6

Node 2

Node 5

Node 4

Node 2

NETWORK Node 5

Node 1

Node 1 Node 4

Node 6

Node 7

(a)

(b)

Fig. 1. The logical layer (a). Mapping of the nodes on the hosts (b).

new node at another (typically less-loaded) host. This operation is supported by all PDNs known. The complementary operation, Leave, is either implemented as a merge of the node with some other node (if supported) or the data are re-inserted to the structure after a node deletion. The load-balancing may require to Migrate a node to another – typically less-loaded – host. If the migrated node knows all nodes that point to it, then this operation can be implemented by notifying all these nodes about the host change. Otherwise, the Migrate operation is realized as a pair of consecutive operations leave and join standardly provided by all PDNs. The framework exploits replication as one of the load-balancing mechanisms. Operation Replicate creates a replica of the specified node at another host. Once a node is replicated, it can decide to forward a search query to a selected replica instead of processing the query locally. Decisions about query forwarding are always made at the main node – the replicas do not have any routing information. The Unify operation removes a specific replica of given node. The schemas of the load-balancing operations are depicted in Figure 2.

REPLICATE Node 5’ Node 5

Node 3

Node 3 Node 3

Node 4

SPLIT Node 8 Node 8

L EAVE

Node 5’

Node 1 Node 2 Node 7

Node 6

Fig. 2. The Split, Leave and Replicate operations.

4.3

Measuring the Load

The cost function considered by our balancing mechanism is the computational load of the hosts and nodes in the system. The source of the load is analyzed precisely in order to select the best balancing action. There are two general causes of an overload: – A query evaluation on the local data is too time-consuming. – A query is waiting for processing since other operations are being processed. While the first situation primarily increases the response time of a single (even isolated) search-query, the second one influences the throughput of the system. According to this observation, the load is measured as two separate quantities. – The processing load is taken as an average time of processing a search query on the local data of a given node. This value considers the data volume, the complexity of the queries, and, eventually, quality of the local index. – The waiting load is measured as an average time a query is waiting in the queue before processed. This value is measured within the scope of the whole physical host and it considers the frequency of queries and the processing load of particular nodes at this host. Both quantities are also influenced by the computational performance of the physical host and by various outer factors. The definitions above contain the word “average”. The average can be taken over a time slot or over a number of processed queries. Since the processing load is independent of the query frequency, this quantity is measured as an average over several last processed queries. On the other hand, the waiting load takes into account the query frequency, therefore this average is taken over a set of queries that fit into a time sliding window. 4.4

Balancing Strategies

The following schemas express the load-balancing strategies applied by every host in the framework. If the waiting load is too large then if the waiting is caused by more then one node then Migrate one of these nodes to another host, else Replicate the load-causing node. If the processing load of a node is too large then Split the node to another host. If the waiting load is small and a replica exists then apply the Unify operation (remove a replica). If the processing load is small then Leave the node. The schemas contain vague terms “too large/small”. The values must be relative to the actual load of the whole system (of other hosts). Each host maintains actual approximations of the global averages of the load quantities. The

information is maintained by a distributed peer-to-peer algorithm [11] based on gossips. The algorithm exploits standard messages (query, insert, management) to exchange the data and increases the load of the network only negligibly. Having actual approximations of the global load values, we can specify that “too large” means over two-times average and “too small” means under half an average. Information about the least-loaded hosts in the system is shared together with the standard messages as well. Figure 3 provides a simple schema of the load-balancing module present at every host.

processing_load waiting_load

S PLIT /L EAVE

Load−balancing module

MIGRATE

average loads REPLICATE/UNIFY

NETWORK

low−loaded peers

Fig. 3. Work schema of the load-balancing module.

5

Conclusions and Future Work

The requirements of the Peer-to-Peer Data Networks for similarity search are beyond the potential of current load-balancing techniques. There are two arguments for this statement: (1) The time demands of the similarity-queries processing are extreme and different for particular network nodes, while the current balancing strategies take into account only either the data volume or pure number of accesses. (2) Not all operations required by the existing balancing techniques are available in the similarity Peer-to-Peer Data Networks. We analyzed requirements of such networks and proposed a load-balancing framework which considers the computational load of the peers, analyzes the precise source of an overload, and chooses appropriate balancing action. To keep a fair load distribution, the framework exploits replication of nodes and controlled data splitting together with a two-level architecture allowing several logical nodes at one physical host. At the moment of writing, the balancing system is being implemented with the view of running comprehensive experiments on several P2P structures for similarity search [7–9]. We expect a verification of the theoretical studies presented in this paper and evaluation of efficiency of the proposed balancing strategies. The lessons learned from the trials will be exploited for further improvement and tuning of the proposed balancing framework.

References 1. Aberer, K., Datta, A., Hauswirth, M.: The Quest for Balancing Peer Load in Structured Peer-to-Peer Systems. Technical report, EPFL, Swiss (2003) 2. Ganesan, P., Bawa, M., Garcia-Molina, H.: Online balancing of range-partitioned data with applications to peer-to-peer systems. Technical report, Stanford U. (2004) 3. Crainiceanu, A., Linga, P., Machanavajjhala, A., Gehrke, J., Shanmugasundaram, J.: P-Ring: An index structure for peer-to-peer systems. Technical Report TR20041946, Cornell University, NY (2004) 4. Aspnes, J., Kirsch, J., Krishnamurthy, A.: Load balancing and locality in rangequeriable data structures. In: PODC ’04: Proceedings of the twenty-third annual ACM symposium on Principles of distributed computing, New York, NY, USA, ACM Press (2004) 115–124 5. Karger, D.R., Ruhl, M.: Simple efficient load balancing algorithms for peer-topeer systems. In: SPAA ’04: Proceedings of the sixteenth annual ACM symposium on Parallelism in algorithms and architectures, New York, NY, USA, ACM Press (2004) 36–43 6. Theoni Pitoura, Nikos Ntarmos, P.T.: Replication, load balancing and efficient range query processing in DHTs. In: Proceedings of the International Conference on Extending Database Technology (EDBT), Munich, Germany. (2006) 7. Batko, M., Gennaro, C., Zezula, P.: Similarity grid for searching in metric spaces. DELOS Workshop: Digital Library Architectures, Lecture Notes in Computer Science 3664/2005 (2005) 25–44 8. Falchi, F., Gennaro, C., Zezula, P.: A content-addressable network for similarity search in metric spaces. In: Proceedings of the 3rd International Workshop on Databases, Information Systems, and Peer-to-Peer Computing (DBISP2P 2005), Trondheim, Norway, August 28 - 29, 2005. (2005) 126–137 9. Novak, D., Zezula, P.: M-Chord: A scalable distributed similarity search structure. In: Proceedings of First International Conference on Scalable Information Systems (INFOSCALE 2006), Hong Kong, May 30 – June 1, 2006, ACM Press (2006) 10. Zezula, P., Amato, G., Dohnal, V., Batko, M.: Similarity Search: The Metric Space Approach. Volume 32 of Advances in Database Systems. Springer-Verlag (2006) 11. Kempe, D., Dobra, A., Gehrke, J.: Gossip-based computation of aggregate information. In: FOCS ’03: Proceedings of the 44th Annual IEEE Symposium on Foundations of Computer Science, Washington, DC, USA, IEEE Computer Society (2003) 482