replica catalog, the Grid Data Mirroring Package (GDMP), the Giggle framework and ... Among database operations, Join is a commonly used, but complex,.
Join Algorithm Using Multiple Replicas in Data Grid* Donghua Yang1, Jianzhong Li1,2, and Qaisar Rasool1 1
School of Computer Science and Technology, Harbin Institute of Technology, China 2 School of Computer Science and Technology, Heilongjiang University, China {yang.dh, lijzh, qrasool}@hit.edu.cn
Abstract. We address the issue of applying join operation to the relational data managed in the form of multiple replicas in data grid. A join algorithm is developed that solves the problem of replica matching and execution node selection by making use of matching theory. To ease the data transference among the grid nodes, we introduce relation reduction and row blocking techniques. The analytical and experimental results depict the effectiveness of the proposed join algorithm for the better network transfer rate and the efficient processing of replicated data to improve query responses in data grid.
1 Introduction Data grid [1,2,3,4] is a distributed architecture for data management that provides the coordinated and collaborative mechanisms for integrating data distributed across network and forms a single, virtual environment for data access and management. The aim is to facilitate world-wide community with geographically-distributed resources for large-scale data-intensive applications and providing fast, reliable and transparent access to data. Many applications can be developed in data grid environment, such as most prominent being global climate simulation, high energy physics and biology computing. Data replication [5] is an optimization technique well known in the distributed systems and database communities as a means of achieving better access times to data, fault tolerance and increasing the performance and robustness of distributed applications. Various replica management systems [6,7,8] exist, such as the Globus MDS-2 replica catalog, the Grid Data Mirroring Package (GDMP), the Giggle framework and the SDSC’s Storage Resource Broker (SRB). These systems differ in several architectural aspects but allow consistent and locatable replicas, with a support of managed replicas’ lifetime. The roles which the database management systems are contributing towards data storage, access, management and transference in distributed environment [9,10,11] is tremendous. Among database operations, Join is a commonly used, but complex, operation that requires more time to complete than other operations and is an active area of attention [12,13]. Existence of autonomous nodes, massive and heterogeneous *
This work is supposed by the National Natural Science Foundation of China, Grant No. 60273082 and 60473075, the key Natural Science Foundation of Heilongjiang Province, Grant No. zjg03-05.
W. Fan, Z. Wu, and J. Yang (Eds.): WAIM 2005, LNCS 3739, pp. 416 – 427, 2005. © Springer-Verlag Berlin Heidelberg 2005
Join Algorithm Using Multiple Replicas in Data Grid
417
datasets and the different and unstable bandwidths among nodes brings new challenges to the Join operation in data grid environment. In this paper, a join algorithm is proposed that facilitates the queries involving join operation to access the replicated relational data over the grid. Our work facilitates the users who issue the queries to access the data via grid that involves the join operation. Firstly a reduction algorithm is applied to the data involved. Second grid nodes, termed as execution nodes, are selected on which query (join operation) will be performed. The reduced data from the first step is then transferred over the grid to the execution nodes for processing and the results are propagated back to users’ nodes. For convenience in data transference and data size we suggest new reducing and merging algorithms and sending techniques. The influence of various parameters on the performance of algorithm is studied by experiments, showing the efficiency and usefulness of the work. The rest of the paper is organized as follows. Section 2 introduces algorithms for relation reduction, block transferring and merging. In Section 3 we discuss Edgeweight-minimum-matching algorithms by employing some important concepts from Graph Theory. Section 4 explores the proposed Join Algorithm in five phases. The results of experiments to measure the algorithm performance are given in Section 5. Finally we conclude the paper and highlight some future work in Section 6.
2 Preliminaries We start with some preliminaries. These include relation reduction algorithm, row blocking data transfer mode and block merge join algorithm. 2.1 Relation Reduction Algorithm Definition 1. Let R, S be the two relations that need to be joined based on the join attribute T. Let R′ and S′ be the subsets of R and S respectively consisting of tuples that satisfy the join condition. If |R′| and |S′| represent the sizes of R′ and S′ respectively, then the selection rate of relations R and S upon the join attribute T is ρR=|R′|/|R|, ρS=|S′|/|S| respectively. We take example of two relations R and S located respectively at some arbitrary nodes A and B in data grid and need to be joined according to some join attribute T. If the selection rates of R and S are small, i.e., only small number of tuples in R and S satisfy the join condition then we need to transfer only these tuples instead of all the tuples in R and S to the execution node where join operation will be performed. The steps of relation reduction algorithm are as follows: Step1. Get T[R] and T[S] from nodes A and B respectively, where T[R] and T[S] are the projection results of R and S according to join attribute T by using commonly known sort-based projection algorithm. Step2. Transfer T[R] from A to B and T[S] from B to A. At node A, get R′ which is the join result of T[S] and R, similarly at node B, get S′ which is the join result of T[R] and S. R′ and S′ are separately the subset of R and S which satisfy the join condition.
418
D. Yang, J. Li, and Q. Rasool
Because of using sort-based projection algorithm, the tuples in R′ and S′ are either ranked in non-decreasing order or in non-increasing order on join attribute T. We assume the order is non-decreasing in this paper. Lemma 1. (1) The join of two relations R and S is equal to the join of R′ and S′ where R′ and S′ are subsets of R and S respectively satisfying the join condition. That is, R >< S=R′ >< S′. (2) R′ and S′ are minimal, that is, there is no pair of relations , such that R′′⊂R′, S′′⊂ S′, and R′′ >< S′′=R >< S. Proof: (1) R′ >< S′=(T[S] >< R) >< (T[R] >< S)=(T[R] >< R) >< (T[S] >< S)=R >< S (2) Assume that there exist the other two relations R′′ and S′′, R′′⊂R′, S′′⊂S′, such that R′′ >< S′′=R >< S. There exists a tuple t, t∈R′and t∉R′′. t does not participate the join operation R′′ >< S′, due to R′′ >< S′=R′ >< S′=R >< S, so t also participate the join operation R >< S, it is obviously contradiction. Hence R′′=R′ and R′ is the minimal. Similarly S′ is minimal. □ 2.2 Row Blocking Data Transfer Mode For transferring datasets from one grid node to others, we suggest row blocking data transfer technique in which tuples are shipped from one grid node to others in blocks. In traditional approach, tuples are shipped one by one through the network. Therefore any short delay in the network would immediately stop the execution of the query at the receiving node because of a shortage of tuples to consume. But in row blocking, the receiving node would have a reservoir of tuples and thus can feed operator even if the tuples of next block is delayed. In this way, the approach compensates for delay in data arrival up to a certain extent. 2.3 Block Merge Join Algorithm Suppose node E is the execution node in which join operation is to be performed over relations R′ and S′. First, we need to set up two input buffers namely IBR and IBS for R′ and S′ respectively. Further we assume two memory spaces MR and MS allocated for R′ and S′ respectively to store received tuples from R′ and S′. Now, for each block of R′ and S′, there are two phases to complete the join operation at node E: (1) insertion phase, each block is inserted into its corresponding memory location. (2) match phase, given a variable h1, for each block R′[h1], first we find every matched block from MS, secondly each tuple in block R′[h1] is probed with the tuples in every matched block and if join condition is satisfied, these two tuples are merged and the result is produced into output buffer. Similarly, for each block S′[h2], the same operation is repeated. We explain the algorithm in detail as follows
：
1.
Set two variants PR and PS for MR and MS respectively, with initial values 1, set the two Boolean variants FR and FS for MR and MS respectively, with initial values False. FR and FS are used to control the changes in PR and PS.
Join Algorithm Using Multiple Replicas in Data Grid
2.
419
When node E receives a block and inserts it into its corresponding memory, it records the minimum and maximum of join attribute T. That is, it records the values of T in the first tuple and the last tuple and builds a value-range for the block. In a similar way we get a value-range for each block in MR and MS. Before merge join operation of two blocks, we can inspect the two valueranges of the blocks. Once there is an overlap between newly inserted block R′[h1] and some block S′[h2] in MS we set FS equal to True. If there is no overlap and FS is False, set PS=PS+1. When a new inserted block R′[h1] in MR wants to probe with some block in MS, it is better to probe from S′[PS]. Because tuples in block S′[PSk](1≤k≤PS-1) have been probed and cannot satisfy join condition. Similarly, when a new inserted block S′[h2] in MS wants to probe with some block in MR, it should probe from R′[PR]. S′[k](k
