Cascading Map-Side Joins over HBase for Scalable Join ... - arXiv

Cascading Map-Side Joins over HBase for Scalable Join Processing Martin Przyjaciel-Zablocki

arXiv:1206.6293v1 [cs.DB] 27 Jun 2012

University of Freiburg, Germany

Alexander Schätzle

Thomas Hornung



[email protected] [email protected]@informatik.unifreiburg.de freiburg.de freiburg.de Christopher Dorner Georg Lausen University of Freiburg, Germany


[email protected]

[email protected]

ABSTRACT

datasets at web-scale is challenging, especially because the computation of SPARQL queries usually requires several joins between subsets of the data. On the other side, classical single-place machine approaches have reached a point where they cannot scale with respect to the ever increasing amount of available RDF data (cf. [17]). An intuitive solution is to parallelize the execution of query processing over a cluster of machines. Renowned for its excellent scaling properties, the MapReduce paradigm [9] is a good fit for such a distributed SPARQL engine. Following this avenue, we introduced the PigSPARQL project in [28]. PigSPARQL offers full support for SPARQL 1.0 and is implemented on top of the Apache Hadoop2 and Pig3 platform. Subsequent evaluations of our prototype have shown that one of the major performance bottlenecks for SPARQL engines on MapReduce is the computation of joins. We have identified two major reasons for this shortcoming:

One of the major challenges in large-scale data processing with MapReduce is the smart computation of joins. Since Semantic Web datasets published in RDF have increased rapidly over the last few years, scalable join techniques become an important issue for SPARQL query processing as well. In this paper, we introduce the Map-Side Index Nested Loop Join (MAPSIN join) which combines scalable indexing capabilities of NoSQL storage systems like HBase, that suffer from an insufficient distributed processing layer, with MapReduce, which in turn does not provide appropriate storage structures for efficient large-scale join processing. While retaining the flexibility of commonly used reduce-side joins, we leverage the effectiveness of map-side joins without any changes to the underlying framework. We demonstrate the significant benefits of MAPSIN joins for the processing of SPARQL basic graph patterns on large RDF datasets by an evaluation with the LUBM and SP2 Bench benchmarks. For most queries, MAPSIN join based query execution outperforms reduce-side join based execution by an order of magnitude.

1.

1. The Hadoop File System (HDFS) - a clone of the Google File System [14] - provides no sufficient built-in index structures. 2. A MapReduce-style join is typically computed during the reduce phase. As a side-effect, join partitions have to be sent across the network.

INTRODUCTION

Most of the information in the classical ”Web of Documents” is designed for human readers which makes it difficult for computers to automatically gather and process this data. The idea behind the Semantic Web is to build a ”Web of Data” that enables computers to understand and use the information in the web [5]. The advent of this Web of Data gives rise to new challenges with regard to query evaluation on the Semantic Web. The core technologies of the Semantic Web are RDF (Resource Description Framework) for representing data in a machine-readable format and SPARQL for querying RDF data. In the last few years, more and more data has been published in RDF or other machine-readable formats. One of the most prominent examples is the Linking Open Data Cloud 1 , a collection of interlinked RDF repositories with billions of RDF triples in total. Querying RDF

In this paper we present the Map-Side Index Nested Loop Join (MAPSIN join), a completely map-side based join technique that uses HBase as underlying storage layer. Our evaluation shows an improvement of up to one order of magnitude over the common reduce-side join. Overall, the major contributions of this paper are as follows: • We describe a space-efficient storage schema for large RDF graphs in HBase while retaining favourable access characteristics. By using HBase instead of HDFS, we can avoid shuffling join partitions across the network and instead only access the relevant join partners in each iteration. 2

1

http://www.w3.org/wiki/SweoIG/TaskForces/ CommunityProjects/LinkingOpenData

3

1

http://hadoop.apache.org http://pig.apache.org

Query 1: SPARQL Basic Graph Pattern Example SELECT * WHERE { ?article title ?title . ?article author ?author . ?article year ?year }

• We present the MAPSIN join algorithm, which can be evaluated cascadingly in subsequent MapReduce iterations. In contrast to other approaches, we do not require an additional shuffle and reduce phase in order to preprocess the data for consecutive joins. Our approach solely relies on the complementary intertwining of a NoSQL data store and the MapReduce framework. • We demonstrate an optimization of the basic MAPSIN join algorithm for the efficient processing of multiway joins. This way, we can save n MapReduce iterations for star join queries with n + 2 triple patterns.

There are other interesting SPARQL operators like FILTER and OPTIONAL that allow more sophisticated queries. For a detailed definition of the SPARQL syntax we refer the interested reader to the official W3C Recommendation [2]. A formal definition of the SPARQL semantics can also be found in [25]. In this paper we focus on efficient join processing with MapReduce and NoSQL (i.e. HBase) and therefore only consider SPARQL BGPs.

The paper is structured as follows: Section 2 provides a brief introduction to the technical foundations for this paper. Section 3 describes our RDF storage schema for HBase, while Section 4 presents the MAPSIN join algorithm. We continue with a presentation of the evaluation of our approach in Section 5, followed by a discussion of related work in Section 6. We conclude in Section 7 and give an outlook on future work.

2.

2.2

The MapReduce programming model was originally introduced by Google in 2004 [9] and enables scalable, fault tolerant and massively parallel computations using a cluster of machines. The basis of Google’s MapReduce is the distributed file system GFS [14] where large files are split into equal sized blocks, spread across the cluster and fault tolerance is achieved by replication.

BACKGROUND

In this chapter we briefly introduce the necessary core technologies used in our work, i.e. RDF, SPARQL, MapReduce, and HBase.

2.1

MapReduce

Map phase

RDF & SPARQL

RDF [1] is the W3C recommended standard model for representing knowledge about arbitrary resources, e.g. articles and authors. An RDF dataset consists of a set of so-called RDF triples in the form (subject, predicate, object) that can be interpreted as ”subject has property predicate with value object”. For clarity of presentation, we use a simplified RDF notation in the following. It is possible to visualize an RDF dataset as directed, labeled graph where every triple corresponds to an edge (predicate) from subject to object. Figure 1 shows an RDF graph with information about articles and corresponding authors. "PigSPARQL" "2011"

title

Article1

author

Alex

author

cite

author

input0

Input3

Martin

author

Article2

title

output0

Mapper

Reducer

output1

Figure 2: MapReduce Dataflow The workflow of a MapReduce program is a sequence of MapReduce iterations each consisting of a Map and a Reduce phase separated by a so-called Shuffle & Sort phase (cf. Figure 2). The input data is automatically split in blocks and distributed over the cluster to enable parallel execution. A user has to implement map and reduce functions which are automatically executed in parallel on a portion of the data. The Mappers invoke the map function for every record of their input dataset represented as a key-value pair. The map function outputs a list of new intermediate key-value pairs which are then sorted according to their key and distributed to the Reducers such that all values with the same key are sent to the same Reducer. The reduce function is invoked for every distinct key together with a list of all according values and outputs a list of values which can be used as input for the next MapReduce job. The signatures of the map and reduce functions are therefore as follows:

year

year

Reducer Mapper

Input4 Input5

Reduce phase

Mapper

input1 Input2

Shuffe & Sort

"2011" "RDFPath"

Figure 1: RDF graph SPARQL is the W3C recommended declarative query language for RDF. A SPARQL query defines a graph pattern P that is matched against an RDF graph G. This is done by replacing the variables in P with elements of G such that the resulting graph is contained in G (pattern matching). The most basic constructs in a SPARQL query are triple patterns, i.e. RDF triples where subject, predicate and object can be variables, e.g. (?s, p, ?o). A set of triple patterns concatenated by AND (.) is called a basic graph pattern (BGP) as illustrated in the following query. The query asks for all articles with known title, author and year of publication. The result of a BGP is computed by joining the variable mappings of all triple patterns on their shared variables, in this case ?article.

map: (inKey, inValue) -> list(outKey, tmpValue) reduce: (outKey, list(tmpValue)) -> list(outValue) We use Hadoop [34] as it is the most popular open-source implementation of Google’s GFS and MapReduce framework that is used by many companies like Yahoo!, IBM or Facebook. Hadoop is enriched with an extensive ecosystem of related projects that are either built on top of Hadoop or use Hadoop for various application fields.

2

Map-Side vs. Reduce-Side Join.

(denoted as cell) are timestamped and thus support multiple versions where the user can specify how many versions should be stored. HBase tables are dynamically split into regions of contiguous row ranges with a configured maximum size. When a region becomes too large, it is automatically split into two regions at the middle key (auto-sharding). Each region is assigned to exactly one so-called region server in an HBase cluster whereas a region server can serve many regions. All columns of a column family within a region are stored together in the same low-level storage file in HDFS. As these files are ordered and indexed by row key, table lookups and range scans based on row key can be done efficiently. However, HBase has neither a declarative query language like SQL nor a built-in support for native join processing, leaving higher-level data transformations to the overlying application layer. In our approach we propose a map-side join strategy that leverages the implicit index capabilities of HBase to overcome the usual restrictions of map-side joins as outlined in Section 2.2.

Processing joins with MapReduce is a challenging task as datasets are typically very large [6, 21]. If we want to join two datasets with MapReduce, L 1 R, we have to ensure that the subsets of L and R with the same join key values can be processed on the same machine. For joining arbitrary datasets on arbitrary keys we generally have to shuffle data over the network or choose appropriate pre-partitioning and replication strategies. The most prominent and flexible join technique in MapReduce is called Reduce-Side Join [6, 21, 22, 34]. Some literature also refer to it as Repartition Join [6] as the basic idea is based on reading both datasets (map phase) and repartition them according to the join key (shuffle phase). The actual join computation is done in the reduce phase. The main drawback of this approach is that both datasets are completely transferred over the network regardless of the join output. This is especially inefficient for selective joins and consumes a lot of network bandwidth. Another group of joins is based on getting rid of the shuffle and reduce phase to avoid transferring both datasets over the network. This kind of join technique is called Map-Side Join since the actual join processing is done in the map phase [34]. The most common one is the Map-Side Merge Join [21, 22]. However, such a join cannot be applied on arbitrary datasets. A preprocessing step is necessary to fulfill several requirements: datasets have to be sorted and equally partitioned according to the join key. If the preconditions are fulfilled, the map phase can process an efficient parallel merge join between pre-sorted partitions and data shuffling is not necessary. However, if we want to compute a sequence of joins, the shuffle and reduce phases are needed to guarantee that the preconditions for the next join iteration are fulfilled. Therefore, map-side joins are generally hard to cascade and the advantage of avoiding a shuffle and reduce phase is lost. Our MAPSIN join approach is designed to overcome this drawback by using the distributed index of a NoSQL storage system like HBase.

2.3

3.

RDF STORAGE SCHEMA FOR HBASE

In contrast to relational databases, NoSQL storage systems do not have a common data model in general. This makes it impossible to define a universal storage schema for RDF that holds for any kind of NoSQL system. Furthermore, there is no common query language like SQL and many systems do not have any query language at all. Hence, the implementation of our join approach strongly relies on the actual NoSQL system used as backend. In our initial experiments we considered HBase and Cassandra [16], two popular NoSQL systems with support for MapReduce. We decided to use HBase for our implementation as it proved to be more stable and also easier to handle in our cluster since HBase was developed to work with Hadoop from the beginning. In [31] the authors adopted the idea of Hexastore [33] to index all possible orderings of an RDF triple for storing RDF data in HBase. This results in six tables in HBase allowing to retrieve results for any possible SPARQL triple pattern with a single GET on one of the tables (except for a triple pattern with three variables). However, as HDFS has a default replication factor of three and data in HBase is stored in files on HDFS, an RDF dataset is actually stored 18 times using this schema. But it’s not only about storage space, also loading a web-scale RDF dataset into HBase becomes very costly and consumes many resources. Our storage schema for RDF data in HBase is inspired by [11] and uses only two tables, Ts po and To ps . We extend the schema with a triple pattern mapping that leverages the power of predicate push-down filters in HBase to compensate possible performance shortcomings of a two table schema compared to the six table schema. Furthermore, we improve the scalibility of the schema by introducing a modified row key design for class assignments in RDF which would otherwise lead to overloaded regions constraining both scalability and performance. In Ts po table an RDF triple is stored using the subject as row key, the predicate as column name and the object as column value. If a subject has more than one object for a given predicate (e.g. an article having more than one author), these objects are stored as different versions in the same column. The notation Ts po indicates that the table

HBase

HBase [13] is a distributed, scalable and strictly consistent column-oriented NoSQL storage system, inspired by Google’s Bigtable [8] and well integrated into Hadoop, the most popular open-source implementation of Google’s MapReduce framework. Hadoop’s distributed file system, HDFS, is designed for sequential reads and writes of very large files in a batch processing manner but lacks the ability to access data randomly in close to real-time [13]. HBase can be seen as an additional storage layer on top of HDFS that supports efficient random access. The data model of HBase corresponds to a sparse multi-dimensional sorted map with the following access pattern: (T able, RowKey, F amily, Column, T imestamp) → V alue Data is stored in tables that are structurally different from tables in relational databases. The rows of a table are sorted according to their row key and every row can have an arbitrary number of columns. However, in contrast to relational tables, two distinct rows of an HBase table do not have to have the same columns. Columns are grouped into column families that must be defined at the time of table creation and should not change too often whereas columns can be added and deleted dynamically as needed. Column values 3

is indexed by subject. Table To ps follows the same design. In both tables there is only one single column family that contains all columns. Tables 1 and 2 illustrate how the RDF graph in Section 2.1 would be represented in the HBase storage schema.

Table 3: SPARQL triple pattern HBase predicate push-down filters pattern table (s, p, o) Ts po or To ps (?s, p, o) To ps (s, ?p, o) Ts po or To ps (s, p, ?o) Ts po (?s, ?p, o) To ps (?s, p, ?o) Ts po or To ps (SCAN) (s, ?p, ?o) Ts po (?s, ?p, ?o) Ts po or To ps (SCAN)

Table 1: Ts po table for RDF graph in Section 2.1 rowkey family:column→value Article1 p:title→{”PigSPARQL”}, p:year→{”2011”}, p:author→{Alex, Martin} Article2 p:title→{”RDFPath”}, p:year→{”2011”}, p:author→{Martin, Alex}, p:cite→{Article1}

Table 2: To ps rowkey ”2011” ”PigSPARQL” ”RDFPath” Alex Article1 Martin

mapping using filter column & value column value column column

gions the resources of a single machine become a bottleneck for scalability. To circumvent this problem we use a modified To ps row key design for triples with predicate rdf:type. Instead of using the object as row key we use a compound row key of object and subject, e.g. (foaf:Person|Alex). As a result, we can not access all resources of a class with a single GET but as the corresponding rows of a class will be consecutive in To ps we can use an efficient range SCAN starting at the first entry of the class.

table for RDF graph in Section 2.1 family:column→value p:year→{Article1, Article2} p:title→{Article1} p:title→{Article2} p:author→{Article1, Article2} p:cite→{Article2} p:author→{Article2, Article1}

4.

At first glance, this storage schema seems to have some serious performance drawbacks when compared to the six table schema in [31] since there are only indexes for subjects and objects. In the six table schema there is, for example, also an index for the combination of subject and predicate, resulting in a simple GET on the corresponding table for a triple pattern with bounded subject and predicate. Using Ts po table, we can only make a GET for the given subject and filter the result for the given predicate. However, we can use the HBase Filter API to specify an additional column filter for the GET request. This filter is applied directly on server side such that no unnecessary data must be transferred over the network (predicate push-down). As already mentioned in [11], having a table with predicates as row keys has some serious scalability problems since the number of predicates in an ontology is usually fixed and relatively small which results in a table with just a few very fat rows. Considering that all data in a row is stored on the same machine, the resources of a single machine in the cluster become a bottleneck. Indeed, if only the predicate in a triple pattern is given, we can use the HBase Filter API to answer this request with a table SCAN on Ts po or To ps using the predicate as column filter. Table 3 shows the mapping of every possible triple pattern to the corresponding HBase table. Overall, experiments on our cluster showed that the two table schema with server side filters has similar performance characteristics compared to the six table schema but uses only one third of storage space. Our experiments revealed some serious scaling limitations of the storage schema caused by the To ps table. In general, an RDF dataset uses a relatively small number of classes but contains many triples that link resources to classes, e.g. (Alex, rdf:type, foaf:Person). Thus, using the object of a triple as row key means that all resources of the same class will be stored in the same row. With increasing dataset size these rows become very large and exceed the configured maximum region size resulting in overloaded regions that contain only a single row. Since HBase cannot split these re-

MAPSIN JOIN

The major task in SPARQL query evaluation is the computation of joins between triple patterns, i.e. a basic graph pattern. However, join processing on large RDF datasets, especially if it involves more than two triple patterns, is challenging [21]. Section 2.2 discussed the most common join techniques in MapReduce and stated the importance of reducing the amount of data transferred over the network as well as typical drawbacks of map-side join approaches. Our approach combines the scalable storage capabilities of NoSQL datastores (i.e. HBase), that suffer from a suitable distributed processing layer, with MapReduce, a highly scalable and distributed computation framework, which in turn does not support appropriate storage structures for large scale join processing. This allows us to catch up with the flexibility of reduce-side joins while utilizing the effectiveness of a map-side join that does not need a shuffle and reduce phase without any changes to the underlying Hadoop framework. First, we introduce the needed SPARQL terminology analogous to [25]: Let V be the infinite set of query variables and T the set of valid RDF terms. Definition 1. A (solution) mapping µ is a partial function µ : V → T . We call µ(?v) the variable binding of µ for ?v. Abusing notation, for a triple pattern p we call µ(p) the triple pattern that is obtained by substituting the variables in p according to µ. The domain of µ, dom(µ), is the subset of V where µ is defined and the domain of p, dom(p), is the subset of V used in p. The result of a SPARQL query is a multiset of solution mappings Ω. Definition 2. Two solution mappings µ1 , µ2 are compatible if, for every variable ?v ∈ dom(µ1 ) ∩ dom(µ2 ), it holds that µ1 (?v) = µ2 (?v). It follows that solution mappings with disjoint domains are always compatible and the set-union (merge) of µ1 and µ2 , µ1 ∪ µ2 , is also a solution mapping.

4

4.1

2

Base Case

4.2

Cascading Joins

Let us assume our RDF graph is stored in a NoSQL storChains of concatenated triple patterns require some slight age system like HBase as described in Section 3 and therefore modifications to the previously described base case. To comspread across a cluster of machines. If we want to process a pute a query of at least three triple patterns we have to basic graph pattern of two concatenated triple patterns p1 process several joins successively, e.g. p1 1 p2 1 p3 . The and p2 , we have to compute the join p1 1 p2 that merges processing of the first two patterns p1 1 p2 correspond to the the compatible mappings of two multisets of solution mapbase case and the results are stored in HDFS. The additional pings. Hence, it is necessary that subsets of both multisets triple pattern p3 is then joined with the solution mappings of mappings are brought together such that it is ensured of p1 1 p2 . To this end, an additional map-phase (without that all compatible mappings can be processed on the same any intermediate shuffle or reduce phase) is initialized with machine. the previously computed solution mappings as input. Since Our MAPSIN join technique computes the join between these mappings reside in HDFS, they are retrieved locally p1 and p2 in a single map phase. At the beginning, the in parallel such that the map function gets invoked for each map phase is initialized with a parallel distributed HBase solution mapping µ2 of p1 1 p2 . The compatible mappings table scan for the first triple pattern p1 where each mafor p3 are retrieved using the same strategy as for the base chine retrieves only those mappings that are locally availcase, i.e. µ2 is used to substitute the shared variables in p3 able. This is achieved by utilizing a mechanism for allocatand compatible mappings are retrieved following the triple ing local records to map functions, which is supported by pattern mapping outlined in Table 3. the MapReduce input format for HBase automatically. We implemented a basic heuristic called variable counting The map function is invoked for each retrieved mapping to reorder triple patterns in a SPARQL BGP by selectivity µ1 for p1 . To compute the partial join between p1 and p2 based on the number and position of variables in a patfor the given mapping µ1 , the map function needs to retrieve tern [30]. The heuristic assumes that triple patterns with those mappings for p2 that are compatible to µ1 based on one variable are more selective than triple patterns with ?article title ?title the shared variables between p11 and p2 . At this point, the two variables and bounded subjects are more selective than NoSQL NoSQL map function utilizes the input mapping µ1 to substitute the bounded predicates or?article=Artikel_1 objects. This reduces the number of Server Server 2 shared variables in p2 , i.e. the join variables. Now, the subintermediate results in?article=Artikel_2 a sequence of joins and thus reduce ?article=Artikel_3 Article1 titel „PigSPARQL“ sub NoSQL Storage System stituted triple pattern p is used to retrieve the compatible the number of requests against HBase. 2 Article1 author ?author … mappings with a dynamic GET request to HBase following NoSQL HDFS the triple pattern mapping outlined in Table 3. Since there Server Algorithm 1: MAPSIN join map(inKey, inValue) is no guarantee that the corresponding HBase entries reside input : inKey, inValue: input mapping Article1 author ?author on the same machine as the GET request, the results of the multiset of mappings (article1 author ?author) request have to be transferred over the network in general.3 Get bindings foroutput: 1 pn+1 ← Config.getNextPattern() However, in contrast to a reduce-side join approach where title ?title \JOIN ?article author ?author 2 µn ← inV alue.getInputMapping() a lot of data is?article transferred over the network, we only trans3 Ωn+1 ← ∅ fer the data that is really needed. Finally, the computed local mappings: ?article title ?title 4 if dom(µn ) ∩ dom(pn+11 ) SCAN 6= ∅forthen multiset of solution mappings is stored in HDFS. 5 // substitute shared vars in pn+1 6 psub 1 n+1 ← µn (pn+1 ) 1 SCAN for local mappings: ?article title ?title 2 7 results ← HBase.GET(psub n+1 ) NoSQL 2 8 else Node 2 map inputs Node 3 2 map inputs 9 results ← HBase.GET(p 2 Node 1 n+1 ) ?title=“PigSPARQL“ ?article=article1 NoSQL ?article=article1 ?title="PigSPARQL" 3 GET bindings: article1 author ?author 3 10 end Storage System 3 GET bindings: article1 author ?author 11 if results 6= ∅ then?article=article2 ?title=“RDFPath“ ?article=article2 ?title="RDFPath" Node 2 GET bindings: article2 author ?authorfor p 12 // merge µn with3 compatible mappings n+1 3 GET bindings: article2 author ?author 3 13 foreach mapping µ in results do 14 µn+1 ← µn ∪ µ 4 15 Ωn+1 ← Ωn+1 ∪ µn+1 map outputs 2 4 2 ?article=article1 ?title="PigSPARQL" ?author=Alex 16 end 2 4 ?article=article1 ?title="PigSPARQL" ?author=Martin HDFS map )inputs 4 17 emit(null,2 Ωn+1 Node 1 ?article=article2 ?title="RDFPath" ?author=Martin ?article=article1 ?title=“PigSPARQL“ 3 ?article=article2 ?title="RDFPath" ?author=Alex 18 end 3 GET bindings: article1 author ?author

1

Node 3

NoSQL Storage System

?article=article2 ?title=“RDFPath“

Figure 3: MAPSIN join base case for the first two triple patterns of Query 1 in Section 2.1

3 3 GET bindings: article2 author ?author Algorithm 1 outlines one iteration of the MAPSIN join (without triple pattern reordering). The input for the map function contains either a mapping for the first triple pat4 map outputs tern or a mapping for previously joined triple patterns. The 4 ?article=article1 ?title=“PigSPARQL“ ?author=Alex ?article=article1 ?title=“PigSPARQL“ ?author=Martin algorithm illustrates why we refer to our join technique as44 ?article=article2 a map-side index nested loop?title=“RDFPath“ join. First ?author=Martin of all, the join is ?article=article2 ?title=“RDFPath“ ?author=Alex completely computed in the map phase. Furthermore, the map 3function is invoked for each input mapping what corarticle1,„PigSPARQL“, Alex respondsà an outer loop. Martin Within the map function, we à to article1,„PigSPARQL“, retrieve the compatible mappings for the next triple pattern using index lookups in HBase.

Figure 3 is an example for the base case of our MAPSIN join that illustrates the join between the first two triple patterns of the SPARQL query in Section 2.1. While the mappings for the first triple pattern (?article, title, ?title) are retrieved locally using a distributed table scan (step 1+2), the compatible mappings for (?article, author, ?author) are requested within the map function dynamically (step 3) and the resulting set of mappings is stored in HDFS (step 4). 5

Node 2

HDFS

4.3

Multiway Join Optimization

ject position, then all mappings for p2 , p3 , ..., pn+1 that are compatible to the input mapping µ1 for p1 are stored in the same HBase table row of Ts po or To ps , respectively, making it possible to use a single instead of n subsequent GET requests. Hence, all compatible mappings can be retrieved at once thus saving n − 1 GET requests for each invocation of the map function. Algorithm 3 outlines this optimized case.

Instead of processing concatenated triple patterns successively as a sequence of two-way joins, some basic graph patterns allow to apply a multiway join approach to process joins between several concatenated triple patterns at once in a single map phase. This is typically the case for star pattern queries where triple patterns share the same join variable. The SPARQL query introduced in Section 2.1 is an example for such a query as all triple patterns share the same join variable ?article. This query can be processed by a three-way join in a single map-phase instead of two consecutive two-way joins. We extended our approach to support this multiway join optimization. Again, the first triple pattern p1 is processed using a distributed table scan as input for the map phase. But instead of using a sequence of n map phases to compute p1 1 p2 1 ... 1 pn+1 we use a single map phase thus saving n − 1 MapReduce iterations. Hence, the map function needs to retrieve all mappings for p2 , p3 , ..., pn+1 that are compatible to the input mapping µ1 for p1 . Therefore, the join variable ?vs in p2 , p3 , ..., pn+1 (e.g. ?article) is substituted with the corresponding variable binding µ(?vs ). The sub sub substituted triple patterns psub 2 , p3 , ..., pn+1 are then used to retrieve the compatible mappings using HBase GET requests. This general case of the MAPSIN multiway join is outlined in Algorithm 2.

Algorithm 3: MAPSIN optimized multiway join map(inKey, inValue) input : inKey, inValue: input mappings output: multiset of solution mappings 1 #p ← Config.getNumberOfMultiwayPatterns() 2 µn ← inV alue.getInputMapping() 3 Ωn+#p ← ∅ 4 // iterate over all subsequent multiway patterns 5 for i ← 1 to #p do 6 pn+i ← Config.getNextPattern() 7 // substitute shared vars in pn+i 8 psub n+i ← µn (pn+i ) 9 end sub sub 10 results ← HBase.GET(pn+1 , ..., pn+#p ) 11 if results 6= ∅ then 12 // merge µn with compatible 13 // mappings for pn+1 , ..., pn+#p 14 foreach mapping µ in results do 15 µn+#p ← µn ∪ µ 16 Ωn+#p ← Ωn+#p ∪ µn+#p 17 end 18 emit(null, Ωn+#p ) 19 end

Algorithm 2: MAPSIN multiway join map(inKey, inValue) input : inKey, inValue: input mapping output: multiset of mappings 1 #p ← Config.getNumberOfMultiwayPatterns() 2 µn ← inV alue.getInputMapping() 3 Ωn ← {µn } 4 // iterate over all subsequent multiway patterns 5 for i ← 1 to #p do 6 Ωn+i ← ∅ 7 pn+i ← Config.getNextPattern() 8 // substitute shared vars in pn+i 9 psub n+i ← µn (pn+i ) 10 results ← HBase.GET(psub n+i ) 11 if results 6= ∅ then 12 // merge previous mappings with 13 // compatible mappings for pn+i 14 foreach mapping µ in results do 15 foreach mapping µ0 in Ωn+i−1 do 16 Ωn+i ← Ωn+i ∪ (µ ∪ µ0 ) 17 end 18 end 19 else 20 // no compatible mappings for pn+i 21 // hence join result for µn is empty 22 return 23 end 24 end 25 emit(null, Ωn+#p )

4.4

One Pattern Queries

Queries with only one single triple pattern, that return only a small number of solution mappings, can be executed locally on one machine. In general however, the number of solution mappings can exceed the capabilities of a single machine for large datasets. Thus, concerning scalability it is advantageous to use a distributed execution with MapReduce even if we do not need to perform a join. The map phase is initialized with a distributed table scan for the single triple pattern p1 . Hence, map functions get only those mappings as input, which are locally available. The map function itself has only to emit the mappings to HDFS without any further requests to HBase. If the query result is small, non-distributed query execution can reduce query execution time significantly as MapReduce initialization takes up to 30 seconds in our cluster which clearly dominates the execution time.

5.

EVALUATION

The evaluation was performed on a cluster of 10 Dell PowerEdge R200 servers equipped with a Dual Core 3.16 GHz CPU, 8 GB RAM, 3 TB disk space and connected via gigabit network. The software installation includes Hadoop 0.20.2, HBase 0.90.4 and Java 1.6.0 update 26. Most of the queries are taken from the well-known Lehigh University Benchmark (LUBM) [15] as this benchmark is a well-suited choice when considering join processing in a Semantic Web scenario since the queries of the benchmark can

In many situations, it is possible to reduce the amount of requests against HBase by leveraging the RDF schema design for HBase outlined in Section 3. If the join variable for all triple patterns is always on subject or always on ob6

3000

Q7 00 Q4 000 500 500 1000 000 1500 500 2000 000 2500

3000

Q8 00 Q5 000 500 500 1000 000 1500 500 2000 000 2500

3000

Q11 00 Q6 000 500 500 1000 000 1500 500 2000 000 2500

59 72

2870 640 3431 800

10853 12166

time in seconds

Join Optimization) Furthermore, the performance gain(Multiway increases with the size 2 PigSPARQL PigSPARQL MAPSIN of the10000 dataset for bothMAPSIN LUBM and SP Bench.

6969 7984

424

104

PigSPARQL PigSPARQL174 325 532 482 1013

HBase HBase

29 4447 7262

1000

LUBM Q1. Return all graduate students that attend a 100 given 10course from a given university department SELECT 1?X WHERE {

1000

1500

2000

2500

3000

time in seconds time in seconds

time in seconds

?X rdf:type ub:GraduateStudent. 3000 ?X ub:takesCourse 2000 1000 } 0 1000 1000

PigSPARQL 1200

1400

1600

100

PigSPARQL 1000 10 1001 1000

1500 2000 2500 3000 LUBM (# universities)

10

PigSPARQL

1 1000

MAPSIN

1800 2000 10002200 LUBM (# universities) 750

2400

2600

2800

3000

1000

500 MAPSIN 1000 250 750 0 500 1000 1500 2000 2500 LUBM (# universities) 250

100 10 3000

1

MAPSIN

0 1000 1000

1500for 2000 2500 3000 1000 Performance 1500 2000 2500 3000 Figure 4: comparison LUBM Q1 LUBM (# universities)

LUBM (# universities)

750

100

500 LUBM queries Q4 (5 triple patterns), PigSPARQL MAPSIN Q7 (4 triple pat10 500 1000 terns), Q8 (5 triple patterns) and250SP2 Bench queries Q1 (3 400 triple 100 patterns), Q2 (9 triple patterns) 1 0 demonstrate the more 300 1000 1500 2000 2500 30005). 1000with 1500 a2000 2500 3000 general case sequence of cascaded joins (cf. Figure LUBM (# universities) LUBM (# universities) 200 10 In these cases, MAPSIN joins perform even up to 28 times 100 faster than PigSPARQL for LUBM(Multiway queries and up to 12 Join Optimization) 1 0 2 times faster SP2000Bench queries. 1000PigSPARQL PigSPARQL MAPSIN 1500 2000 2500 MAPSIN 3000 1000for 1500 2500 3000 LUBM (# universities)


10000

1000 Q8. Return the email addresses of all students LUBM PigSPARQL MAPSIN 100 that10000 are members of any department of a given university 3000 10

SELECT ?X, ?Y, ?Z 1000 1 WHERE { 1000 100

2500

1

Q6 10000 1000 100

1

Q1

600

1 ub:memberOf 1000 1000 1500?Y. 2000

1000 1200 1400 ?X ub:emailAddress ?Z }1600

0 1000

1800

2000

2200

2400

2600

2800

3000 3000

LUBM (# universities) PigSPARQL MAPSIN

PigSPARQL

1000 1000 100 100 10 10 1 1000

1500 2000 2500 3000 LUBM (# universities) 1000 1500 2000 2500 3000 LUBM (# universities)

3500 2800 1000 2100 750 1400 500 700

100 10

0 250 1000 0 1000

PigSPARQL

1000

MAPSIN

1 1500 2000 2500 LUBM (# universities) 1500 2000 2500 LUBM (# universities)

3000 3000

MAPSIN

Figure for LUBM Q8 1000 5: Performance comparison 1000 PigSPARQL

7

1

1200

10

1

10

3000

?X 2500 3000 1500 2000 2500 ?Y ub:subOrganizationOf . LUBM (# universities) LUBM LUBM (# universities) LUBM 0

2

100

10

1800 2000

1500

?Y rdf:type ub:Department. 2000

10000

Q1 1000

2400

?X rdf:type ub:Student. 3000

LUBM queries Q1, Q3, Q5, Q11, Q13 as well as SP Bench 955demonstrate the 80 base case with a single join bequery Q3a tween two triple patterns (cf. Figure 4). For the LUBM PigSPARQL HBase queries,164 MAPSIN joins performed 8 to 13 times faster com21 PigSPARQL HBase pared to319the reduce-side33 joins used by PigSPARQL. Even 78 14 for the less selective SP246Bench query, our MAPSIN join re469 149 48 quired only 620 5360the PigSPARQL execution time. 214 one third of 780 284 931 355



easily be formulated as SPARQL basic graph patterns. FurPigSPARQL multi‐joinMAPSIN MAPSIN multi‐join thermore, we also considered the SPARQL-specific SP2 Bench PigSPARQL PigSPARQL MJ HBase HBase MJ Performance Benchmark [29]. However, because most of the 642 436 63 23 2 SP Bench rather complex 1202 queries are 861 121 queries that37use all different kinds of SPARQL 1758 1297 1.0 operators, 167 we only evaluated 53 some of2368 the queries as1728 the focus of our efficient 182 work is the 62 computation of joins, 2173 i.e. SPARQL235 basic graph patterns. 2919 81 We decided 3496 to use both 2613 benchmarks 279as they have 92 different characteristics and LUBM queries are generally more LUBM 2 selective than SP Bench queries. Both benchmarks offer PigSPARQL HBase synthetic data generators that can be used to generate ar2 22 SP bitrary large datasets. Bench we generated datasets PigSPARQL174 MAPSIN For MAPSIN 329 33 from 200174 million up to 1000 23 million triples. For LUBM we 484 324datasets from4434 generated 1000 up to 3000 universities and 640 5351 used the475 WebPIE inference engine for Hadoop [32] to pre800 6653 634 compute955 the transitive closure. For loading data into HBase, 790region size to80 70 MB using snappy compression. we set the 512 944 times for both 84 tables T The loading s po and To ps as well as all datasets are listed separately in Table 4. PigSPARQL HBase We compared our MAPSIN join approach with a reducePigSPARQL78 MAPSIN 14 MAPSIN side join149 based query execution using PigSPARQL [28], a 48 174 21 214 6033 on top of Pig. Pig is an Apache SPARQL 1.0 engine built 324 6942 by Yahoo! Research that offers top-level284 project developed 480 355 a high-level the analysis of very large datasets 642 language for8449 424 10459 The crucial point for this choice with Hadoop 805 MapReduce. was the 961 sophisticated and 72 efficient reduce-side join implementation of Pig [12] that incorporates sampling and hash PigSPARQL HBase PigSPARQL multi‐joinMAPSIN MAPSIN multi‐join join techniques which makes it a challenging candidate for 532 47 PigSPARQL PigSPARQL MJ HBase HBase MJ comparison. Pig also supports map-side joins as outlined in 1013 62 642 However, these 436 joins cannot 63 be easily cascaded 23 Section1480 2.2. 68 1202 861 121 37 as the 1985 preconditions for93 the next iteration are not fulfilled 1758 1297 167 53 without a join postprocessing using shuffle and reduce phase. 2472 114 2368 1728 182 62 In order to avoid caching effects that could possibly in2928 123 2919 2173 235 81 fluence 3496 our experiments, HBase was 279 restarted before each 2613 92 query execution. For detailed comparison, all query execuPigSPARQL HBase in Table 5. We illustrate the perfortion times are listed 628 50 mance comparison of PigSPARQL and MAPSIN for some PigSPARQL HBase 1172 6422 selected queries that represent the different query types. 174 1731 7733 evaluations can be found in ApFor completeness, all query 329 pendix2318 A484 for LUBM and9344Appendix B for SP2 Bench.

3000

Q13 00 Q7 000 500 500 1000

time in seconds

time in seconds

805 961


Q6 00 Q3 000 500 500 1000 000 1500 500 2000 000 2500

750

100

time in seconds

3000

23

2 4: 500 34 SP Bench & LUBM loading times for tables Ts po and To ps (hh:mm:ss) 10 600M 200M 400M 800M 1000M 51 250 ∼ 200 million ∼ 400 million ∼ 1600 million ∼ 800 million ∼ 1000 million 53 0 70 2500 3000 1000 1500 2000 2500 3000 00:28:39 00:45:33 01:01:19 01:16:091000 1500 200001:33:47 LUBM (# universities) LUBM (# universities) 84 00:27:24 01:04:30 01:28:23 01:43:36 02:19:05 PigSPARQL MAPSIN total 00:56:03 01:50:03 02:29:42 02:59:45 03:52:52 1000 1000 LUBM 1000 1500 2000 2500 3000 PigSPARQL MAPSIN MAPSIN 750 100 # RDF triples ∼ 210 million ∼ 315 million ∼ 420 million ∼ 525 million ∼ 630 million 174 21 500 Ts po 324 00:28:50 00:42:10 00:52:03 00:56:00 01:05:25 33 10 250 42 To ps 480 00:48:57 01:14:59 01:21:53 01:38:52 01:34:22 1 0 49 total 642 01:17:47 01:57:09 02:13:56 02:34:52 02:39:47 1000 1500 2000 2500 3000 1000 1500 2000 2500 3000

time in seconds

Q5 00 Q1 000 500 500 1000 000 1500 500 2000 000 2500

174 Table 324 2 SP Bench 475 # RDF 634triples Ts po 790 To ps 944

MAPSIN 1000

1000

MAPSIN

time in seconds

Q4 00 000 500 000 500 000

MAPSIN


Q3 00 000 500 000 500 000

PigSPARQL

PigSPARQL


Q1 00 000 500 000 500 000

LUBM

MAPSIN

Q1 1000 100

750 500

1000 100

400 500

100 10 10 1 1000 1 1000

1500 2000 2500 3000 LUBM (# universities) 1500 2000 2500 3000 LUBM (# universities)

PigSPARQL

10

300 250 200 0 100 1000 1500 2000 2500 0 LUBM (# universities) 1000 1500 2000 2500 LUBM (# universities)

MAPSIN

1 3000 3000

Q6

10000

1000

Q11 00 000 500 000 500 000

Q13 00

time in seconds

time in seconds

400M

600M

time in seconds

0 2351500 2000 250081 3000 1000 279 LUBM (# universities)92

2173 3000 1500 2000 2500 LUBM (# universities) 2613

1000

LUBM Q4. ReturnPigSPARQL names, emailsMAPSIN and phone numbers of all department 1000 1000professors working for a given

HBase

1000 78

214 10 284 1 355 3000 424

1000

SELECT ?X

PigSPARQL 2000

2500

1600 47 1800 2000 2200 LUBM (# universities)

3000

2400

2600

2800

3000

1

1500

2000

2500

10000

3000

1000

1000


MJ

MJ

PigSPARQL MJ

MJ

MJ

1500 2000 2500 3000 LUBM (# universities) MJ MJ

MJ MAPSIN

10000 1000 100 10 1

500

200M

400M

600M 250 SP²Bench

1 1000

1500

2000

2500

3000

0 1000

2500

MJ

MJ

750

1

1

Q1

1000 1500 2 LU

Q8

LUBM

1000

100 10

0 1000

10M

1

700 1000 1500 2000 2500 3000

Q7

LUBM

1000 MJ 100

MJ

MJ

MJ

1400 1

1

10000

Q1 3000

1

2400 100

101000 1500 2000 2500 3000

Q6

8

2500

1

Q6

MJ: Multiway‐Join of PigSPARQL/MAPSIN MAPSIN LUBM (# universities) 10000 MJ MAPSINMJ MJ 1000

100

2400

2000

0 1000

1000

10

1500

10

1800 10 Overall, the MAPSIN join approach clearly outperforms 100 10 1200 Both approaches reveal the reduce-side joins of PigSPARQL. 1 10 1 a linear scaling behavior with the600 input1000 size but 1500 the slope2000 1 1000 1500 2000 2500 3000 0 of the MAPSIN join is much smaller. Especially for LUBM Q1 1000 1500LUBM 2000 2500 3000 Q4 1000 1500 2000 2500 3000 LUBM LUBM (# universities) LUBMjoins by an10000 LUBM (# universities) LUBM 10000 reduce-side queries, MAPSIN joins outperform 1000 order of magnitude as these queries are generally rather se- 1000 1000 PigSPARQL MAPSIN 100 lective. Moreover, the application of our multiway join opti10000 3500 100 mization results in a further significant improvement of the 100 2800 101000 10 10 total query execution times. 2100

1000

0 1000

1500 2000 2500 3000 PigSPARQL LUBM (# universities)

100

600

HBase

1

3000

1200

100

100

MAPSIN

1800

10

29

1000

Figure for LUBM Q6 10000 7: Performance comparison 3000 100

500

53 69 79

2500

Q1

3000

200

PigSPARQL

750

46 PigSPARQL

2000

1500 2000 2500 LUBM (# universities)

MAPSIN

10

time in seconds

114 123

1500

1

300

1000

MAPSIN

10 Especially interesting are queries250Q4 of LUBM and Q1, Q2 2 of SP Bench since these queries support the multiway join 0 PigSPARQL1 HBase 1000 as1500 2000 2500 3000 1500 2000 2500 3000 optimization outlined in50Section 4.3 all triple patterns 6281000 LUBM (# universities) LUBM (# universities) share the same join variable. This kind of optimization is 1172 64 also supported by PigSPARQL such MAPSIN that both approaches 1731 77 PigSPARQL 500 a single multiway join 1000 2318 can compute the query 93 results with 400 2870 6). The MAPSIN 108 (cf. Figure multiway join optimization 100 300 3431 121 improves the basic MAPSIN join execution time by a factor 200 of 1.4 10(SP2 Bench Q1) to 3.3 (LUBM Q4), independently 100 of the data size. For the LUBM queries, the MAPSIN 0 PigSPARQL1 HBase multiway 1000 join 1500 optimization performs 191500 to 28 times faster 1000 2000 2500 3000 2000 2500 3000 21 LUBM (# universities) than the164 reduce-side based multiway joinLUBM (# universities) implementation of 319 33 PigSPARQL. For the more complex SP2 Bench queries, the

PigSPARQL1 1741000

10

400

1000

1000 93 comparison Performance for LUBM Q4

100

100

500

100

3000

62

68 PigSPARQL

469 10000620 1000780 931

1000


1000

HBase 1400

1000

WHERE { ?X rdf:type ub:Student }

1000 1200

3000

LUBM Q6. Return all students 0 1

2000

PigSPARQL 0 1000 532 1013 1480 1000 1985 Figure 6: 2472 100 2928

2500

250

1000

14 48 60 69 1500 84 104

149 100

(Multiway Join Optimization) PigSPARQL MAPSIN

time in seconds

PigSPARQL 10000

2000

time in seconds

time in seconds

?X ub:telephone ?Y3 }

MAPSIN

1500

1000M M (Multiway Join Optimization) P M MAPSIN 177 PigSPARQL 2519 214 154 1476 174 834 6005 999 235 1050 274 151 414 167

2000 performance improvements degrade to a factor of approximately10008.5. 0 The remaining queries (LUBM Q6, Q14 and SP2 Bench 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 3000 Q10) consist of only one single triple pattern (cf. Figure 7). LUBM (# universities) Consequently they do not contain a join processing step PigSPARQL MAPSIN and illustrate primarily the advantages of the distributed 1000 1000 HBase table scan compared to the HDFS storage access of 750 100 PigSPARQL. Improvements are still present but less signif500 icant, resulting in an up to 5 times faster query execution. 10

PigSPARQL HBase?Y3 SELECT ?X, ?Y1, ?Y2, 750 100 22 WHERE {174 500 329 33 ?X rdf:type ub:Professor. 10 250 484 44 ?X ub:worksFor . 1 640 0 53 ?X ub:name ?Y1. 1000 1500 2000 2500 3000 1000 1500 2000 2500 3000 800 66 LUBM (# universities) LUBM (# universities) ?X ub:emailAddress ?Y2. 955 80 PigSPARQL

P MAPSIN 2018 1187 4745 845 340

3000

time in seconds

2919 1000 3496


800M

M PigSPARQL 153 10000 118 1000 100 671 10 178 1 111

time in seconds

P 1527 896 3514 641 254

time in seconds

Q8 00 000 500 000 500 000

200M

time in seconds

Q7 00 000 500 000 500 000

1

time in seconds

join Q6 00 23 000 37 500 53 000 62 500 81 000 92

72

P M MAPSIN P M PigSPARQL MAPSIN multi‐join Q11000 PigSPARQL multi‐joinMAPSIN 545 58 1026 118 1000 PigSPARQL PigSPARQL MJ HBase HBase MJ Q1 multijoin 310 42 600 87 750 436 63 23 100 642 Q2 multijoin 1168 241 2341 444 500 121 1202 861 37 Q3a 101758 227 70 435 1297 53 139 250 167 Q10 2368 99 40 182 174 1728 62 84

time in seconds

Q5 00 000 500 000 500 000

MAPSIN


961 SP2 Bench

time in seconds

Q4 00 000 500 000 500 000

Q5 Q6 PigSPARQL Q7 174 Q8 Q11 324 480 Q13 642 Q14 805

time in seconds

Q3 00 000 500 000 500 000

PigSPARQL

Table 5: Query execution times for PigSPARQL1000 (P) and MAPSIN (M) 1000 in seconds 1500 2000 2500 750 3000 P M P 100 M P M P M 500 475 51 634 10 53 790 70 944 84 250 480 42 642 49 805 59 961 72 1 0 1758 167 2368 182 2919 235 3496 279 1000 1500 2000 2500 3000 1000 1500 2000 2500 3000 LUBM (# universities) 1297 53 1728 62LUBM (# universities) 2173 81 2613 92 329 33 484 44 640 53 800 66 MAPSIN 955 80 PigSPARQL 1000 149 48 214 60 284 1000 69 355 84 424 104 MAPSIN 1013MAPSIN 62 750 1480 68 1985 100 93 2472 114 2928 123 21 1172 64 1731 77 2318 33 2870 108 3431 121 500 33 319 33 469 46 620 10 53 780 69 931 79 250 42 325 44 482 72 645 84 800 108 957 128 1 0 49 3000 1000 2500 3000 149 43 214 70 288 79 1500 2000 364 89 1000 15004342000 2500 107 59

PigSPARQL MAPSIN MAPSIN LUBM 1000 174 23 P M 324 34 Q1 324 34 475 51 Q3 324 33 634 53 Q4 1202 121 790 70 Q4 multijoin 861 37 944 84

time in seconds

Q1 00 000 500 000 500 000

LUBM

800M

200M 400M 6

1000M

Q3a 1500

2000

2500

3000

SP²

6.

RELATED WORK

HadoopRDF [18] and SHARD [27] are MapReduce based RDF engines that store data directly in HDFS and do not require any changes to the Hadoop framework. These systems are able to rebalance automatically if cluster size changes but join processing is also done in the reduce phase. Our MAPSIN join does not use any shuffle or reduce phase at all even in consecutive iterations. As we only request the data from HBase that is really needed, we reduce data transfer cost to a minimum.

Single machine RDF engines like Sesame [7] and Jena [35] are widely-used since they are user-friendly and perform well for small and medium sized RDF datasets. RDF-3X [23] is considered one of the fastest single machine RDF engines in terms of query performance that vastly outperforms previous single machine systems but performance degrades for queries with unbound objects and low selectivity factor [18]. Furthermore, as the amount of RDF data continues to grow, it will become more and more difficult to store entire datasets on a single machine due to the limited scaling capabilities [17]. OWLIM [20] is a fast and scalable RDF database management system that offers both a single and a cluster edition but it requires high end and thus costly hardware [18]. There is also a large body of work dealing with join processing in a MapReduce environment considering various aspects and application fields [4, 6, 19, 21, 24, 26, 36]. In Section 2.2 we briefly outlined the advantages and drawbacks of the general-purpose reduce-side and map-side (merge) join approaches in MapReduce. Though map-side joins are generally more efficient, they are hard to cascade due to the strict preconditions. In our approach, we leverage HBase to overcome this shortcoming without the use of auxiliary shuffle and reduce phases, making MAPSIN joins easily cascadable. In addition to these general-purpose approaches there are several proposals focusing on certain join types or optimizations of existing join techniques for particular application fields. In [24] the authors discussed how to process arbitrary joins (theta joins) using MapReduce and provide algorithms to minimize total job completion time, whereas [4] focuses on optimizing multiway joins. However, in contrast to our MAPSIN join, both approaches process the join in the reduce phase including a costly data shuffle phase. Map-Reduce-Merge [36] describes a modified MapReduce workflow by adding a merge phase after the reduce phase, whereas Map-Join-Reduce [19] proposes a join phase in between the map and reduce phase. Both techniques attempt to improve the support for joins in MapReduce but require profound modifications to the MapReduce framework. Our approach requires no changes to Hadoop and HBase at all, thus making it possible to run on any Hadoop cluster out of the box, e.g. Amazon’s Elastic Compute Cloud (EC2)4 . In [10] the authors propose a new index and join technique called Trojan Index and Trojan Join that do not require modifications to the MapReduce framework. These techniques reduce the amount of data that is shuffled during join execution at the cost of an additional co-partitioning and indexing phase at load time. However, in contrast to our approach, shuffle and reduce phases cannot be completely avoided. HadoopDB [3] is a hybrid of MapReduce and DBMS where MapReduce is the communication layer above multiple single node DBMS. The authors in [17] adopt this hybrid approach for the semantic web using RDF-3X. However, the initial graph partitioning is done on a single machine and has to be repeated if the dataset is updated or the number of machines in the cluster change. As we use HBase as underlying storage layer, additional machines can be plugged in seamlessly and updates are possible without having to reload the entire dataset. 4

7.

CONCLUSION

In this paper we introduced the Map-Side Index Nested Loop join (MAPSIN join) which combines the advantages of NoSQL storage systems like HBase with the well-known and approved distributed processing facilities of MapReduce. In general, map-side joins are more efficient than reduce-side joins in MapReduce as there is no expensive data shuffle phase involved. However, current map-side join approaches have some strict preconditions what makes them hard to apply in general, especially in a sequence of joins. The combination of HBase and MapReduce allows us to cascade a sequence of MAPSIN joins without having to reorder or sort the output for the next iteration. Furthermore, with the multiway join optimization we can reduce the number of MapReduce iterations and HBase requests. Using an index to selectively request only those data that is really needed also saves network bandwidth, making parallel query execution more efficient. The evaluation with the LUBM and SP2 Bench benchmarks demonstrate the advantages of our approach compared to the commonly used reduce-side join approach in MapReduce. For most of the benchmark queries, MAPSIN join based SPARQL query execution outperforms reduce-side join based execution of PigSPARQL (using Apache Pig) by an order of magnitude while scaling very smoothly with the input size. Lastly, our approach does not require any changes of the Hadoop platform. Consequently, SPARQL queries can be run directly on an instance of Amazon’s Elastic Compute Cloud (EC2) whenever the own capacities are exceeded. In our future work, we will investigate alternatives and improvements of the RDF storage schema for HBase and incorporate MAPSIN joins into PigSPARQL in a hybrid fashion such that the actual join method is dynamically selected based on pattern selectivity and statistics gathered at data loading time.

8.

REFERENCES

[1] RDF Primer. W3C Recommendation. http://www.w3.org/TR/rdf-primer/, 2004. [2] SPARQL Query Language for RDF. W3C Recom. http://www.w3.org/TR/rdf-sparql-query/, 2008. [3] A. Abouzeid, K. Bajda-Pawlikowski, D. J. Abadi, A. Rasin, and A. Silberschatz. HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads. PVLDB, 2(1):922–933, 2009. [4] F. N. Afrati and J. D. Ullman. Optimizing Multiway Joins in a Map-Reduce Environment. IEEE Trans. Knowl. Data Eng., 23(9):1282–1298, 2011. [5] T. Berners-Lee, J. Hendler, and O. Lassila. The Semantic Web. Scientific American, 284(5):34–43, 2001.

http://aws.amazon.com/ec2/ 9

[6] S. Blanas, J. M. Patel, V. Ercegovac, J. Rao, E. J. Shekita, and Y. Tian. A Comparison of Join Algorithms for Log Processing in MapReduce. In SIGMOD Conference, pages 975–986, 2010. [7] J. Broekstra, A. Kampman, and F. van Harmelen. Sesame: A Generic Architecture for Storing and Querying RDF and RDF Schema. In International Semantic Web Conference (ISWC), pages 54–68, 2002. [8] F. Chang, J. Dean, S. Ghemawat, W. Hsieh, D. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. Gruber. Bigtable: A Distributed Storage System for Structured Data. ACM Transactions on Computer Systems (TOCS), 26(2):4, 2008. [9] J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. Communications of the ACM, 51(1):107–113, 2008. [10] J. Dittrich, J.-A. Quian´e-Ruiz, A. Jindal, Y. Kargin, V. Setty, and J. Schad. Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing). PVLDB, 3(1):518–529, 2010. [11] C. Franke, S. Morin, A. Chebotko, J. Abraham, and P. Brazier. Distributed Semantic Web Data Management in HBase and MySQL Cluster. In IEEE International Conference on Cloud Computing (CLOUD), pages 105 –112, 2011. [12] A. F. Gates, O. Natkovich, S. Chopra, P. Kamath, S. M. Narayanamurthy, C. Olston, B. Reed, S. Srinivasan, and U. Srivastava. Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience. PVLDB, 2(2):1414–1425, 2009. [13] L. George. HBase - The Definitive Guide: Random Access to Your Planet-Size Data. O’Reilly, 2011. [14] S. Ghemawat, H. Gobioff, and S. Leung. The Google File System. In ACM SIGOPS Operating Systems Review, volume 37, pages 29–43. ACM, 2003. [15] Y. Guo, Z. Pan, and J. Heflin. LUBM: A Benchmark for OWL Knowledge Base Systems. Web Semantics: Science, Services and Agents on the World Wide Web, 3(2):158–182, 2005. [16] E. Hewitt. Cassandra - The Definitive Guide: Distributed Data at Web Scale. Definitive Guide Series. O’Reilly, 2010. [17] J. Huang, D. J. Abadi, and K. Ren. Scalable SPARQL Querying of Large RDF Graphs. PVLDB, 4(11):1123–1134, 2011. [18] M. F. Husain, J. P. McGlothlin, M. M. Masud, L. R. Khan, and B. M. Thuraisingham. Heuristics-Based Query Processing for Large RDF Graphs Using Cloud Computing. IEEE Trans. Knowl. Data Eng., 23(9):1312–1327, 2011. [19] D. Jiang, A. K. H. Tung, and G. Chen. Map-Join-Reduce: Toward Scalable and Efficient Data Analysis on Large Clusters. IEEE Trans. Knowl. Data Eng., 23(9):1299–1311, 2011. [20] A. Kiryakov, D. Ognyanov, and D. Manov. OWLIM A Pragmatic Semantic Repository for OWL. In WISE Workshops, pages 182–192, 2005. [21] K.-H. Lee, Y.-J. Lee, H. Choi, Y. D. Chung, and B. Moon. Parallel Data Processing with MapReduce: A Survey. SIGMOD Record, 40(4):11–20, 2011. [22] J. Lin and C. Dyer. Data-Intensive Text Processing

[23] [24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34] [35]

[36]

10

with MapReduce. Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers, 2010. T. Neumann and G. Weikum. RDF-3X: a RISC-style engine for RDF. PVLDB, 1(1):647–659, 2008. A. Okcan and M. Riedewald. Processing Theta-Joins using MapReduce. In SIGMOD Conference, pages 949–960, 2011. J. P´erez, M. Arenas, and C. Gutierrez. Semantics and Complexity of SPARQL. ACM Transactions on Database Systems (TODS), 34(3):16, 2009. M. Przyjaciel-Zablocki, A. Sch¨ atzle, T. Hornung, and G. Lausen. RDFPath: Path Query Processing on Large RDF Graphs with MapReduce. In ESWC Workshops, pages 50–64, 2011. K. Rohloff and R. E. Schantz. High-Performance, Massively Scalable Distributed Systems using the MapReduce Software Framework: The SHARD Triple-Store. In Programming Support Innovations for Emerging Distributed Applications (PSI EtA), pages 4:1–4:5, 2010. A. Sch¨ atzle, M. Przyjaciel-Zablocki, and G. Lausen. PigSPARQL: Mapping SPARQL to Pig Latin. In Proceedings of the International Workshop on Semantic Web Information Management (SWIM), pages 4:1–4:8, 2011. M. Schmidt, T. Hornung, G. Lausen, and C. Pinkel. SP2Bench: A SPARQL Performance Benchmark. In IEEE 25th International Conference on Data Engineering (ICDE), pages 222–233, 2009. M. Stocker, A. Seaborne, A. Bernstein, C. Kiefer, and D. Reynolds. SPARQL Basic Graph Pattern Optimization Using Selectivity Estimation. In Proceedings of the 17th international conference on World Wide Web (WWW), pages 595–604, 2008. J. Sun and Q. Jin. Scalable RDF Store Based on HBase and MapReduce. In 3rd International Conference on Advanced Computer Theory and Engineering (ICACTE), volume 1, pages 633–636, 2010. J. Urbani, S. Kotoulas, J. Maassen, F. van Harmelen, and H. Bal. OWL Reasoning with WebPIE: Calculating the Closure of 100 Billion Triples. In ESWC, pages 213–227, 2010. C. Weiss, P. Karras, and A. Bernstein. Hexastore: Sextuple Indexing for Semantic Web Data Management. PVLDB, 1(1):1008–1019, 2008. T. White. Hadoop - The Definitive Guide: Storage and Analysis at Internet Scale (2. ed.). O’Reilly, 2011. K. Wilkinson, C. Sayers, H. A. Kuno, and D. Reynolds. Efficient RDF Storage and Retrieval in Jena2. In SWDB, pages 131–150, 2003. H.-C. Yang, A. Dasdan, R.-L. Hsiao, and D. S. P. Jr. Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters. In SIGMOD Conference, pages 1029–1040, 2007.

00 000 Q13 500 00 000 000 500 500 000

ti time in seconds time in seconds time in seconds

(Multiway Join Optimization)

500 MAPSIN PigSPARQL MAPSIN 1000 250 750 0 500 1000 1500 2000 2500 3000 LUBM (# universities) 250

1

time in seconds time in seconds time in seconds time in seconds time in seconds

PigSPARQL

1000

}

0 10 1000

1200

1400

PigSPARQL

time in seconds time in seconds time in seconds time in seconds

1 10000

1000

1000 1000 100 10010 10001 10

1600

MAPSIN

1500 2000 PigSPARQL 2500 3000 LUBM (# universities)

PigSPARQL 1000

SELECT 100?X

1500

PigSPARQL

750

1 1000

Q1

3000

1000 100 100 10

MAPSIN 2500

3000

101

1500 2000 2500 1500 2000 2500 3000 LUBM (# universities) 1000 1 0 0 MAPSIN 1000 1500 2000 2500 1000 1500 2000 PigSPARQL 2500 3000 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 LUBM (# universities) LUBM (# universities) 500 1000 LUBM (# universities)

time in seconds time in seconds time in seconds

10

MAPSIN

0 MAPSIN 1000 1500 2000 2500 LUBM (# universities) 1000

2000400 250 300 0 2001000 100

?X rdf:type LUBM (# universities) ub:Student

750

100

500 500

WHERE3000 {1 2000 10 1000 }

1000

500 1800 2000 2200 2400 2600 2800 3000 250(Multiway Join Optimization) LUBM (# universities)

Q6. Return all students

500

MAPSIN

1000 100 rdf:type ub:Person. 0 750 10 100 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 3000 ub:memberOf 1 LUBM (# universities) 500 1000 1500 2000 2500 3000 10 250 PigSPARQL MAPSIN 3000 1000 1000 1 0 2000 1000 1500 2000 2500 3000 1000 1500 2000 2500 3000 750 1000 LUBM (# universities) LUBM (# universities) 100

?X ?X

time in seconds

Q1 1 1000

3000

10000 100

3000 3000

1000 1000 10 100 100 1 10 1 Q610 1

400

PigSPARQL PigSPARQL

100 10000 1000 10 1000 100 100 1 1010 11

1000


PigSPARQL 1000 1500 1500 2000 2000 2500 2500 3000 3000 1000 LUBM (# universities) LUBM LUBM (# universities)

MAPSIN 300 MAPSIN

3000 1000 200 2400 750 100 1800 5000 12001000 250 600


PigSPARQL

SELECT 100?X ?Y 10000 1000

WHERE1000 { 10 100 ?X rdf:type ub:Student. 1 100

PigSPARQL

?Y rdf:type 1000 ub:Course. 1500 2000 2500 10001010

3000

LUBM (# universities) LUBM ?X ub:takesCourse ?Y.


10000

3000

Q1 1

1000

00 MAPSIN 1000 1500 2000 2000 2500 2500 3000 3000 1000 1500 LUBM (# universities) LUBM 3000 LUBM (# universities)

Q1 100

1000

10 100

1800 MAPSIN 3500 500 1200 2800 400 600

MJ

MJ

1000 100 10000

1

1001 10 10

1000

1000 100 11 1000 100 10 100

10 1

1000 100

10 2000 2500 LUBM 1 500 0 10000 10000 1200 1000 1000 1500 2000 2500 3000 1500 2000 2500 3000 250 MJ: Multiway‐Join of PigSPARQL/MAPSINQ1 PigSPARQL MAPSIN LUBM (# universities) LUBM (# universities) 600 1000 1000 100000 0 MJ MJ PigSPARQL MAPSIN 1000 1500 1500 2000 2000 2500 2500 3000 3000 1000 1500 1500 2000 2500 2500 3000 MJ 3000 100 100 1000 1000 MJ 2000 1000 LUBM (# universities) LUBM (# universities) 1000

10 1000 1500 2000 2500 3000 1000 100

Q1

10

MAPSIN 2100 MAPSIN 1000 3000 1 1400 2400 1000 750 700

1 MJ

10000

2800

PigSPARQL PigSPARQL

1

Q6

MJ

MJ

10 1000

1

10 Q1

2100 300 0 MJ: Multiway‐Join of PigSPARQL/MAPSIN MAPSIN 1400 1500 2000 2500 3000 200 1000 10000 LUBM (# universities) LUBM 700 100

MJ 1000 11 00 100 PigSPARQL MAPSIN 1000 1500 1500 2000 2000 2500 2500 3000 3000 1000 1500 2000 2000 2500 2500 3000 3000 1000 1500 1000 ub:teacherOf ?Y LUBM (# universities) LUBM (# universities) 10000 3500 100 LUBM (# universities) LUBM (# universities) MJ

}

1

Q6

Q7.10000 Return all students that attend any course of a given 1000 associate professor from a given2400 university department PigSPARQL MAPSIN




1000 100 1 100 1000 1500 2000 2500 3000 LUBM (# universities) 10 10


000 Q14 Q11 500 500 00 000 1000 000 500 1500 500 000 2000 000 2500 500 3000 000 Q14

PigSPARQL

LUBM

LUBM (# universities) LUBM

LUBM (# universities) LUBM

PigSPARQL PigSPARQL

1000 10000 1000 1500 2000 2500 3000 10 Q61000 LUBM 1100

1500

Q41800

10 100 750 1

500 1000 10 3500 250 750 Q72800 1

MAPSIN MAPSIN

MJ

10

MJ

1000 1500 2000 2500 3000 LUBM

MJ

1 1000 1500

Q8

10000 10000 2100 1000 1500 2000 2500 3000 1000 1500 2000 0 500 1001 MJ LUBM 2000 2500 3000 Q41400 10 1000 1500LUBM MJ1000 1500 MJ2000 2500 3000 1000 1000 MJ 250 LUBM (# universities) LUBM (# universities) 10000 10000 1000 10 MJ 700 MJ MJ MJ MJ 0 100 100 11 MJ 1000 1000 0 MAPSIN 1000 1500 1500 2000 2000 PigSPARQL 2500 3000 3000 1000 1500 1500 2000 2000 2500 2500 3000 3000 100 1000 2500 1000 LUBM (# universities) LUBM (# universities) 1000 1000 10 10 LUBM (# universities) LUBM (# universities) 100 100 MJ: Multiway‐Join of PigSPARQL/MAPSIN PigSPARQL MAPSIN 750 10 1 1 PigSPARQL 10000 MAPSIN 1000100 10 10 PigSPARQL MAPSIN 200M 200M 400M 600M 800M 1000M 500 1000 MJ 1000 1000 MJ MJ Q11 10 Q3a 1 SP²Bench 1000 1400 100 250 750 1000 1000 1500 2000 2500 3000 1001000 1500 2000 2500 3000 100 300 MJ Q6 1 Q8 MJ Q7100 LUBM LUBM MJ 5000 2001000 10 10 1000 1500 2000 2500 3000 1500 2000 2500 3000 10000 10000 10000 10 10 LUBM (# universities) LUBM (# universities) MJ 250 100 MJ MJ 1000 1000 MJ 1000 1 11 100 MJ MJ MJ PigSPARQL MJMAPSIN 1000 1500 2000 2000 2500 3000 3000 10001500 15002000 20002500 25003000 3000 1000 1500 2000 MJ 1000 1500 2000 2500 3000 1000 1500 2500 100 1000 100 MJ LUBM (# universities) LUBM (# universities) 500 Q1 1000 Q4 100 LUBM (# universities) LUBM (# universities) LUBM LUBM

2500

Q1

11


00 000 Q13 Q8 join 500 500 00 000 1000 000 23 500 1500 500 37 000 2000 000 53 2500 500 62 3000 000 81 Q13 92 00

PigSPARQL MAPSIN

WHERE { 1000 1000

750 14 250 48 WHERE {149 500 PigSPARQL HBase PigSPARQL HBase 101214 60 0 ?X rdf:type ub:Publication. 250 532 47 174 22 1000 1500 2000 2500 3000 2841000 1500 2000 2500 69 3000 ?X ub:publicationAuthor LUBM (# universities) LUBM (# universities) 1013 62 1329 0 33 3551000 1500 2000 2500 84 3000 1000 1500 2000 2500 3000 1480 68 484 44 424 104 LUBM (# universities) LUBM (# universities) } (Multiway Join Optimization) 1985 93 640 53 PigSPARQL MAPSIN PigSPARQL MAPSIN PigSPARQL MAPSIN 2472 114 800 66 1000 1000 10000 2928 123 80 PigSPARQL955 HBase

47 500 1013 62 10 PigSPARQL MAPSIN 10 PigSPARQL HBase PigSPARQL HBase 250 68 1 1000 10001480 628 50 78 14 1000 1500 2000 2500 3000 1985 93 1 7500 1172 64 3000 149 48 1002472 1000 1500 2000 2500 3000 1000 1500 2000 2500 114 3000 LUBM (# universities) LUBM (# universities) 1731 77 500 214 60 2928 123 2000 102318 93 284 69 250 1000 2870 108 (Multiway Join Optimization) 355 84 1 0 0 Q4. Return with name, email address and PigSPARQLall professors MAPSIN PigSPARQL MAPSIN 3431 1213000 424 104 1000 1000 1500 2000 2500 1200 1400 1600 1800 2000 2200 1500 2400 2000 2600 2500 2800 3000 3000 PigSPARQL HBase 10000 telephone number workingLUBM (# universities) for a given LUBM (# universities) university departLUBM (# universities) 628 50 1000 ment PigSPARQL MAPSIN 1172 64 100 PigSPARQL MAPSIN PigSPARQL HBase 1000 1000 ?X ?Y1 ?Y2 SELECT ?Y3 PigSPARQL HBase 1000 1000 1731 77 10 164 21 WHERE { 532 47 750 93 1 750 1002318 319 33 100 1013 62 ?X rdf:type ub:Professor. 1000 1500 2000 2500 3000 2870 108 500 500 469 46 1480 68 ?X ub:worksFor . 10 3431 121 3000 250 10620 53 1985 93 250 ?X ub:name ?Y1. 2000 69 1 780 0 2472 114 ?X ub:emailAddress ?Y2. 1 0 1000 1000 1500 2000 2500 3000 1000 1500 2000 2500 3000 9311000 1500 2000 123 79 3000 2928 1000 1500 2000 2500 3000 2500 PigSPARQL HBase LUBM (# universities) LUBM (# universities) ?X ub:telephone ?Y3 0 LUBM (# universities) LUBM (# universities) 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 3000 164 21 } 319 33 LUBM (# universities) (Multiway Join Optimization) PigSPARQL MAPSIN PigSPARQL HBase PigSPARQL HBase MAPSIN PigSPARQL MAPSIN 500 PigSPARQL 1000 469 46 PigSPARQL MAPSIN 174 29 10000 628 50 400 53 1000 1000620 325 44 1000 100 1172 64 300 780 69 750 482 72 100 1731 77 100 931 79 200 10 500 84 10645 2318 93 100 101800 108 2870 108 2500 1 1000 1500 2000 2500 3000 9571000 1500 2000 121 128 3431 1000 1500 2000 2500 3000 PigSPARQL HBase 2500 3000 1 0 LUBM (# universities) LUBM (# universities) 3000 1000 1500 1000 1500 2000 2500 3000 2000 2500 174 29 3000 LUBM (# universities) LUBM (# universities) 2000 325 44 PigSPARQL MAPSIN PigSPARQL HBase 1000 PigSPARQL HBase 10000 3000 482 72 PigSPARQL MAPSIN 16 0 78 164 21 2400 645 84 500 1000 1000 1000 1200 1400 1600 149 43 1800 2000 2200 2400 2600 2800 3000 319 33 1800 800 108 LUBM (# universities) 400 100 214 70 469 46 100 1200 957 128 300 288 79 620 53 10 PigSPARQL 600 MAPSIN 200 1000 100010 364 89 780 69 1 1000 434 1000 1500 2000 107 1500 2000 2500 3000 931 79 3000 750 0 1000 PigSPARQL HBase 2500 100 1 LUBM (# universities) LUBM LUBM (# universities) LUBM 1000 1500 2000 2500 3000 781000 1500 2000 2500 16 3000 500 LUBM (# universities) LUBM (# universities) 10149 43 PigSPARQL MAPSIN 250 SP²Bench PigSPARQL HBase 214 70 10000 3500 MAPSIN PigSPARQL 1174 0 29 10000 3000 288 79 2800 1000 1500 2000 2500 3000 1000 1000 1500 2000 2500 3000 325 44 LUBM (# universities) LUBM (# universities) 2400 364 89 2100 1000 100 482 72 1800 434 107

3000

750

100

10000 SELECT ?X 2000

10

1000 100532 100

LUBM (# universities) 0 MAPSIN 1000 1500 2000 2500 LUBM (# universities) 1000

Q5. 1Return1000 all persons of a given 0 1500 that are 2000members 2500 3000 uni(Multiway Join Optimization) 1000 1500 2000 2500 3000 1000 1500 2000 2500 3000 versity department PigSPARQL MAPSIN PigSPARQL MAPSIN 3000 LUBM (# universities) LUBM (# universities)

1000 10 onds

00 23 000 37 Q11 Q7 500 53 500 00 000 62 1000 000 500 81 1500 500 000 92 2000 000 2500 500 3000 000 Q11

LUBM

SELECT 78 100 ?X


1000 10 10000

time in seconds

00 000 Q7 Q5 500 500 00 000 1000 000 500 1500 500 000 2000 000 ‐join 2500 500 3000 000 Q7 23 00 37 000 53 Q8 Q6 500 62 500 00 000 81 1000 000 500 92 1500 500 000 2000 000 2500 500 ‐join 3000 000 Q8

1000

time in secondstime in seconds time in seconds

00 000 Q6 Q4 500 500 00 000 1000 000 500 1500 500 000 2000 000 2500 500 3000 000 Q6

A.

1 1000

APPENDIX

time in seconds time in seconds time in seconds time in seconds time in seconds time in seconds time in seconds

00 000 Q5 Q3 500 500 00 000 1000 000 500 1500 500 000 2000 000 2500 500 3000 000 Q5


944 84 642 49 805 59 961 72 PigSPARQL LUBM MAPSIN MAPSIN 174 21 PigSPARQL multi‐joinMAPSIN MAPSIN multi‐join 324 33 PigSPARQL PigSPARQL MJ HBase HBase MJ PigSPARQL MAPSIN 42 MAPSIN 480 642 436 63 23 174 23 642 49 1202 861 121 37 324 34 805 59 1758 1297 167 53 475 51 961 72 2368 1728 182 62 634 53 Q1. 2919 Return all graduate students 235 that attend a81given 2173 790PigSPARQL multi‐joinMAPSIN 70 MAPSIN multi‐join course3496 from a given university department 2613 279 92 PigSPARQL944 PigSPARQL MJ84 HBase HBase MJ SELECT ?X 642 436 63 23 WHERE { 1202 861 121 37 PigSPARQL HBase ?X rdf:type ub:GraduateStudent. PigSPARQL MAPSIN 1758 1297 MAPSIN 167 53 174 22 ?X ub:takesCourse 174 21 2368 1728 182 62 329 33 324 33 2919 2173 235 81 484 44 480 42 } 3496 2613 279 92 640 53 642 49 PigSPARQL MAPSIN 1000 1000805 800 66 59 955 80 961 72 750 PigSPARQL HBase 100 174 22 500 MAPSIN multi‐join 10329PigSPARQL multi‐joinMAPSIN 33 250 PigSPARQL HBase PigSPARQL HBase HBase MJ 484 PigSPARQL MJ44 1642 0 63 78 14 436 23 6401000 1500 2000 2500 53 3000 1000 1500 2000 2500 3000 149 48 1202 861 121 LUBM (# universities) 37 LUBM (# universities) 800 66 214 60 1758 1297 167 53 955 80 PigSPARQL MAPSIN 284 69 2368 1728 182 62 1000 1000 355 84 2919 2173 235 81 PigSPARQL MAPSIN Q3. Return all publications of a given assistant professor 750 104 1000 100 3496 92 from1000 a424 given university PigSPARQL HBase 2613department 279

nds seconds

000 2000 2500 3000 Q3 00 000 Q4 Q1 500 500 00 000 1000 000 500 1500 500 000 2000 000 2500 500 3000 000 Q4

100

PigSPARQL

10000400 10

MAPSIN Join Optimization) 300 (Multiway

10000 10

400M

MJ SP

1500

2500

M L

50 500 400 1172 64 100 10 300 PigSPARQL1731 HBase 250 77 200 164 21 10 2318 93 1 0 1001000 319 33 1500 2000 2500 3000 1000 1500 2000 2500 3000 2870 108 LUBM (# universities) LUBM (# universities) 46 1469 0 3431 121 1000 1500 2000 2500 3000 6201000 1500 2000 2500 53 3000 LUBM (# universities) LUBM (# universities) Q8. Return all students with email address that are memPigSPARQL MAPSIN 780 69 500 bers1000 of any department 79of a given university MAPSIN PigSPARQL 931 HBase PigSPARQL

00 000 Q13 500 00 000 000 500 500 000 000 500 000 Q13

164 21 300 WHERE { 2400 1000319 33 200 10 1800 ?X rdf:type ub:Student. PigSPARQL HBase 46 100469 100 1200 174 29 ?Y rdf:type ub:Department. 53 1620 0 10 6001000 3251000 1500 ?Y. 44 3000 ?X ub:memberOf 1500 2000 2500 2000 2500 780 69 LUBM (# universities) LUBM (# universities) 1482 0 72. ?Y ub:subOrganizationOf 9311000 1500 2000 2500 79 3000 1000 1500 2000 2500 645 ?X ub:emailAddress ?Z 84 LUBM (# universities) LUBM LUBM (# universities) LUBM PigSPARQL MAPSIN 800 108 } 10000 3000 128 MAPSIN PigSPARQL 957 HBase PigSPARQL

00 000 Q14 500 00 000 000 500 500 000 000 500 000 Q14

174 100 1000325 PigSPARQL 10 100482 78 1645 10149 8001000 1214 957 2881000 364 10000 PigSPARQL 434

400 3000

Q1 3000

1500 2000 2500 LUBM (# universities) MAPSIN

3000

MAPSIN

750 43 1400 70 500 700 10 79 1288 0 ?X rdf:type ub:ResearchGroup. 250 3000 1000 1500 2000 2500 3641000 1500 2000 2500 89 ?X ub:subOrganizationOf LUBM (# universities) LUBM (# universities) 1 0 4341000 1500 2000 2500 107 3000 } 1000 1500 2000 PigSPARQL PigSPARQL MJ HBase HBase MJ2500 LUBM (# universities) LUBM (# universities) PigSPARQL MAPSIN 545 310 58 42 1000 1000 1026 600 118 87 PigSPARQL MAPSIN SP²Bench 750 1000 1000 1527 896 153 118 100 500 2018 1187 154 750 177 100 10 2519 1476 174 250 214

Q1 00M 00M Q2 00M 00 00M 00 000M 00 00 000

PigSPARQL101 PigSPARQL MJ HBase HBase MJ 0 250 100058 1500 2000 2500 5451000 1500 2000 2500 310 3000 42 LUBM (# universities) LUBM (# universities) 1 0 1026 600 118 1500 2000 2500 87 1000 1500 2000 2500 3000 1000 PigSPARQL1527 PigSPARQL MJ HBase 153 HBase MJ118 896 LUBM (# universities) LUBM (# universities) PigSPARQL MAPSIN 2066 1168 8982 241 2018 1187 154 1000 1000 177 4080 2341 444 PigSPARQL MAPSIN 2519 1476 214 174 750 500 1000 6039 3514 671 100 500 400 8131 4745 834 100 10 300 10037 6005 43597 999 250

Q2 00 00 Q3a 00 00M 00 00M 000 00M 00M 42 000M 87

200 10 PigSPARQL PigSPARQL MJ HBase HBase MJ 1 0 1001000 2500 3000 20661000 1500 2000 1168 8982 1500 2000 2500 241 3000 LUBM (# universities) LUBM (# universities) 1 0 40801000 1500 2000 2341 444 3000 1000 1500 2000 2500 2500 3000 PigSPARQL6039 HBase 3514 HBase MJ 671 LUBM (# universities) LUBM (# universities) PigSPARQL MAPSIN 227 187 70 8131 4745 834 500 1000 435 279 139 (Multiway Join Optimization) 10037 6005 43597 999 400 417 178 100641 PigSPARQL MAPSIN MAPSIN 300 PigSPARQL 10000 845 536 235 200 10 1000 1050 707 274

18 Q3a 54 00M 74 00M Q10 00M 00 00M 00 000M 42 00 87 00 41 18 000 44 54 71 Q10 74 34 00 99 00 00 00 000 41

PigSPARQL1 HBase HBase MJ 0 10 100070 1500 2000 2500 3000 2271000 1500 2000 2500 187 3000 1 LUBM (# universities) LUBM (# universities) 435 200M 279 139 400M 600M 800M 1000M PigSPARQL HBase 417 HBase dist.178 3000 641 (Multiway Join Optimization) 99 1 40 845 536 235 2000 PigSPARQL MAPSIN PigSPARQL MAPSIN 174 84 1050 7071 274 10000 1000 254 1 111 1000 0 340 1 151 100 200 300 400 600 700 800 900 1000 414 1 500 167



3000 3000


3000 3000

10 1

200

300

400

4500


1000M

111 151 167

3000 500 600 700 1500 SP²Bench (triples in million)

800

900

1500

1000

0 200 400 600 (Multiway 800 1000 200 400 600 800 1000 Join Optimization) SP²Bench (triples in million) SP²Bench (triples in million) PigSPARQL MAPSIN

2000

2500

MAPSIN

2001000

3000


100

LUBM

MAPSIN

1 200M 3000 PigSPARQL 10000 2000 1000 1000 100 0 10

200

300

1

12

400M

1000 100 1000 10 100 1 10 1 1000

10


3000

600M (Multiway 800M 1000M Join Optimization)

PigSPARQL

300

400

4500 500 600 700 SP²Bench (triples in million) 3000

MAPSIN

900

1000

1000M

800

900

1000

1500

(Multiway Join Optimization) 0 MAPSIN 600 PigSPARQL 800 1000 200 400

200 400 SP²Bench (triples in million)

200 400 600 800 1000

600 800 1000 SP²Bench (triples in million)

6000

PigSPARQL

200 400 600 800 1000 SP²Bench (triples in million) 200 400 600 PigSPARQL 800 1000 200 SP²Bench (triples in million) 400 600 PigSPARQL 800 1000

4500

MAPSIN

1250 3000 1000 1500 750 0 500 200 250 0 1250

400 600 800 1000 SP²Bench (triples in million)

MAPSIN 200 400 600 800 1000 SP²Bench (triples in million)

1000


750 MAPSIN 500 500 400 250 300 0 200 200 400 600 800 1000 SP²Bench (triples in million) 100

200 400 600 PigSPARQL 800 1000 SP²Bench (triples in million)

500

0

400 100

MAPSIN

6000 200

1

1

0 1000

500 600 700 800 SP²Bench (triples in million) 400M 600M 800M

10

10

200 PigSPARQL

400

10000

1 100

400

MAPSIN

10000 1000

10 1000

3000

LUBM


2000

100 10000

10 MAPSIN 1000 1500 2000 2500 500 LUBM (# universities)

3000

100

1000 1 100 1000 1500 2000 2500 3000 LUBM (# universities) 10

1000

Q1


300(Multiway Join Optimization)

PigSPARQL

10 10000

10000


6000

750

Q1 10000

300

1

10000

3000


1

100

100

2500

100 0 1000

3000

HBase dist.

1 1 1

2000

1000 400 250

200M

SP²Bench (triples in million)

400M1 Join Optimization) 600M (Multiway 40 800M PigSPARQL MAPSIN 1 84

1500

1000

PigSPARQL

1000

100

200M

1500 LUBM (# universities)

100

1000





44 71 34 99

}

500

HBase

1000

10 ?X rdf:type ub:UndergraduateStudent 10 LUBM (# universities)

100 149 SELECT 100?X SP²Bench WHERE 10 {214

PigSPARQL 1 99 3000 174 10000 2000 254 1000 1000 340 100414 0

200M

1

10

1000

250 LUBM (# universities) 100 400M 600M 0 800M 1000M 10 MAPSIN 2000 PigSPARQL 2500 3000 SP²Bench 1000 1500 2000 2500 3000

WHERE {1

1000 1000 Q11. Return all research of a given university 78 16 groups 2100

10


SELECT ?X 100

00 000 500 000 500 000 Q1 00M 00M 00M 00M 000M

100

0 1000 500 1000

10

1

500 500 1000Return all undergraduate students Q14. 10

3500 2800

2400

1800 MAPSIN Q4 10000

100

101

1000 1500 2000 2500 LUBM (# universities) LUBM

0 1000


3000


1000

29 44 HBase 72 16 84 43 3000 1500 2000 2500 108 LUBM (# universities) LUBM 70 128 1500 2000 2500 79 3000 LUBM (# universities) PigSPARQL 89 107 HBase PigSPARQL

2400 3500 1800 2800 1200 2100 600 1400 0 7001000

3000

e in seconds




SELECT ?X ?Y ?Z 10000 100

PigSPARQL

LUBM


750 500

1000 100 628

1000 1000 1500 2000 2500 PigSPARQL 3000

Q1 1000100

3000

00 000 Q11 500 00 000 000 500 500 000 000 500 000 Q11

1000 10000

10 PigSPARQL/MAPSIN MAPSIN 1000MJ: Multiway‐Join of 1500 2000 2500 LUBM 1 10000 3500 1200 MJ 10000 10000 1000 MJ MJ MJ 2800 10 1000 Q1 1000 600 100 1000 1000 2100 1 100 100 M 1000 MJ MJ 14001000 1000 1500 2000 2500 3000 1500 MJ2000 2500 3000 100 100 10 10 LUBM (# universities) LUBM LUBM (# universities) LUBM 10 700 10 10 10 0 1 1 1 MAPSIN 1000 1500 2000 PigSPARQL 2500 3000 1000 1500 2000 2500 3000 110000 1000 1500 1 1 2000 2500 3000 1000 1500 2000 2500 3500 LUBM (# universities) LUBM (# universities) 1000 1500 2000 Q1 Q4 2800 1000 1500 2000 2500 3000 LUBM 2500 3000 LUBM 1000 1500 2 Q6 1000 Q8 Q7 LUBM LU LUBM 10000 1000 PigSPARQL 100002100 MAPSIN 10000 10000 100 1000 1000 1400 Q13. Return all persons that1000 are alumni MJ of a givenMJuni- 1000 100 MJ 1000 1000 10 MJ 750 700 MJ 100 100 100 versity MJ MJ MJ 1 0MJ 500 100 100 10 SELECT ?X 1000 1500MJ 2000 2500 3000 10 10 1000 1500 2000 2500 3000 10 250 LUBM (# universities) LUBM (# universities) 10 WHERE10{ 1 1 1 1 0 ?X rdf:type 11000 1500ub:Person. 1 1000 1500 2 2000 2500 3000 1000 1500 2000 2500 3000 PigSPARQL MAPSIN 1000 1500 2000 2500 3000 1000 1500 2000 2500 3000 200M 400M LU 6 ub:hasAlumnus ?XLUBM 1000M 200M 400M 600M 800M Q6 Q8 Q7 LUBM 1000 1000 LUBM (# universities) LUBM (# universities) Q3a SP² SP²Bench 10000 10000 }Q1 750 100 MJ PigSPARQL MAPSIN MJ MJ 1000 1000 MJ 500 1000 1000 MJ MJ MJ MJ 10 MJ 10000 100 100 250 750 MJ

3000


tim

1172 64 1800 2000 2200 2400 2600 2800 1000 1200 1400 1600 2472 114 1731 77 LUBM (# universities) 1 0 2928 123 1000 1500 2000 2500 23181000 1500 2000 2500 93 3000 LUBM (# universities) LUBM (# universities) PigSPARQL MAPSIN 2870 108 1000 1000 121 MAPSIN PigSPARQL3431 HBase PigSPARQL time in seconds time in seconds

000 500 500 000 000 500 000 Q8

300 200 100

MAPSIN 200 400 600 800 1000 SP²Bench (triples in million)

1

200M 400M 6

Q3a

SP²

41 44 71 34 99


time in second time in second




10000 100 1000 PigSPARQL

1000 PigSPARQL

MAPSIN

300

HBase400

400 500 600 700 HBase dist. 300 SP²Bench (triples in million) 1 40 200

800

900

3000

1000

?inproc 10000 3000

200M dc:title

400M ?title.

600M 1250

?inproc dcterms:partOf ?proc. 1000 2000 100 1000

750

?inproc swrc:pages ?page. 10 0 foaf:homepage ?url. ?inproc

500

300

400

500

250 600

700

1 0 ?inproc dcterms:issued ?yr SP²Bench (triples in million) 200 400 600 800 1000 SP²Bench (triples in million)

}

200

800

500 6000 400

1000 100

4500 300

100 10 10

3000 200 100 1500

1

0 200


PigSPARQL


MAPSIN

10000

1250

1000

1000 750

100

500

10

250

1

0 200 400 600 800 1000 SP²Bench (triples in million)

200

PigSPARQL


MAPSIN 500

1000

400 100

300 200

10

100 1

0 200 400 600 800 1000 SP²Bench (triples in million)

200

900

1000

400

600

800

1000

SP²Bench (triples in million) MAPSIN 6000

MAPSIN

1250 4500 1000 3000 750 1500 500 0 250 200 0 200

400 600 800 1000 SP²Bench (triples in million) 400

600

800

1000

SP²Bench (triples in million) MAPSIN 1250

PigSPARQL

1000 1000

1000

MAPSIN

200

400

600

800

1000

1


13

300 250 200 0 100 200 0

400 600 800 1000 SP²Bench (triples in million) 400 600 800 1000 SP²Bench (triples in million) MAPSIN

200

200 400 600 800 1000 SP²Bench (triples in million) PigSPARQL

time in seconds

1000


(Multiway Join Optimization) PigSPARQL MAPSIN 1000 10000

1

500

1000

1000M

900

1 10

200 400 600 800 200 400 600 800 1000 1000SP²Bench (triples in million)

?subj ?predSP²Bench (triples in million) person:Paul_Erdoes

1000

?inproc rdfs:seeAlso ?ee.

200

800M

10 100

}


MAPSIN

100 1000

WHERE 10 {1

0 SP²Bench (triples in million)

PigSPARQL

1000 10000

SELECT10?subj ?pred

10000

?inproc 10 dc:creator ?author. ?inproc ?booktitle. 1 bench:booktitle PigSPARQL

800

500 Q10. Return all subjects that stand in any relation to 750 100 400 person ”Paul Erd¨ os” 500 100

4500

200 400 600 800 1000 200 ?inproc 100 rdf:type bench:Inproceedings.

0 600 (Multiway 800 1000 200 Join Optimization)

10000

100?inproc ?author ?booktitle ?title (Multiway Join Optimization) SELECT 3000 PigSPARQL MAPSIN PigSPARQL MAPSIN 10 ?proc ?ee ?page ?url ?yr 1500 1000

400

200 400 600 800 1000 SP²Bench (triples in million) PigSPARQL

99 10 174 1 Join Optimization) 84 100 (Multiway 254 1 0 111 PigSPARQL MAPSIN Q2. 1Return all inproceedings with the given properties 1000 2000 2500 3000 100003401000 1500 2000 25001 3000 151 1500 query (slightly modified version of the original without LUBM (# universities) LUBM (# universities) 1 BY) 6000 167 1000414 OPTIONAL and ORDER

WHERE {1

1000M

10000


time in secondstime in seconds

200

1000

4500 3000 500 600 700 SP²Bench (triples in million) 1500 bench:Article. 400

PigSPARQL

}

500

0

1000 1000

200

500

100 PigSPARQL

800 LUBM 900

400M 600M 800M (Multiway Join Optimization) PigSPARQL MAPSIN

FILTER (?property=swrc:pages) SP²Bench (triples in million)

HBase MJ

1000

500 700 Q1600 SP²Bench (triples in million)

1 ?article ?property ?value

3000

MAPSIN

100 227 187 70 PigSPARQL1010 HBase HBase dist. 250 139 435 279 99 1 40 1 417 1641 0 178 1741000 200M 1 3000 600M 100084 800M 1500 2000 400M 2500 1500 2000 1000M 2500 845 536 235 3000 LUBM (# universities) 254 1 111 LUBM (# universities) 1050 707 274 2000 340 1 151 PigSPARQL MAPSIN 414 1 167 1000

1 400

10000 2000

10 ?article rdf:type

750

HBase


1

SELECT ?article 100 0 WHERE { 200 300


1000PigSPARQL


42 87 18 54 74

PigSPARQL MJ HBase MJ 1168 241 PigSPARQL10 HBase HBase MJ 250 100 4080 rdf:type bench:Journal. 2341 444 ?journal 227 187 70 1 0 6039 3514 1 (1940)"ˆˆxsd:string. 671 ?journal dc:title "Journal 4351000 1500 2000 2500 279 3000 139 1500 2000 2500 1000 8131 4745 834 ?journal dcterms:issued ?yr LUBM (# universities) LUBM (# universities) 641 417 178 10037 6005 43597 999 } 845 536 235 PigSPARQL MAPSIN (Multiway Join Optimization) 1050 707 274 2066 WHERE 10 {

10

6000 Q3a. Return all articles with property swrc:pages

750 300 HBase 500 200 8982

100 SELECT ?yr PigSPARQL

MAPSIN

200M

500 1000

1000

2000 PigSPARQL 10000 1000 1000 0 100 200 300 10

3000

Q1. Return the year of publication 400 of ”Journal 1 (1940)”


41 Q10 44 00 71 00 34 00 99 00 000

3000 3000

SP BENCH


42 Q3a 87 00M 18 10 00M 54 00 00M 74 00 00M 00 000M 00 000

B.

time in seconds

Q2 00 3a 00 00M 00 00M 00 00M 000 00M 000M

2800 750

1000

100 PigSPARQL PigSPARQL MJ HBase HBase MJ 2100 500 100 545 310 42 1400 58 10 PigSPARQL10 PigSPARQL MJ HBase HBase MJ 250 1026 600 87 700 118 2066 1168 8982 241 1 0 153 1527 896 118 1 0 40801000 2341 444 1500 2000 2000 2500 2500 3000 3000 1000 1500 2000 2500 1000 2000 2500 20181000 1500 1187 177 1500 154 LUBM (# universities) LUBM (# universities) LUBM (# universities) LUBM (# universities) 6039 3514 671 2519 1476 214 174 2 8131 4745 834 PigSPARQL MAPSIN 10037 6005 43597 999

time in seconds

Q1 00M 2 00M 00 00M 00 00M 00 000M 00 000

400 100 10 1

200 400 600 800 200 400 600 800 1000 1000SP²Bench (triples in million)

300 200 100 0 200


Cascading Map-Side Joins over HBase for Scalable Join ... - arXiv

Cascading Map-Side Joins over HBase for Scalable Join ... - arXiv

Suggest Documents

Cascading Map-Side Joins over HBase for Scalable Join Processing

Scalable Join Patterns

Analysing Analysing Cascading over MapReduce

Scalable and Efficient Self-Join Processing technique in RDF ... - arXiv

Radio Resource Allocation for Scalable Video Services over ... - arXiv

Load Shedding for Window Joins over Streams

Comparing Different Models for Investigating Cascading ... - arXiv

Exploiting Join Cardinality for Faster Hash Joins - UBC's Okanagan ...

MD-HBase: A Scalable Multi-dimensional Data Infrastructure - CiteSeerX

Scalable and Adaptive Online Joins - VLDB Endowment

Adaptive Geospatial Joins for Modern Hardware - arXiv

MD-HBase: A Scalable Multi-dimensional Data Infrastructure - CiteSeerX

Scalable Algorithms for Nearest-Neighbor Joins on Big ... - Silviu Maniu

Map-Side Merge Joins for Scalable SPARQL BGP ... - CiteSeerX

MD-HBase: A Scalable Multi-dimensional Data Infrastructure - CiteSeerX

Scalable Sweeping-Based Spatial Join - VLDB Endowment

gSMat: A Scalable Sparse Matrix-based Join for ...

A Scalable and Skew-insensitive Algorithm for Join Operations ... - Hal

Reservoir Sampling over Memory-Limited Stream Joins

Estimating the Propagation of Interdependent Cascading ... - arXiv

Usage Patterns for Apache HBase

ApacheCon2009: Practical HBase

LNCS 4016 - Load Shedding for Window Joins over ... - Google Sites

Load Shedding for Window Joins over Streams - Journal of Computer ...