Stubby: A Transformation-based Optimizer for MapReduce Workflows

Recommend Documents

See Spot Run: Using Spot Instances for MapReduce Workflows - Usenix

also find that SI termination due to budget constraints during the job can have ..... Open AppEngine Application Development and Deployment. In. International ...

optimizer for the ... - CiteSeerX

Heuristics to Minimize Multiple-Valued Programmable Logic Arrays*. John M. Yurchak ... A manual, Yurchak and Butler [15], exists giving com- plete information ...

A Multi-Query Optimizer for Monet

Although the users act independently, formulating different queries, it is not unusual that the ... thermore, they point out that a multi-query optimizer should at least perform as well as a .... Monet is a main-memory database kernel developed at CW

MAPREDUCE FOR INTEGER FACTORIZATION

Jan 4, 2010 - [4] Israel Kleiner. 2005. Fermat: The Founder of Modern Number Theory. Mathematics Maga- zine, Vol. 78, No. 1 (Feb., 2005), pp. 3-14.

Mapreduce Notes Install mapreduce locally:

... datasets to test whenever possible. Install mapreduce locally: Based on http:// www.stanford.edu/class/cs246/cs246-11-mmds/hw_files/hadoop_install.pdf.

SHAHED: A MapReduce-based System for ... - users.cs.umn.edu

Such inefficiency makes it hard for .... As a result, we have developed a simple data recovery ...... machines have 16GB of memory, 2TB hard disk and a quad.

Workflows for eScience

Call the Grid Scheduling and Execution service client code and pass to it the output .... mosaic generated from images obtained with IRAC; the bright galaxy left of ...... We will not describe the S5 inspiral pipeline in detail here, suffice it to sa

Hashdoop: A MapReduce Framework for Network ... - NECOMA

mental difficulty in order to detect network traffic anomalies by using Hadoop. .... Astute [14], an anomaly detector monitoring traffic stationarity. The packet count ...

Twister: A Runtime for Iterative MapReduce - CiteSeerX

Jun 22, 2010 - Both Twister and Hadoop use JDK (64 bit) version 1.6.0_18, while. DryadLINQ and MPI uses Microsoft .NET version 3.5. Cluster ID. Cluster-I.

A MapReduce Framework for Heterogeneous Computing Architectures

Computing Language (OpenCL), the developer still needs to manage the ...... It was initially developed by Apple, and then submitted to Khronos Group to develop its .... account the data transfer overhead when making scheduling decisions.

DyScale: a MapReduce Job Scheduler for ...

... Department of Computer. Science, College of William and Mary, Williamsburg, VA 23187, USA. ... Zhuoyao Zhang is with the Department of Computer Science, University ...... [28] J. Polo et al., âPerformance management of accelerated mapreduce wor

A Data Centric approach for Workflows - CiteSeerX

of rules to achieve, or contribute to, an overall business goal. We believe that ... [5] specification built on top of existing Web Service standards addresses the ..... Model for recovery, steering and analysis purposes. Even the optional data ...

anvaya: a workflows environment for automated ...

Pune Ð 411007, Maharashtra, India ... workflow environment that has an advantage over existing software as it enables extensive pre ... functionalities to carry out exhaustive comparative analysis via \custom tools," which are tools ... encountered

A Framework for Distributed Preservation Workflows

opment of preservation tools for the automated characteriza- tion, migration, and comparison ... vation plans and automated workflows represents a major research goal in this area. ..... and email notification capabilities. It currently provides a.

Towards a Benchmark for ETL Workflows

that a plethora of ETL tools currently constitutes a multi-million dollars market. ... a principled organization of test suites for the problem of experimenting with ETL ...

anvaya: a workflows environment for automated

automate data-flow between tools that hitherto, required manual intervention. ..... datasets can be processed in practical time scales as compared to the \endless wait .... Anvaya Client: Platform independent (tested on Windows XP/Vista and Linux ...

Cloud MapReduce: a MapReduce Implementation on ... - Google Sites

a large-scale system design and implementation if we build on top of it. Unfortunately .... The theorem states that, of

A Secure Task Delegation Model for Workflows

Oct 28, 2008 - Email: [email protected], [email protected], [email protected] ... of mechanisms is task delegation that allow at workflow.

A Corporate Semantic Wiki for Scientific Workflows

has been made with standards for service-oriented computing, grid computing, cloud computing, business process management and semantic representations ...

Towards a Benchmark for ETL Workflows

of the ETL workflow is preferred in many data warehouse projects, due to the .... overlapping groups of nodes which are attached to the body of the butterfly.

Distributed Workflows: A Framework for Electronic Commerce

secure distributed workflows is presented in section 6. In section 7 we discuss ... Most of the workflow transaction management theory for multilevel secure ...

A Learning Optimizer for a Federated Database ... - Semantic Scholar

general, monitoring needs to utilize monitoring tools of the remote databases or cleverly .... provides the best performance and least impact on the remote server.

A Grey Wolf Optimizer for Modular Granular Neural Networks for ...

Jul 10, 2017 - end while return . Pseudocode 1: Pseudocode of the grey wolf optimizer. where is the maximum number of subgranules that the.

Incoop: MapReduce for Incremental Computations

speedups in practice. Even though it would be possible to devise solutions that work with various types of data processi

Stubby: A Transformation-based Optimizer for MapReduce Workflows

Download PDF

3 downloads 5358 Views 438KB Size Report

Comment

Aug 1, 2012 - Web clicks, social media, scientific experiments, and datacenter monitoring are among sources .... HiveQL. Workflow Scheduling and Execution Engine ... Stubby will find the best plan subject to the given annotations, while ...

Stubby: A Transformation-based Optimizer for MapReduce Workﬂows

arXiv:1208.0082v1 [cs.DB] 1 Aug 2012

∗

Herodotos Herodotou Duke University

Duke University

[email protected]

[email protected]

[email protected]

ABSTRACT

J1

M1 R1

J1.K1={C} J1.K2={O} J1.K3={O}

D02 J2

M2 R2

J2.K2={O} J2.K3={O}

D2

D1 J3

M3 R3

J3.K1={O} J3.K2={O} J3.K3={O}

M4

J4.K1={O} J4.K2={O} J4.V2={S,Z,P}

D3 J4 D4 J5

M5 R5

D5

J7

M7 R7

J5.filter={50