The Utility of Exploiting Idle Workstations for ... - Semantic Scholar

The Utility

of Exploiting

Idle Workstations

Anurag

Acharya,

Computation*

Guy Edjlali, Joel Saltz

Department University

for Parallel

of Computer

of Maryland,

Science

College Park 20742

{acha,edjlali,saltz}Qcs.umd.edu

has been has been in operation

Abstract

sin for about In this paper, stations

we examine

for parallel

the following what

computation.

questions.

fraction

First,

of time available?

opportunity

for parallel

bution

computation

might

machine

I expect

Finally,

real machine

benefit

this happen?

workstation

allel machine,

pools.

of well-known

parallel

To answer

programs

the system

lel programs

idle

updating

of the

have

parallel

data job

uously

the equivalent

par-

between able

pools.

problems, for adaptive an adaptive

available

and

not

parallel

tions

with

fraction

a very significant the

of time.

Second,

parallel.

That

will be is avail-

parallel is, they

programs are able

- for example,

and

t,o

configura-

Addition

the

If

oscillates

processor

or deletion

a small

on the performance

requirements

of time.

computation

may have no effect,

effect

application

a parallel

a

be contin-

rapidly

even if each

and

progress,

period

of processors

processors.

workstation

migration

of processors

configurations

powers-of-two

of a single

1 Introduction

busy,

the com-

may need one

To make long

a par-

remaining

to allow

data/process

a group

progress

perfectly

the

in

written

running

user,

information.

written are

Reconfiguration

number

little

programs

to stop

This

computation.

programs

for a sufficiently

run only on certain

application.

have

that

of a large

are often

paral-

job

has no effect,

parallel

primary

repartitioning,

requires

to make

most

of free

of execu-

workstations.

a workstation

by its

location

able for a large

we have

for

When

same

available

the state

thereof,

data-parallel

style).

or more of data

on a

of a group

hold

(most

for parallel

progress

on other

to be reconfigured.

from

on these workstation required

fluid dynamics

processes

that

or the lack

is reclaimed

putation

workstations

so for data-parallel

style

job

traces

the execution

and have used it to build

computational

allel

this in con-

not

is particularly

in an SPMD

the workstation-

14day

of the practical support

This

about

have held out the promise

of execution

does

an SPMD

of Wiscon-

manages

the workstation-availability

implicitly,

on one workstation,

assumption

Fourth,

programmer

at least

on the progress

is useful

on.

can be achieved

To determine

To gain an understanding

tion

how big a parallel

we have analyzed

we have simulated

is

to

to get for free by harvesting

how much

questions,

developed

To state

of size S,

and how hard does a parallel

to work to make availability

a pool

[9, 17, 19, 251 that

vary

idle

First,

assume,

is the distri-

information

computation

can a user expect?

if I have

should

This

to place

of harvesting is less clear.

results

for adapting

what

utility

cycles

how stable

currently

[6].

of k

how frequently

to stop

Third,

The

computation

at the University

and which

of the

the stability

indicates

have

idle-times?

workstations

machines.

three

This

of workstation

terms,

an estimate Second,

300 workstations

pool, for

to find a cluster

and how does

availability.

how much benefit crete

to answer

a workstation

computation.

in processor

for selecting

We attempt

provides

with the size of the cluster? a parallel

idle work-

given

This

of free machines

changes

of exploiting

can we expect

workstations a cluster

the utility

8 years

effect

or

depending

number

on

of available

machines. Exploiting

idle

workstations

area.

This

which

have indicated

are unused partly

popularity

rapid

Batch-processing running many

years.

‘This

have

A well-known

research

utilize been

In this

is Condor under

contract

for

swer

the

pool,

for what

following

stable

which

bility

#F19628-

SIGMETRICS 0

1997

to republish,

to lists, requires

ACM

‘97

Seattle,

to poet on aarvera,

prior specific

WA,

permission

how much crete

a fee.

USA

0-89791-909-2/97/0006...$3.50

225

terms,

machine

in processor

than

time

benefit

idle-times? 1?

to place

that This

a pool

I expect

That

information

of size S,

for

what is

is, what

is currently

To state

how

to stop

Third,

computation

how

the sta-

indicates

have

availability.

can a user expect?

if I have

should

an estimate Second,

This

might

a workstation

workstations

to find a clus-

provides

and how does

computation

idle to an-

a workstation

computation.

of workstation that

be idle for longer for selecting

or to

and/or

given

This

of free machines

to changes

the probability

notice, the title of the publication and its data is given that copying is by permission of ACM,

Inc. To copy otherwise, redistribute

available?

a parallel

the distribution

tage, the copyright appear, and notice

First,

vary with the size of the cluster?

adapting

Permiaaion to make digital/hard copy of part or all thia work for personal or classroom we is granted without fee provided that copies are not made or distributed for profit or commercial advan-

of exploiting We attempt

of time can we expect

for parallel

is a cluster

frequently

#353-1427

the utility

computation.

questions.

fraction

of the opportunity

use for

[15],

we examine

for parallel

ter of k workstations

of workstations.

idle workstations

paper,

workstations

of workstations

in production

by ARPA

subcontract

research by studies

[9, 17, 19, 251 and

power

example

was supported

Syracuse

partly

fraction

of time

in the

that

a popular

fueled

a large

growth jobs

been

been

fraction

systems

sequential

94-C-0057,

that

for a large

by the

has

has

is

idle will is useful

on.

Fourth,

this in con-

how big a parallel

to get for free by harvesting

idle

machines.

Finally.

real machine

how much

on a

that

programmer

have

utilize

in three different

ways.

benefit

can be achieved

and how hard does a parallel

to work to make this happen? We have addressed To answer analyzed ferent

the

workst,ation-availability traces

l&day

sizes

locations

(40,

these

from

(College

Park,

workstation

pools.

To gain

that

arise

programs

in an adaptive that

environment

and to adapt

parallel trying we have

to these

version

using an IBM

SP-2

the workstation

as a cluster

availability

ger and Sun

[14] use an analytic-model-based

the feasibility

of running

dedicated

workstation

synthetic

models

allel

program

about

pool.

of both

behavior.

behavior

propose

schemes

workstation

most

slave style. master-slave

Their

memory

study

It is difficult

to draw

programs

There

Second.

rewriting

existing

suitability

processor.

of dedicated

analysis

approach

availability partition

programs.

and

trace,

a job

and a suite show that

t,he workload approach

scheduling

to that

differences.

policy.

They

from

the presence

jobs

in the CM-5

are, however,

bet.ter

jobs.

job execution

nell.

short,-lived section

for interactive

jobs

used

Diego).

We found

16 processors

We take the position can be met

that,

or less that

by a small

over

To

cluster

2.1

Traces

The

first

for what

on the

of available

average

workstations

are for comparison

with

the probabil-

for all the workstations

the three metrics

trace

is from

traces.

in this

We then

and the other

the workstation

at the UC Berkeley

workstations.

This

02/15/94

present,

measures

03/31/94.

and

This

trace

the

cluster

and contains

trace

and availability

covers The

periods

for

trace

segment

workstations.

We refer

for about.

period

60

between

we received

marked which

of the CAD

data

a 46-day

had

t.he

in for each worksta-

was used by Arpaci

14-day

second

trace

for

status

several

criteria,

trace

This

The

see

monitor

et al in [2].

We ex-

had the largest to this

trace

number

as the ucb

project third

land.

This

and covers

226

using

workstation data

covers

a 14-day

to be available

user

preferences,

for batch

jobs.

status

period

the

Condor

uses

to decide We collected

information

the web interface

trace

is from the public

of Computer contains

a 14-day

period

Science,

data

be-

whenever

workstation University

for about

from 09/24/96

if a this

once

provided

[5]. We refer to this as the uisc trace

pool

for about

of this trace,

it available.

the Condor

minutes

Condor

and contains

For the purpose

marked

including

t,he Department

and

trace

is available

by sampling

Condor

of

the

and 10/07/96. was considered

Condor

ery three

is from

of Wisconsin

workstations.

workstation

t.he need for interacdedicated

First,

traces.

a workstation

(Cor-

(for details,

describe

run-

We refer

we have com-

trace.

we have computed

for idle-times

We first

Second,

computation

available

measures

Finally,

tween 09/24/96

ma-

90%

These

metric.

of I; workstations

In addition,

for each

study.

300

to a year)

centers

metric.

ity distribution

The

a small

performance

(six months

time?

fraction worksta-

availability

is a workstation

at the University

re-

runs.

for

to run undisturbed?

measures

studies.

what

of L free

is a cluster

expect

two-week

trace.

are

parallel

as the

clusters

three

For each of these traces, First,

how does the number

with

of traced

Our

a time-sliced

Usually,

workstation

to find a cluster

stability

of time

tracted

of short-lived

from three supercomputer

San

3.2).

mode.

as the

tion.

on the interac-

number

pools.

two metrics.

to this

two other

busy

Their

there

of free

is, how long can a parallel

group

CM-5

partition.

Most large

long-term

the

to process

et al but

the need for interactive

and

tive response

of a large job trace.

of

pools

the need for interactive

are provided

we analyzed traces

Maui

CM-5

and assume

run in a batch

of processors understand

parallel

deduce

is able

et al focus

jobs

a

as

com-

programs.

of Arpaci

we present

we have analyzed

the parallel-availability

a trace-based-

for a 32-node

availability

we expect

That

all three

on a workstation

pool

Arpaci

of parallel

parallel number

trace

to a 32-node

sponse chines

arrival

programs

workstation take

study

a 60-workstation

submitted

performance

their

to a differ-

et al [2] study

of five data-parallel

is closest basic

base

de-

and execu-

Finally,

stable?

previous

If the

very large amounts

They

program.

on the average,

vary

et al [al]

the total

the

and second,

in a master-

parallel

and non-dedicated

parallel

we describe We then

the implementation

parallel

computed

fraction

is reclaimed,

increase

Arpaci

Next,

availability

We refer

puted

with this approach.

are not written

on the master

to estimate

results.

for how long,

to this

conclusions

approach.

programs

and would require

with

ning on k processors

and par-

by the master

would greatly

to

on simple

et al [4] and Pruyne

are two problems

and their

computation,

can

tions.

on real workstation

on a master-slave

computation.

from three workstation

we have

on non-

is based

parallel

volume

for executing

tive

traces

as the

approach

applications availability

Carreiro

based

programs

munication

several

To determine

Leuteneg-

on which a task is being executed

ent workstat,ion.

Workstation

for parallel

for parallel

workstation

the task is killed and is reassigned

results

above

approaches.

parallel

of real parallel

pools from their work.

First,

2

fluid dy-

into using idle workstations one of three

for parallel

questions.

computed

sys-

in their

and one of

mentioned

has taken

in utilizing

of our conclusions.

of time

research

success

jobs.

and the metrics

experiments

tion of an adaptive

workload.

Previous

group at the I:niver-

the workstation-availability

the traces

our experience

summary

performance

of workstations

traces

examine

that

so. we follow

We have also

its actual

computation study

developed

of a computational

and have measured

for sequential

scribe

parallel

changes

changes.

idle workstations

the opportunity

of the

to run

to detect

who have had excellent

our simulation

on

goal of schemes

In doing

Livny and the Condor

We first

the

be the primary

workst,ations.

t.he lead of Miron

We describe

To deter-

programs

should

non-dedicated

sity of \Visconsin

of dif-

an understanding

allows programs

an adaptive

program

pools

and at different

we have simulated

when

fashion,

tem support

we have

and Madison).

machine,

of well-known

problems

sequential

workstation

13erkeley

practical

namics

three

parallel

of a group

developed

questions,

60 and 300 workstations)

mine the equivalent execution

these questions

throughput

ev-

by the

trace. clust,er

of

of Mary-

40 workstations to 10/07/96.

For

the purpose

of this

be available

if the load average

five minutes. The

trace.

We refer

number

these

figures

as a measure

was 52 for the ucb We use

location, used.

to the

these

The

machines

College which

ence graduate personal

Park

belonging

that

for their were

that

both

these

clusters

compute

several

2.2

pools

available

of relatively hours).

are representative

of most

in university

would

and perWe ex-

Figure

1 presents graph

For each

pool,

metric

however,

that

for all three

drops

that

a substantial

Each

2 presents

the stability

shows

how the metric

graph

These pool

graphs

show that

are stable

to a third This

holds cost

out

of reacting

one minute,

point

that

workstation

large

time,

these clusters

88 workstations Figure

the probability

1 but a cluster

Figure

are idle varies indicated

available. t.o 80% agrees

the

dips.

of the

results

workstations

with previous

results

pools.

In each

graph,

fraction that,

in a pool

3

as

probability

that

an idle workstation

should

of minbetween

of compu-

We considered five minutes varied

> T)

less than

half.

4 shows The

etc)

The

by

to t,his

shorter

than

longer

minimum

T

than

value of

and the maximum

value

distribution

of

value of the idleness-cutofl

trace,

70 minutes

for the aisc

trace.

simple

for selecting

suffice.

To summa-

We refer

the cumulative

average

for the ucb

0.5.

those

We found

workstation

half; idle periods

was 18 minutes

higher

widely.

Idle periods

than

only

long.

each

=

traces,

an idle period

strategies between

We note that

the large

(such

as LIFO.

available

all of these

than the 26 minutes

for the umd

Given

worksta-

values are sig-

reported

by Douglis

[8]

workstations.

of

have

repeated

correspond

throughput.

by

programs

research

that,

groups

work-

two scenarios:

applications

of the entire

without

(1) gaps;

set of applications,

scenarios

keep the pool

an approximate

equivalent

on all three

which includes

[22] and three We simulated

these

provide

The

programs

programs

achieve

the execn-

parallel

upper

bound

machine

also

busy

at,

on the

is used as the

metric.

the horizontal

This

Since

they

parallel

of individual

execution

might

we simulated

by one or more

processing.

gaps.

programs

of eight

benchmarks

execution

all times,

are

a suite

studied

(2) repeated without

that

are available.

been

that parallel environments,

of well-known

We selected

ing on parallel

nights.

60%

the benefit

workstation

the NAS parallel

at the

on the average,

How much benefit can a user expect?

pools.

We first

of the pool that. is

describe

We then describe to drive them.

3.1

the question

P(z

greater

tion of a group

[2, 9, 171.

we try to answer

that

Figure

random

in shared

only for

Weekdays

t.

as the idleness-cutofl.

To estimate

- what is the

will be idle for longer

Finally,

we used as benchmarks. and the information

we present

in t,his suite

model.

Figure

5 shows

running

on dedicated

have been obtained

227

the programs

our simulations

used

the results.

Benchmarks

All programs

Distribution of workstation idle-time

In this section,

T such

nificantly

pool as per

and weekends

time

> t) that

we characterized

in 1990 for the Sprite

An

of workstations

by the humps

indicate

tions

1 shows

upturns

on weekend

fraction

avg shows the average These

The

(tens

up

a cluster

is stable

2).

for the three

nights

the shorter-than-usual line labeled

For example,

we

we looked

in the availability

distribution

and 90 minutes

FIFO,

at any given

for the ucb and mad traces

with time

by the dips;

Figure

Instead,

for the placement

P(z

value of the idleness-cutoff,

minutes. Even if

progress.

are available

(see Figure

how

trace

size.

is as high

be found in the nisc

of idle periods

3 shows

event

of 88 workstations

right, end of the graphs number

applications. significant

are not stable.

can always

time

was 40 minutes

pools.

cluster

Given

availability.

long idle periods

were at least

the idleness-cutofi

half

and clusters

even though

clusters

five and a half minutes t,o a small

to make

to note is that

with

minutes

to a reclamation

important

than

for five to thirty

for parallel

it is possible

with

fraction

larger

of workstation

occurring

than

that

was 9 hours.

for the umd

for all three

varies

to fifteen

promise

of

up to half the size of the

are stable

longer

the idleness-cutoff

for over half the time.

metric

clusters

for four

of the pool

Except

slow - clusters

the t,otal size of t.he pool are available Figure

last

T had a probability

size.

more or less linearly

for each pool,

the drop is relatively

pools.

a cluster

a recruitment

Our goal was to help select

rize the information,

cluster

for which

(20-70%)of the pool is always available. trace,

with

of 26 minut,es;

study,

the best throughput.

the probability

idle periods

environments.

varies

idle

were idle for 30 sec-

in their

workstations

had a probability

of time

is available

available

measure

how the metric

the fraction

I; workstations k. Note,

the availability

shows

in terms

For each workstation

Parallel-availability metrics

Each

2.3

provided

at distribution

the

the

of 3 minutes

we computed

research.

servers

that

been

[9] mention

for an average

that,

plenty

have

tation.

of

departments.

machines

to be idle

at h>

has been

time are more

that

Douglis&Ousterhout

utes to several

group and is used

and compute-intensive

It spans

together

pool consists

cluster,

likely

machines

et al [Z] mention

multiple

as well as for

Berkeley

to a single research

sci-

looked

experience

did not focus on the issue of recruitment.

available

computer

than

long period.

the relative

and

are (were)

of publicly

assignments

The

pool includes

workstation

period

way they

consists

etc).

purposes

workstations.

time

used by junior

for course

(mail

personal

Madison

pect

pool

students

workstations

in size,

also vary in the

are primarily

purposes

for both sonal

pools

variations

common

t.hat have been idle for a short

onds

threshold

In addition

has hern previously

[2. 91. The

to be reclaimed

Arpaci

of the size of the corresponding

pools.

The

machines

for a relative

The aver-

and 39 for the umd trace.

This question

researchers

likely

in the pools

periods.

workstations

trace

time tl

several that

participating

the tracing

of participating

277 for the aisc

than

to

to this as the umd trace.

throughout,

trace,

was considered

was below 0.3 for more than

of workstations

was not constant age number

a workstation

are programmed

the parallel

speedups

machines.

from publications

in the SPMD

for the

benchmarks

These

numbers

[l, 3, 22, 23, 261. The

I, und

-

/

% 2 1 E = 6 f 2 !i

06

i\\ /

/

0.61

04i ~ 0.2; I

0

Figure

\

5

10

15 20 25 CklerSW

1: Availability

free workstations workstations

Figure

5

IO

2: Stability

35

40

for the three

pools.

and how this fraction

was 52 for ucb,

\’

0

30

These

varies

277 for uisc

graphs

with

show for what fraction

the cluster

of time

size Ic. For comparison,

can we expect the average

to find a cluster

number

and 39 for mad.

’ ’ lend

15 20 25 alBier she

30

35

for the three

40

0

pools.

These

0’ 10

graphs

20

30 40 Cluster sue

50

plot the average

60

period

0

a cluster

8 M

1w

is stable

’ 19 200 clwter me

for against

1

% 2

0.8

E ". c Jz

0.6

E 5 i

0.4

B I;

0.2

0

of k

of participating

und wg ;ri I;

25ccco !iccccQ75mm Tme (secondr)

le+ce 1.25&e

Figure

0.2

0

3: Fraction

25x03

5xm 75m Tme@mds)

of workstations

228

le+c6

0

2xcol

7xm Tl$Ed$

available

for the three

pools.

’ 250

300

the cluster

size.

081 I

0.8

K

i

0.8

x a s 5 e

0.7 0.6 0.5

0.7

t

0.6 i

8

0.5 I

0.4

8

0.4 ’

0.3

e B

0.3

0.2

s 5 e u.

0.2

0.1 t

t

0.1 ! Oi

OL

0

Figure programs datasets l

themselves

are described

nas-bt:

this program

of the idleness-cutoff

We used class

tions

[ZZ]. The

IBM

SP-2

tions

running

this

running

is 7955 325.8

B

120 -

square

time

program

discrete

and

SP-2

This

program

memory

requirement

f tpde:

ferential

is

factorization

is 8312 seconds MB.

to solve

time on one

machines.

time

program

algorithm

to solve

equation

[22]. The

running

SP-2

is 228

requirement

on configurations

5: Speedups

using

is 461

with

SP-1

is 286 seconds and the total

dsmc3d:

This

program

partial

sec-

l

[I]. IBM

simulation

hydro3d:

have

seconds

and the total

unstructured: the Navier-Stokes

used

3.2

with

this

memory

requirement

this is a flow solver equations

about

the use of unstructured

capable complex

grids

is a parallel

relativistic

and the total

data

We

on 1,2,3,4,5,10,25

implementation

hydrodynamics

for this

of the Intel

memory

program

of 3 + l[26].

Delta

required running

Its runis 406000

is 89.2 MB. We on 1,8,16,32

and

Simulations

To compute

to study [23]. The

of the iPSC/SSO

is 68814

is 134 MB.

64 processors.

requirement

in three dimensions

time on one processor

on dedicated

Paragon

required

running

ning time on one processor

processors.

is a Monte-Carlo

Intel

memory

for this program

dimensional

dif-

of the

memory

120

and 50 processors.

MB.

algorithm

processors

runs on configurations

running

through

FFT

100

programs

of the

and the total

have data

powers-of-

a Poisson

3-D

on sixteen

the flow of gas molecules

l

the

running

powers-of-t.wo

time

solves

The

is 1.75 GB.

l

this program

of the benchmark

on one processor

seconds

processors.

a multigrid IBM

Figure parallel

40 60 60 Number of Processors

and the to-

This

with powers-of-two

of the

20

seconds

equation

... .. ._

with

two processors. s nas-f

,..‘.

SP-2

a block-lower-triangular

memory

runs

----

[22].

of the IBM

is 174.8

processor

: :’

matrix-

equations

The running

Poisson

the total

.-.---~

60 -

0

equations.

this implements

on one

a pentadiagonal

uses

runs on configurations

time

runs on configura-

approximate

requirement

onds

80 -

re-

runs on configurations

program

of the IBM

the scalar

program

on one processor

the Navier-Stokes

nas-mg:

of the

memory

of processors.

this

tal memory

on one processor

pools.

of processors.

uses

block-upper-triangular processor

equa-

and the total

and the total

This

number

nas-lu:

Navier-Stokes

lcm1wJl2cm25oM)mm L~ofdkpemd(seconds)

for the three

nas-bt nas-w nas-lu unstructured dsmc hydro3d nas-w ideal

100 -

based on block-

for the Navier-Stokes

seconds MB.

This

number

program

algorithm

the

time

seconds

is 1.3 GB.

0 nas-sp: The

to solve

is 10942

with square

based

uses an approach

matrices

quirement

l

below.

distribution

for all the NAS benchmarks.

tridiagonal

l

4: Cumulative

5cca

the equivalent

ios mentioned

above,

the execution

of SPMD

workstation

is 4876

pools.

tion availability

is 30 MB.

grams time.

of solving

the

229

machine

programs

simulator

takes

and a description and

programs

computes

simulation

to complete

the execution

used

study

can

of

on non-dedicated as input

worksta-

of the parallel the

total

measure

the size of the parallel in this

for the scenar-

a high-level

The equivalent-parallel-machine

be required

[3]. Its running

parallel

The

traces

to be simulated

by determining

geometries

parallel

we performed

machine

is computed that

in the same run

only

pro-

execution would

time.

All

on a fixed

wt of configurations

(e.g.

time falls in between

t.wo c,onfiguration.

is used to comput,e

the equivalent,

All the programs purpose the

time

taken

t,o execute

per iterat,ion

We characterize

the

program

data

by code inspection

t,he last

t.wo.

For the last

size information

from

of these

which cannot station.

data.

reasonably

to a change

required

of available

the evicted

Since SPMD

program

is another,

For

excellent,

possibly

cheaper, can

If sufficient to processes

t,he program

in (loose)

be moved;

able,

is moved

moving

number

of the there

at the new location

data

workstations

this scheme

- expanding

the number

of processors

new processes).

There

tion can happen

only when the data is in a “clean”

usually

means

startup

cost

nication The

are two points that

outside

every

parallel

also includes

schedules.

adaptation

simulator

loops.

the cost

will reach.

Second,

we have assumed

It models

and a variable

part

that

includes

required

cost. is paid at least to a change

for eviction.

it, is paid every time

in processor

The

to account

once

availability.

DEC

Server

Alpha

tion

4/2100

also models

of one program

set.tling

period

Since

Digital

period

and the start

Unix

between

3.2D.

of another.

We used

idle workstations

et al [2] f ecus

Arpaci

mance

of parallel

policy.

They

are relatively

a scheduling

jobs

deduce

and assume

plentiful,

strategy

on the interactive a time-sliced

the need for interactive

most and

compute

perforfrom

ration

230

from Park),

machine However,

may

strategies

effect

varying

an aggregate

There

show

parallel

no effect,

programs

a small

effect.

depending of avail-

important.

hovers

ef-

limztecl

or delet.ion

the number

variation Since

periods,

number.

program.

The

when

around

“magic

and squares.

of the experiment.

the graphs

load.

is particularly

7 shows the temporal

run for widely

we believe

configurntdon

Addition

workst,ations

primar)

would elim-

Instead,

and

t,o a

We rule out as the

on the performance

requirements

for

suggest,ed

equivalent,

two clusters.

have

and

measures

the rule does not

to the fact that

like powers-of-two

effect..

The

l:% rule of thumb

pool.

effect

This

pools can pro-

25 (Berkeley)

processors.

the

we conclude

these

configurations.

of available

as t,he exemplar ations,

t,hese results.

in the sequential

workstation significant

over the period grams

ma-

of time-slicing.

refers

the avpossi-

is due to (1) the limited

effect

machines.

measures:

as using a large quantum

(2) difference

Figure

In

b>

to avoid

scheduling

of the effects

fect

numbers”

a

implied

for week-long

was computed

for the other

in the

the difference

the number

scheduling

response

the results

that

able

our goal

as possible.

match

on the application

The

in

equivalent

of 29 (College

workstation

difference

of a single

the comple-

variations

median

et al [2] for the parallel

or a very

as

the

From

dedicated

pool

configuration

from a

t,irne.

and to U-

machine

applications

idle workstations

can run only on certain

of two seconds.

was t,o use as simple t,heir study,

running

a settling

for

adapts

empirically

due to outliers.

Berkeley

inate

at

startup

We used 64 ms/MB

measure

cause of the difference

for the initialization

copy cost which we obt,ained

median

(Madison)

match

cost

congestion process

The

machine

harvesting

the

copy cost

an application

the memory simulator

startup

the memory

both ends. the time on the wire and end-point

t.ime. Thereafter

this

parallel

of the different and

by Arpaci

cost in two parts:

of the process

the equivalent

two aggregate

non-dedicat,ed

15 MB/s-per-

the eviction

cost that consists

motion

that

throughput

We have computed

the

commu-

for one week of simulated long-term

equivalent

92/109

the process

of recomputing

for sequent.ial

of time-of-day/day-of-week

vide the equivalent

and

who have had

first-come-first-served

runs.

that

That

a simple

be the

worksta-

Livny and the

of Wisconsin

erage

ble bias

adaptastate

should

non-dedicated

usage.

1 presents

chine.

to

requires

First,

a point-to-point,

a fixed eviction

the data

up a

is used.

assumes

link interconnect.

to note.

processor

In our study,

technique

running;

cost is not specific

the effect

the performance

are not avail-

can be met, by

Results

Table

and text

that, are already (this

3.3

the program

We take the

policy.

us to study

workstation

copies

scratch

allowed

derstand

[28] and

response

idle workstations

we assume

scheduling

This

of

synchrony,

in utilizing

We ran our experiments

an executing

Just

success

and t,est.ing

jobs.

throughput

that, utilize

at, the IJniversit,y

In our study

batch

the

has to pay the cost of starting

new process

of the code

stop

(as in Lo-

(as in Accent

alternative.

need not, be moved.

the need for interact,ive

group

run

that interac-

for debugging

and that

t,hrre

jobs

on this and on our own

runs are batch

cluster

Fig-

for the

of the short

we speculate

desired

long-

from threr

of short-lived

In doing so, we follow the lead of Miron

jobs.

to disk and

traces

or less)

Based

the need

and San Diego).

90%

machines,

is usually

goal of schemes

Condor

process

run multiple

are usually

that

can

processors.

or less.

jobs how

we analyzed

distribution

over

most production

position

work-

program

and a small

involve

applications

which

for t,he process

in a part

purposes;

sets

than

to memory

the rest in lazily

Sprit.e [8]). All of these schemes

with parallel

more

[24]) or it could

copy the stack

pages over and fault

otherwise,

data

of a single

an SPMD

(as in Condor

or it could

experience

tions.

ways in which

in the number

processors

primary

and copy it over from memory

data

large

Maui

usage

cases,

on sixteen

that

it. elsewhere

data

very

(Cornell,

parallel machines.

understand jobs,

run for two minutes

In all three

a small dedicated

process

process.

the

that

(jobs

traces.

with no more than 128 MB

restart

same

jobs

except

we obtained

fit in the memory

it, could checkpoint

cus [20]),

centers

the processor

on any of the machines.

are many

example,

supercomputer

the size of the

To better

from parallel

ure 6 shows

above.

parallel

to a year) job execution

tive performance have

experiments

(six months

of short-lived hlost

mode.

response

cited

for all benchmarks

benchmarks

trace.

We obtained

by the size of its

two programs,

and did not, perform

adapt

term

number

arrival

run in a batch

for interactive

For the

publications.

workstations

There

process

We obtained

We assumed

this amount

iteration.

publications

the size of each

of t,he program

ever.

machine.

are iterative.

of a large

in the CM-.5 job

interpolat.ion

the speed of execution

each

from

partit.ion

Many

parallel

we characterize

the prrsence

If the rsecut.iou linear

used in this study

of this study.

by the time

powers-of-t,wo).

Beside

pro-

it is not possible

We have

the impact

are sharp

in the performance the benchmark selected

the obvious

diurnal

of the limited

changes

t.o

nas-bt vari-

configo-

in performance

as

Figure

6:

Processor

and Dee 2 1995; based

on jobs

(Maui)

usage

distribution

the Maui

executed

and 14822

results

Average

on jobs

jobs.

The

executed

Cornell

bet,ween

results

Jan

are based

proc on

startup

Madison

Berkeley

Park

proc avail

39

52

277

34

32

169

time is fixed portion

the cost of initiating

(if need be),

communication

schedules.

ular adaptation

mechanism

how the equivalent

par mc

1

Median

par mc

1

quite

chine

over one week.

The

to be two seconds. ratio

The

of the equivalent

equivalent

process

startup

fraction

parallel

parallel

time

ecution

is assumed

in the parentheses machine

of nas-mg

ma-

and the size of the

that

availability

crosses

of all our benchmarks,

nas-bt

on the maximum number

number

of processors).

ence in the nature and the graph

of the graphs whereas

mostly

The jaggedness

station

availability

often

and forced

switches

thick

indicates

that

band

the program

st.ead,

when

was available.

different

The

configurations.

availability

graph

of intensive in Figure

impact

of processors

plentiful

and

cessors

configurations.

In-

of the graph

cannot

for

While

use (see the corre-

tral

the

In the experiments process

startup

time

described

above,

was two seconds.

we assumed Recall

that

that

parts

dip.

The

in this

process

231

The

gradat,ion

for

in the t,o

events.

with different

the Berkeley

flexibility

nas-bt,

levels of CIX-

to have

a relatively

the maxi-

small

The

number

two run on powers-of-two of nas-fftpde

of configuration

this

smaller

time

than

flexibility

that

it is

16 processors. can

apparent

are able period,

first

of pro-

processors.

is so large

it is most

two programs

number

reclamations.

runs on square

of the graph, first

we comprograms,

pool for this compar-

is likely

frequent

dataset

flexibility,

pool for three

and relatively

during

be seen

to salvage

nas-bt

in

in the CCILbeing

some more

t,owards the end since it can run on 25 processors.

On the other

the

of a single

with

the effect

successful

Impact of change in eviction cost

and about

the pools can be attributed

be run on configurations

computation

3.4

The tot.al ex-

the performance

of reclamation

for situations

and the other

several

3).

costs.

and nas-fftpde

programs,

However,

a replacement

time.

of configuration

We selected

as configuration

mum

of these

away,

is largest,

for umd being

processor

As a result,

across

the effect

nas-lu

The

were

was taken

ison

numbers”

performance

drop in the performance

on a single

by startup

difference

figurability.

work-

and

(2) the relat,ive

applications

it runs for a very short

the performance

nas-bt,

for

lies

nas-bt

the

pools;

for the other

and

Impact of configuration flexibility

pared

consists

that

“magic

deep dip in the middle

to a period

graphs

for uisc

around

workstations

a workstation

The

indicates

did not have to change

ucb corresponds sponding

that

hand.

(1)

for all three

in t.he frequency

To examine

to note is the differ-

the graph

hovered

between

3.5

for umd and ucb on one hand

on the other

band.

Note can run

for dsmc3d,

by ucb and nmd; the drops

is swamped

performance

(it runs on square

point

umd and ucb are jagged of a thick

thresholds.

is the one that

of configurations Another

for aisc

certain

nas-lu

four applications

two observations:

on 16 processors.

differences

the workstation

the curves

8

pro-

the performance

nas-bt.

for the other

time is 228 seconds

nas-mg

pool.

we plot

- dsmc3d,

cause for the sharp

is that

19 seconds

is the

graph,

Figure with

small.

The primary per-application

varies

The performance

followed

out exper-

machine

nas-mg.

for uisc.

the impact

we repeated

parallel

In each

drop in the performance

29 (0.74)1 25 (0.48)1 92 (0.33) 29 (0.74)1 25 (0.48)1 109 (0.39)

up the

on the part,ic-

costs.

for four applications

drops sharply

13651

It includes

startup

achieved

We make

cost.

To determine

shows

for nas-mg

Diego).

cost depends

used.

of process

nas-lu.

Average

This

for a wide range

between

18 are

and the cost of recomputing

iments

cost.

Jun

results

the cost of starting

cost on the performance,

approximately

Average

between Diego

of the eviction

the adaptation,

a new process

cess startup

1:

executed

and the San

Jan 1 and Dee 31. 1995. The total number of short-lived jobs are 53015 (San jobs per day is 145, 56 and 88 respectively. Th e average number of short-lived

of eviction

Table

on jobs

1 and Aug 31, 1996;

between

(Cornell).

College Average

for short-lived

are based

period.

hand,

nas-fftpde

makes

We would like t,o point

virtually out that

no progress the period

35

35 iE

30

E

25 20

E

15

!

25

d e

20

E

15

2

1 lu" J

30

f

f 2 x

it

a w

10

0

Qmm 2ciml Tme(smxkl

Figure

10

0’ 0

6cmnl

7: Variation

of equivalent

2m

parallel

4oml Time(semrck)

machine

0

Emxu

over a week. nas-bt

2mxJ 4m Tme(semdr)

mm

was used as the exemplar.

120

ie

10

i $ io5 UI

\\ ' ..

s-

-.

0

50

IOGxlLio~:w,~)

(a) College

40 I

60

----. _---.__

’ 10

’ 1 20 30 40 c&c4 pfocas slalllp (d)

2 50

’ w

"..,, "... "..

@J t

8: Variation

of the equivalent

: z Y E % a a

30

t :

30

25

2 2

E 3 3 ii

15

n&u m-ng

"‘.Y...\, “'--.,; ...._. _

.-. _

-.

/ c

JIII>,-...;... .~~..~~ .__. 1 0

20 30 10cc6ldpc-slarnp,~~

50

60

(c) Madison machine

with process

startup

cost.

40

40,

nar-u-

I

35

20

parallel

2:

@I

(b) Berkeley

1

I

35

01 0

Park Figure

‘1.

5-

---%____

0

‘\.\

E a = c x ‘c d s s

100

Me-1

35

1

ie

30

25

!

25

2 a

20

f e 8

20

E

15

t E

15

10

4 s

10

2 a w

10

0

5

5

5

I 0

m

4cmu Tme(mmis)

0 6oim

0

Figure

2mm

9: Impact

4Lxm Tme(seco&)

of configuration

232

mm

0

flexibility.

2mm

400000 Tme(sem&)

6olcinl

in question

4

is of the order

of two days

Num processors

Evaluation on a real machine

To gain arise

an understanding

when

fashion,

to detect

changes

changes.

of a parallel measured

workstations

above

fluid dynamics

(called

which keeps track process

Finch)

in workstation machine.

central

The

The

and lives

cm the submitting

machine.

Global

are made by the central

application by the

processes

and provide

requests

and to reclaim

(reclamation

or otherwise)

which selects

to the event. manager

which

allocation

coordinates

the response.

Unix environments.

Currently,

this

multiblock solves

study,

we used

computational

the thin-layer

face (multiblock program,

the

for eviction. cesses

TLNS3D

is at any other this

point,

point.

loop

request

is received

eviction

is delayed

As described

the additional

delay

is quite

We used the Adaptive

small.

introduced,

brary

[IO] for parallelizing

forms

the data

mal data

of the schedule. in providing IBM

Research

efficient

communication

ber or the identity

of its processors Multiblock

services.

provides

requires

The

similar

DRMS

would like to make is that

this support

implemented

programmer.

We needed

by a parallel

to make four changes

it to run in an adaptive to initialization

code

fashion.

which

includes

li-

proceswith

per iteration

for

to allow

us to compute

of the running

is required.

is, the time

a user

she has made

the adaptive

Second,

the

has to wait

a reclamation

time

for her

request.

We

The first part consists

by the application

(the time to repartition,

as well as compute

the new communication

part consists

and the

the equivalent

3 shows

parallel

the slowdown

to the original

of this experiment,

non-adaptive

since

The

version version.

the workstation

the

version

check

if there

of the For the

pool was assumed

was required.

of using the adaptive

is understandable

by the Finally,

machine.

of adaptive

and no adaptation

the overhead

of time spent

application-manager.

We note

is negligible.

for an adaptation

is a pending

rest of the adaptation

message

code is not used if

are no adaptations. 4 presents different

the

application-level

configurations.

to the magnitude

cost

The

cost

of the change

of adapting

is roughly

pro-

in the number

and the size of the data

partition

10 shows the equivalent

parallel

owned

of

by each

processor. Figure

two and four copies these

experiments,

and others

we

many become

a call

the central

233

of the program the

first

follow in sequence.

nodes

compete

to allow

we added

contacting

no adaptation That

processors

does not have to be

First,

the cost

and the second

portional

needed

from the College

We ran this program

the time

our experiments First,

coordinator

between

[16] from

to the program

2 shows

this time into two parts.

Table

the nor-

appli-

workstation

for our experiments

once

there

per-

point

when

on a socket.

is not unique The

of this

from one to sixt,een

datasets

event is no more than checking

the num-

system

things,

and this

as the

traces

workload.

have divided

This

recomputation

functionality.

other

the completion

SP-2

workstation

that

this library

PART1

among

we added

configurations.

to be quiescent

as well as

Changing

and a Third.

the data

Finally,

of Finch

IBM

availability

Table

measures.

period

program,

motion

communication,

schedules.

Adaptive

these

execution

which, about

configurations

code compared

section,

It also manages

nodes.

the performance

meshes.

Table

the

PART1

routine

a 16-processor

we computed

point

This library

as well as the data

To achieve

pre-computes

for this

events

succeeds.

which repartitions

coordinator

We used two input

central

till all pro-

in this

we added code to the top

if the check

routine

pool as the sequential

schedules)

a

timestep.

Multiblock

for normal

for adaptation.

communication

for eviction.

at least

the application.

partitioning

the repartitioning

later

using

move the data

SPMD

when

the central

of the time spent

Al-

that

as the safe

The

for the period

for adaptation

routine

the adaptation

for eviction.

over a 3D sur-

to a different

time-step

Second,

to check

to a finalization

version

is portable

application

loop

We designed

application-

is an iterative

corresponds

top of the

If a reclamation

reach

This

16

version.

to be unused

a call

three

must respond

equations

to the non-adaptive

for resources.

the different

User

from

relative

pool was assumed

and moves it to destination

different

a co-

extracted

dynamics

[27]).

each iteration

We chose program

fluid

Navier-Stokes

0.4%

arrays

sors.

is done

it runs on Suns,

a template

0.1%

in powers-of-two

of

phas and RS/6000s. For

0.1%

3: Slowdown

Park

de-

use.

Finch

0.1%

pool and 16 workstation

to the central

that

0.1%

We evaluated

a pair of programs

the corresponding

2

cation

the job

coordination

are sent

8

program.

It runs

it for personal

dat,aset

informs

can use to make the

the application

It then informs

when

we assume

user of the workstation

available

coordinator

resem-

of adaptation

user environment

workstation

across

for the purpose

Currently,

that the primary

of its

of the job.

application-manager.

operative

and a mantrack

coordinator

resource

0.5%

we wrote

to changes

coordinator;

0.1%

call to an adaptation

response

is created

0.1%8

of the time-step

[24] and runs on a central

for the duration

0.1%

coordinator

of

traces

coordinator

keeps

central

manager

application-manager

is submitted cisions

which

the application’s

availability.

bles the Condor

availability

a central

1

of this experiment.

and have

as a cluster

availability

application

and coordinates

to

version

program

0.1%

workstation

workload.

uses

of the workstation

for each

and adapt

SP-2

Table

allows pro-

an adaptive

using an IBM

as the sequential

system

progress

environment

and one of the workstation

mentioned

ager

in their

that

4

1

that

in an adaptive

support

We have also developed

computational

problems

programs

s,ystem

its performance

Our

practical

to run parallel

we have developed

grams these

trying

of the

2

dat,aset

as it wants

for the available

In

is allowed

to start

first.

first

time

nodes

copy is assigned

and the other

and for

the

as

copies

nodes

that,

the computation.

As a result.

the

the others.

The

first copy achieves

better

performance

largest

parallel

machine

equivalent

for one,

together.

The

at start

remaining during

copy

machine

running

than

is 11 processors

for the

Num procrssors

1

2

4

8

16

dataset.

I

319 ms

196 ms

134 ms

106 ms

87 ms

dataset

2

510 ms

380 ms

209 ms

150 ms

130 ms

Table

2: Time

1

1

1

1

2

2

2

4

4

8

Num of dest proc

2

4

8

16

4

8

16

8

16

16

125 ms

188 ms

214 ms

250 ms

62 ms

93 ms

115 ms

28 ms

48 ms

19 ms

time

Table

4: Application-level

first dataset

and 13 processors

for the second

corresponds

to 69%

of the size of the

comparison,

the equivalent

set of umd traces

and 81%

parallel

was computed

average

time

machine

size of data

the guest

wait time grams

for the job

running

for eviction

running

was 1.67 seconds. period

the average

The number

of this experiment

6

pro-

that

the average

time

of adaptation

pro-

are

two

there

tions

for parallel

erable

variance

achieve

over the

pool.

between

In this paper, and

we considered

servers.

With

the use of idle workstations

the current

the size of data-intensive

stations

for

Dahlin

their

memory

et al [‘i] study

to increase scribe

a low-level

virtual

memory

to back

to use idle memory

suite

of sequential

Franklin scheme

client-server

database

cation servers

buffer

have

been

Iftode et al [13]. similar track jects

system.

by introducing

them

for their

Condor

machines guest

which

memory

describe

performance,

have large

primary

advantage

of t,he

This

effect

of workstations

like powers-of-t,wo

are:

tions

server

though these

to thirty

imposes

large

larger

half the total

is a wide of idle

can

The

im-

minimum

expected

for today’s

the

Eviction

of

minutes

as the

minutes.

234

than

for over half the time.

fraction

(20%-70%)

size

More-

of the worksta-

clusters

are available

are not stable.

at any given

Clusters

up to half

for four to fifteen

minutes for five

minutes.

length

memories.

a

drops more or less

up to a third of the pool are stable

There

particularly

of

for which

available.

clusters

and clusters

harvesting

memory

Even time,

workstations

of time

is available

the size of the pool are stable

layer.

significant

is always

of the

fraction

with k. Clusters

over, a substantial

of multicomputers

pages does not have the same urgency

of the parallel

throughput.

to 80% The

of I; workstations

[18] and

et al

without

by a

that can run on a large

numbers”

of the pool are available

as

Iftode

If done properly,

60%

memory

as needed.

than

better

of our study

average,

linearly

It keeps

memory

conclusions

cluster

in a

machines

support

other

memory

ob-

server

to take

jobs

“magic

On the

memory

for their

achieved

when the number

a pool are available.

and ships

for long periods

The

is

available

hierarchy

achieve

between

a memory

manager.

throughput

are less able

around

by

Navier-Stokes

and squares.

goal was to avoid repli-

central

memory

computation.

pact on the interactive

a unified

is provided

an adaptive

that can run only on a small num-

important

hovers

could

parallel

the size of the

also on the flexibility

Jobs

they

of a

and all the clients

by Narten&Yavagkar

on the system

be shared

by a factor

servers.

idle workstations

fewer requirements

scheme

performance

Explicit

the memory

a remote

Harvesting

that

this

available

that all of

pools of all the clients

to the corresponding extending

system

of the

proposed to the

set of configurations

that

of a dedicated

not only on the characteristics

in availability;

is particularly

de-

the buffer

pools

of the idle memory

be often

Their

changes

idle memory

but

et al [12] describe

dynamic

option.

file pages the

tasks

Narten&Yavagkar

in spirit

propose

show

for the servers

of pages between

well as the

They

to improve

load but

et al [ll]

management

data-intensive

1.5 and 3.5.

Feely

the parallel

consid-

For the three

to three-fourths

pool depends

being run on it.

is, however,

we found

and

First,

idle worksta-

for this conclusion

Finch

ber of configurations

idle work-

of using

up not just

as well.

able

management

exploiting

size.

jobs

study.

achieved.

to that

evidence

Second,

of sequential

as

in the number

be an attractive

file cache

global

memory

tasks, could

the feasibility

the effective

uses idle memory

growth

equal

with

non-dedicated compute

There

we studied,

one-third

Supporting

template.

Other Related work

of our

in harvesting

computation.

pools

our experience

5

conclusions utility

in the performance

performance

machine

was 487.

primary

is significant

non-dedicated

for eviction

events

processes.

Summary of conclusions

There

and the

For multiple

wait

of guest

1).

3.3).

was a part of. For

on the pool,

was 1.19 seconds.

together,

eviction

(dataset

For

for the entire

of processors

process

by itself

cost of adaptation

That

pool.

the user had to wait for a guest on the number

program

data set.

to be 74% (see section

cess to leave depended a single

for the two datasets.

Num of src proc Remap

The

per iteration

variance

periods length

distribution

different

workstation

can be expected

to a maximum that

has

of the

workstations.

of an idle period

of 18 minutes

average,

in the

across

varied

from

of 9 hours.

been

idle

to be idle for another

a On

for five 40-90

Figure

10: Equivalent

graph

on the-right.

parallel

It, is not t,oo difficult

l

run in an adaptive benign.

That

adverse

impact

Also,

is,

large.

However,

graph

SPMD

programs

to

Workstations.

This

conversion

is

METRICS

the

modifications

do not

have

an

ment

of the programs.

we would

on real machines.

like to caution is unable

that

In Proceedings

ing’94,

from

[4] N.

pages

gramming,

We would like to thank

Remzi

Arpaci

traces.

We would like to thank

Condor

group

at the University

the web interface like to thank

of Wisconsin

to the Condor

St,even

Hotovy

status

and George

puting

Center

Intel

for the trace

Paragon,

mance

and Peter

Computing

to their

Kremenek

IBM

center

of the jobs

Young

for the trace

SP-2,

Diego Maui

of the jobs

Regan

November

1994.

D.

Gelernter,

M.

Piranha

Condor

Condor

status

R.C.

Caching:

prove

Perfor-

First

submitted

File

System

Symposium

Operating

mark.

vision, F.G.

Algorithm In

Gustavson,

Proceedings

Performance

and M. Zubair.

for the 3-D FFT

NAS Parallel

of SHPCC’94

Computing

Conference),

An Ef-

puter

Bench-

1990.

(Scalable pages

High-

[9] Fred

129-33,

T.E.

Arpaci, Anderson,

of Parallel

A.D.

Dusseau,

and D.A.

and Sequential

A.M.

Patterson. Workloads

Vahdat, The

L.T.

21(8):757-85,

Liu,

Interaction

on a Network

Douglis

cess Migration: Implementation.

May 1994. [23 R.H.

Sciences,

[lo]

of

G. Edjlali, Parallel

235

1996.

Using

267-80,

PhD

G. Agrawal,

to Imof the and Im-

Nov 1994. Migration

thesis,

Computer

in the Sprite Science

Engineering

of California,

Alternatives

Software August

Design

System

Ousterhout.

Design

Memory

In Proceedings

Process

University

and John

and D. Patterson.

Remote

of Electrical

Programming

Pro-

http://m~w.cs.~isc.edu/-

T. Anderson,

Transparent

Department

of Parallel

1996.

on Operating

System.

and

and Their

http://mmu~.cs.u~isc.erlu/-

Performance.

pages

plementation,

SP-2.

Agarwal,

Jourdenais, Strategies

Journal

monitor.

R. Wang,

Analysis

of Supercomput-

Feb 1995.

monitor.

Cooperative

References

ficient

International

status

[8] F. Douglis.

[l]

Scheduling:

23( 1):5-33,

[7] M. Dahlin,

to their

High

In Proceedings

cgi-bin/condor-status/-server+fo&

Supercom-

submitted

of the

[6] The

Center

%67-

A Scalable

for Fluid-flow

cgi-bin/condorstatus/-sewer,

We would

Theory

to their IBM

of the San

[S] The

the

for providing

monitor.

of the Cornell

for the trace of the jobs submitted Moore

for the UC Berkeley

and W. Smith.

8-17,

Implementation.

workstation

SIG-

pages

Systems,

Environment Grids.

Carriero.

D. Kaminsky.

Acknowledgments

ACM

on Measure-

Conference

of Computer

T. Tysinger,

on Unstructured

failures.

availability

and the

1995.

High-performance

does no

to recover

of the 1995

International

and Modeling

[3] D. Bannerjee,

the reader

is based on a scheme

and as such

on the left is for the first dataset

Joint

78, May

seen by a user is not unacceptably

that this conclusion

The

to convert

on the performance

delay

checkpointing

for one, two and four programs.

dataset.

environment.

useful gains are possible

The eviction

l

machine

is for the second

Practice

Berkeley,

Transparent and and

Di-

and Com-

the

Sep

ProSprite

Experienct,

1991. A. Sussman, in an Adaptive

and J. Saltz. Environment.

Data In

Proceedings

of the Ncnth International pages

ing Symposium, [ll]

M. Feely.

W.

Morgan,

and C. Thekkath. agement

201-12,

[12] M. Franklin,

M. Carey,

of the eighteenth

In 538-47,

[14] S. Leutenegger tributed pages

[15] M. Litzkow

97-101,

and

[17]

IBM

Research,

Matt

Mutka

Naik,

Narten

Livny.

pages 5-12, [20] G. Popek

Workshop cessing,

November

and

[23] S.

The

M. Livny. with

A. Woo,

Ames

Sharma,

Memory

Operating

as a of

Systems,

Research

of Supercomputing’94,

Proceedings

on

Operating

Press,

of

the

Systems,

Processing

Center,

Irregular pages

of the

for Parallel

Pro-

1995.

Technical

Runtime

Sys-

on Dy-

In Proceedings

Strategies

April

Distributed

1985.

Parallel

Ponnuswami,

for Adaptive

in a Shared

In

and M. Yarrow.

R. Das, and J. Saltz. port

Per-

1991.

In Proceedings

Workstations

CARMI.

2.1 Results.

R.

July

The LOCUS

MIT

on Job Scheduling

Benchmarks

Capacity

1987.

pages 259-78,

[22] W. Saphir, NASA

Idle

Symposium

Resources

Available

Systems.

and B. Walker.

Pruyne

20239,

1992.

tem Architecture.

namic

RC

Environment.

Remote

Yavagkar.

Environment. ACM

The

12(4):269-84,

Using

Nichols.

Eleventh

Allocation

Report

on Workstation

April

Computing

[21] J.

Technical

Workstation

Workshop

Vatsa,

The Report

August B.

NAS

1996.

Moon,

Problems. 97-108,

Parallel

NAS-96-010,

Y-S

and Compile-time

Hwang, Sup-

In Proceedings November

In Proceedings processing

March M.D.

Idle Sys-

Engineering,

and M.R.

Strayer.

Equations: of the Sixth

for

Parallel SIAM

scientific

con-

computing,

1993.

Sanetrik,

and E.B.

AIAA-93-0677.

Flow Solver; Aerospace

Software

C. Bottcher,

block

Sciences

1994.

236

The

Meeting

Parlette.

Develop-

Multigrid-Based

Multi-

In Proceedings and Exhibit,

Use of Copy-on-Reference

System.

Science,

PA, April

pages

A Program-

Resource

Owned

R.

26-30,

on

for Hydrodynamic

on parallel

Finding Distributed

1989.

and Efficient

puter

of the IEEE Systems,

Konuru.

and Miron

and

132-6,

R.

Dynamic

in Distributed

the third

[19] David

and

for

Evaluation,

Resource pages

Distributed

D. J. Dean,

methods

Migration

with the Condor

In Proceedings

Dis-

of the January

1993.

May 1996.

of a Privately

[18] T.

Dis-

Condor

Dobbs * Journal.

Lantz.

ment of a Flexible 313t

1990. V.

Umar,

[28] E. Zayas.

Experiences

Distribution.

formance

Comput-

1993.

System.

Environment Data

Distributed

Homogeneous

on Experimental

[16] 3. Moreira, ming

Sun.

of Supercomputing’93,

November

act

[27] V.N.

of

Keith

November

pages

The Dr.

Workstation-based

Transactions

ference

Digest

Spring’93

and

a

Implementation.

for

In Proceedings

Batch

Workshop

Servers

in IEEE

Spline

In

Litzkow.

15(11):1444-57, [26] A.S.

Memory

Feb 1993.

and M. Livny.

Distributed

tem.

in a Non-dedicated

System. 143-52,

Prin-

Aug 1992.

Memory

COMPCOIV

and X.-H.

ing Feasibility

Machines

Conference

596-609,

K. Li, and K. Petersen. pages

Global

Theimer

of the

Architectures.

International pages

Data Bases,

Multicomputers. Papers,

DBMS

M.

System.

Feb 1995.

[25] Marvin

Man-

System

and M. Livny.

Proceedings

[13] L. Iftode,

Memory

and

Processing

20(2):42-4,

H. Levy,

In Proceedings

on Operating

in Client-Server

Tannenbaum

tributed

Dee 1995.

Management on Very Large

Global

Cluster.

[24] T.

Process-

1995.

A. Karlin.

Implementing Symposium

ACM pages

Parallel

April

F. Pighin.

in a Workstation

fifteenth ciples,

827-32.

1987.

PhD

Carnegie

thesis, Mellon

in a Process

Department University,

of ComPittsburgh