Optimistic Parallelization of Communicating Sequential Processes

2 downloads 32323 Views 1MB Size Report
University of California,. Berkeley and. IBM. T.J.. Watson. Research. Center. Robert ... of whether the line was displayed successfully. Process X repeatedly calls.
Optimistic David

Parallelization

F. Bacon,

University Robert

of Communicating

of California, E. Strom,

Berkeley

IBM

T.J.

dfb(lcs.berkeley.

edu,

Abstract

and

IBM

Watson

Research

non-interference

using the framework

for managing communications,

probability, anteed. cation

and aborting

program.

and process

PutLine,

of guarded

copying

Machinery.

without

fee

and

sequential

specific

e 1991

ACM

lines until

calls all out-

delays

are long

relative

such an implementation

X cannot

be parallelized

is inefficient.

by conventional

of the dependency code from

to the speed of com-

between

the previous

Process

static

analysis

each call and the

call.

processes

otherwise,

of tha

code.

If the continuation the process

makes fur-

is once again

split,

with the effect that a series of two-way calls is converted into a series of one-way sends. This transformation is termed call streaming. It is extremely valuable when bandwidth

of

this

mate[rial

is high but round-trip

delays are long.

Every optimistic protocol relies on a guess. In the case of call streaming, the guess is that the first thread’s call (1) will be serviced before the second thread’s call, and (2) will return auccesafully. Some systems, such as

ie

appear,

and

Association

or to republish,

notice

is given

the X-window

for Computing requires

formance

a fee

permission. 0-89791-390-6/91

return

calls to PutLine,

if not, they must be

all or part

its date

ie by permission

and/or

the line

X repeatedly

We contrast our

that the copiee are not made or distributed for advantage, the ACM oopyright notice and the

To copy

Y

threads. The first thread makes the call and waita for the return, while the second executes the continuation after the call, assuming that the call completed with a

are run in parallel;

publication

of whether

Process

putation,

return

run sequentially. granted provided direct commercial

Process

It takes a single line

an indication

successfully.

guar-

is an appli-

manager.

PutLine.

passing it successive output

because

has been extensively studied. Existing approaches use static analysis [2, 5] to prove that two sequential code segments do not interfere. If this can be proven,, then

copy

Y a window

X

We can improve performance by transforming process X into an optimistic execution containing two parallel

of how to parallelize

to

be statically

process

syshigh

put has been delivered or until it receives an unsuccessful return code. In a straightforward implementation, process X waits after each call for a return from Y before making the next call. If Y is remote, and commu-

ther

segments

it cannot

suppose

a service named

successful The problem

to prove.

even though

was displayed

thread creation, committing,

in the transformed

difficult

per call and returns

Introduction

of the

Center

For instance,

exports

approach with related work.

that

Center

However, many circumstances arise in distributed tems where non-interference can be guessed with

cornputatiow, in which each computation is tagged with the set of guesses on which it depends. We present an algorithm

title

Research

[email protected]

nication We analyze the problem

Permission

Watson

processes not known at compile-time, and because the non-determinism introduced by communication makes

Such a transformation is valuable in the case where S1 and SZ cannot be para.llelized by static analysis either because S2 reads a value from S1 or because S1 and S2 each interact with an external process. The optimistic transformation works under a weaker set of conditions: (1) if the value Sz reads from S1 can usually, but not always, be correctly guessed ahead of time, and (2) if S1 and S2 interact with an external process, conflicts which violate the ordering of S1 and % are possible but rare. Practical applications of this approach include executing the likely outcome of a test in pwdlel with making the test, and converting sequences of calls into streams of asynchronous sends.

b&h

T.J.

Processes

This technique is rarely applicable to a process in a distributed system because it may communicate with other

We present a transparent program transformation which converts a sequential execution of S1; SZ by a procesl; in a multiprocess environment into an optimistic parallel execution of SI and S2.

1

Sequential

/0004/0155...$1

.50

155

protocol,

by providing

trade

off correctness

an asynchronous

for per-

send-baaed

in-

terface, and requiring the user to handle asynchronous notification of errors. In contrast, in our approach we abort the second thread and undo all its side effects if ei-

ever, the processes are compiled and instantiated separately. This means that there is no a pm”ori ordering of events between processes. Instead, there is a partial or-

ther of our guesses was wrong.

This way our results

correct

or wrong,

der of events between processes which is dynamically determined by their communication. These assumptions

whether

we usually

we guess right

guess right,

we still

obtain

are

and provided

are typical

a performance

of distributed

systems.

improvement. This

paper

presents

an optimistic

protocol

Wlthln ments

for trans-

a process there is a sequence of program segS1; S2. Both S1 and S2 may include interac-

tions with other processes in the system. We assume the that there is some mechanism by which the compiler is told that it is desirable to parallelize S1 and S2. This mechanism could be programmer supplied pragmas, run-time profiling, static analysis, or a combina-

forming sequential code into parallel code. We use the call streaming transformation as a running example. However, our approach is applicable to parallelizing arbitrary sequential constructs S1; S2 in the case where

tion of these methods. ●

S2 depends upon S1. Static analysis will indicate that the two process fragments are not parallelizable,

either

because

for

some execution

path

We further assume that either S2 uses no values defined without in S1 (the only ‘guess” is that S1 terminates

S2

reads a value from S1, or else because S1 and S2 each interact with some external process, such as a server in a distributed ●

what

system.

with

S2), or that

of S1 with

collection

Transformations

[10] and is similar

ery [9]: we guess that

conflicts

of a process

occur.

will

not

When

Recov-

We illustrate

serial segments

a conflict

complex

occurs,

Formulation

and

to a database

that

a somewhat

than

more

we gave in the

server (process

by calling

process

Y),

which

actually

Z, the network

writes

filesystem

OK is usually

true.

Figure 1 illustrates the code of the original program segment. Figure 2 illustrates a typical execution in an unoptimized implementation, using a time-line diagram.

Hermes [8]. A feasible target environment is the Mach operating system [1], which supports both parallelism and distribution. program.

with

of call streaming

server. S2 then makes a direct call to Write to the filesystem server. The only value passed from S1 to S2 is the variable OK, which indicates whether the call successfully updated the database. The compiler has been told that it is desirable to parallelize S1 and S2, and

Example

as a sequential

our transformation

example

the value

We assume a high-level model in which independent sequential processes communicate by message passing or by making inter-process calls, as in CSP [3], Ada [12], or

Each process is written

el + ez.

introduction. Suppose that process X is a client containing two segments S1 and S2. S1 is an Update call

we roll back the processes that depend on the incorrect guess and restart them, running the computation in the original serial order.

2

is a

on Optimistic

to Optimistic

between

A trace

In this implementation,

events. Specifically, if el is an event in the execution of S1 and e2 is an event in the execution of S2, then we must preserve

work

program.

events.

by our mechanism. We preserve both the data values of each input and output event, and the ‘happens before” relationship [7], which we denote by ‘+n, between these

the guess is incorrect. If a bad guess is made, the program still runs correctly, but the average performance will be worse because of excessive rollbacks. on earlier

of the original

of observable

the observable events are the messages sent and received by all computations except those which are “aborted”

cution speed. The transformed computation completes earlier whenever the guess is correct, and later whenever

is baaed

S1 to S2.

Our transformation generates a parallel execution of S1 and S2. To be correct, our implementation must pre-

of success.

serve the traces

solution

has been told

in S1 and used in S2.

a high

Our program transformation is transparent. That is, the programmer makes no change to the source code, and sees no change in the program behavior, except for exe-

Our

the compiler

to guess for values defined

These values are said to be passed from

There is a way to guess the result probability

interfering

There First,

How-

156

are three

obstacles

to parallelizing

there is a data dependency

S1 and

on the boolean

S2:

value

x

/*

S1

*/

/*

S2

*/

OK = Update(Item,Value); if OK Urite(File,’’Di.d

Figurel:

OK between

Program

analysis

Segment

S1 and S2.

Z are independent

call

Y */

/*

call

z */

processes

another-and

X,

I!,

and

inter-procedural

calls is not possible.

on communication

Finally,

delays, X’s call to Z could

arrive before Y’s call to Z, which happens-before relationship.

clearly

Wewillovercome

trackingdependen-

these problemsby

violates

the

ciesdynamically, byexploiting theguessthat OKistrue, and by guessing that the messages arrive at Z in the correct order. X into

Given

these assumptions,

two parallel

threads:

Figure 2: No Call Streaming. Processes X, Y, and Z communicate via calls and returns (labeled C and R).

we can transform

one to execute

end of S1, and one to execute

up to the

Sa and its continuation.

3 Figure

3 illustrates

a successful

(non-aborting)

pendency tracking and will happens-before relationship logical

ordering

of events within

ordering

the dotted

between

line,

and

original

events.

The example above provides the motivation for optimistic parallelization and gives a rough idea of our approach. We now provide the theoretical framework in which we solve the problem and show how we use this framework to create an optimistic algorithm.

each thread,

(2) the

depicted

ordering

The execution

3.1

by

in Figure

2, and is therefore

Our

in Fig-

solution

is based on applying

the theory

of Op-

a correct

work for deriving optimistic distributed protocols from conservative protocols. In this framework, a process

If the call to Update we call this

(right-hand)

Transformations

timistic Transformations [10] to the case of time faults and value faults. Optimistic Transformations is a frame-

implementation.

at all,

Optimistic

between

the same events in the same order as the

execution

Execution

below. The by: (1) the

the two threads

(3) the causal

send events and receive ure 3 generates

be explained is determined

Optimistic

execu-

tion of the transformed process X. The two threiads are overlapped in real time, whereas they were sequential in Figure 2. The dotted arrow pointing backwards in real time indicates that the last event of the execution of S1 logically precedes the first event in the execution of S2; the labels in braces are used for dynamic de-

physical

z

Y

to Be Parallelizled

Second,

ofone

across the RPC

depending

it”);

/*

thread

The parallelized

returns

false

a value ~auli,

or fails

may choose at some point

to return

is aborted.

thread

will

also be aborted

to make a guess, and initiate

a computation baaed on this guess. For each guess, there is some stable predicate pn which confirms the guess —

and the parallelized

that is, if pn is true, then the computation Xn based on thh guess is correct. In this case, pn is said to be the

if an exe-

commit guard of z~. At any given time, the predicate ~ is either known to be true, known to be false, or in doubt. While pn is in doubt, there is a live computation to compute pn. Therefore, any commit guard predicate

cution such as the one in Figure 4 occurs. In this case, the call from X waa received by process Z before the call from Y. This contradicts the logical order in which the return from Z to Y precedes the return from Y to X, which precedes X’s call to Z, An examination of the diagram shows that there is a cycle in the ‘happens before” relationship. Such a cycle is a violation of

is eventually

causality,

(possibly

discovered

At any given time,

and is called a time fault.

157

empty)

to be either

a computation set of identifiers

true or false.

x~ is associated of in-doubt

with a

predicates

{PI, . . . . Pk). T~ commit

To prepare for rollback, a process may take a state checkpoint at each point prior to acquiring a new commit guard predicate. This approach is used in Time

This set is called the commit guard set. guard predicate of Xn is the conjunction

Ai=lpi” empty

If the commit guard set of a computation is then the commit guard predicate is vacuously

true—this

means that

the validity

Warp [4]. Alternatively, a process may take less frequent checkpoints, and log input messages, restoring the state

of the computation

by resuming

is independent of any guesses. Such a computation is said to be committed. A computation with a non-empty commit

guard

computation by rolling

set is called eventually

optimistic.

either

back to an earlier

For each in-doubt cess which

will

hood of pi. The predicate

pi, there must exist compute

the truth

is said to commit

from This

the checkpoint

approach

and replaying

is used in Optimistic

ery. A process may also rollback

optimistic

by inverting

logged Recovthe steps

of its computation. The particular technique used for rollback is a performance tuning decision and does not affect the correctness of the transformation.

or else aborts

state.

predicate eventually

An

commits

messages.

a pro-

or false-

when it re-

3.2

Algorithm

Description

solves to true, and abort when it resolves to false. When an in-doubt

predicate

computations are notified,

containing pi in their commit and must abort. If a predicate

putation’s

commit

pi is committed

guard

or aborted,

set commits,

is removed from the set. When becomes empty, the computation

all We now apply the above framework rithm for Optimistic Parallelization.

guard set in a com-

that

the commit commits.

a new commit

the union of its previous mit

guard

guards

track

set of the

guard

commit

When we run Sa without waiting for S1 to complete, are making two optimistic guesses:

set

set consisting

no value fault



no time-fault



of

ternal observers

to see possibly

fault, ... +

e. is in S2 and ek in S1. The existence chain implies that eo (in Sz ) precedes which contradicts the logical semantics

there

is

ek where of such a

ek (in S1 ), S1; S2.

These guesses constitute the commit guard predicate associated with the parallel execution of S2. If S1 completes without violating either of these guesses, then the predicate

is committe~

otherwise

it is aborted.

since we do not allow exincorrect

outputs.

When

back.

its exter-

aborts,

it discards

nal messages and rolls back.

External

input

messages

always have an empty aborted.

set, since they

are never

guard

el +

compu-

messages of rolling

a computation

For each time

commit

a computation commits, it releases its external and discards any state it created for purposes When

and passes

way,

Messages to or from processes that can not be rolled back, such as workstation displays, printers, and systems not participating in our protocol, are called ezternai messages. External messages sent by a guarded must be buffered,

occurs.

a causal chain of events eo +

tations.

computation

S1 terminates,

between

In this

dependencies

occurs:

we

the guessed values to S2, and

guard set and the com-

message.

the causal

an algo-

predicate guard

Each message carries with it a tag containing the commit guard set of the computation which sent the message. When a computation receives such a tagged message, it acquires

to create

If the predicate

is committed,

without

the possibility

commit

guard

execution

of rollback

predicates).

of S2 proceeds

(unless there are other

A control

message called

a

commit message is sent to other processes which depend upon the predicate which committed; this may enable them to commit aa well. If the predicate is aborted, S2 is rolled back and re-executed. Additionally, all its side effects are undone. A control message called an abo?’t message is sent to other processes which depend upon the predicate which aborted; this may cause further rollbacks.

To implement optimistic transformations, the compiler generates the guarded computations and the comput~ tions which check the guesses. The run-time environment encodes each guess, maintains a commit guard for each current in-doubt computation, propagates commit guards on messages exchanged between processes, dktributes control messages notifying other processes that a guess has committed or aborted, and maintains the ability to roll back state.

Provided that the overhead of our method is small and the relative frequency of aborts is sufficiently low, we will achieve a performance gain by parallelizing S1 and

S2. Let

158

So; S1; S2 be the

code of a sequential

process

X

x

) J

I

● .-’

xl

{}

)

,,.H

{}

{xl}

● ’

, : : :

: : :

C2{} —

: :::1

‘2 + E4

4

(}

,-: T

J--

: :;&

!r

B [xl)

c

{xl}

R

b

{X. 11

B

}

{Xli ):

,.”

R

I

: :



RI{)

A

A cl{

{Xl}

.,’

3

:

D)

c

Figure 3: Successful Optimistic Call Streaming. Commit guard sets are indicated next to the process time

Fimre 4: Aborted Optimistic call Streaming. ~1 is + {z 1}, indicating a-cycle in aborted at D since {xl}

lines and on the message identifiers,

the happens-before in Figure & The

that

and points

o:f in-

A, 1?, etc. The dashed line indicates

terest are labelled

A precedes Z1 in logical

relationship. The results are shown cycle includes C3, Ra, R1, and the

dashed line.

time.

The two threads the state, except

which is to be transformed by running S1 and Sa in parallel. We call the boundary between SO and S1 the fork point and the boundary between S1 and S2 the

will that

each run on a separate copy of in the right thread, we start by

setting each vi to the guessed value bi. If there is no antidependency between S1 and S2 — that is, there is

join point. Without loss of generality, does not itself contain a computation

assume that S1 which is being

no variable read by S1 and overwritten by Sz — then is unnecessary. This is the case in our call the COpy

parallelized.

passed from

streaming

Let {vi}

to S2 and let {bi} {hi}

depend

be the variables

be the values guessed for {vi}.

S1

After

only on the state at the fork point.

the fork,

parallel, When execution

reaches the fork point,

we shall label

*1.

The name Z1 will

(1) as an eveni!, Z1 denotes

the join

X have an empty commit of X is guarded by {x1}, commit

denote

asserting

that

the left

thread

will

guard

thread

in is

guard set. The right thread and when it sends C3, this

set is appended

receives C3 at point

point

to the message; when Z

B, its commit

guard

to {z1}. This guard is also sent with not change the guard at point C.

between computations S1 and Sz in process X; (2) as a commit guard predicate, z 1 denotes the commit guard predicate

of S1 and S2 proceed of Sz in the right

process receiving such messages will inherit the commit guard al. In Figure 3, Y, Z, and the left thread of

ecut es S1; the right thread executes S2. The left thread has the same commit guard set as So. The right thread has a commit guard set containing an additional predicate which

the execution

but the computation

guarded by xl. Any messages sent from the right thread will be annotated with the commit guard name z 1. Any

the transformed

program executes a fork, splitting the computation into two parallel threads. By convention, the left thread ex-

two things:

example.

The

set is changed

R3, but this does

complete

with no value fault and with no time fault. It is possible that the right thread contains further fork points, in which case there will be a right-branching forking struc-

When the left thread completes, a check is made that the expected values {bi} were assigned to {vi}. If the

ture. In general, c~ will name the n~~ join point event in X and will encode the predicate that the nth forked

The right thread and any causal dependents are rolled back immediately, and an abort message is propagated to any processes which depend upon z 1. In Figure 3,

computation fault.

will complete

without

a value fault

check fails, predicate

or time

x 1 is aborted

thb would mean rolling

159

due to a value fault.

Z back to point

B and aborting

Y 10

x ‘1

{)

~{xl}

C3{xl

.

m.,

: : .

cl{

:

4

~%-t “... -----

point D logically precedes the beginning of the right thread, we have xl -+ Zl, meaning that a time fault has occurred. Z must be rolled back to point A, Y must

: : : : .:

(xl ~ ...........

“-.. . ----(} -- . . . . . .

I

}

R2{ }

I

back to point

aborted,

aa is shown in Figure

There

\

i R3{ ) a T

is another

possibility,

case we can commit ... ......... ‘----------Figure

5: Abort

Z must re-read

the right

Normal Computation/Messages A~fied Compunction/Messages

we can abort

Control Messages

we will

and Re-execution.

Note

message C2 after rolling

thread

that

back.

namely

Zl,

message and learn

that

have detected

of the fork.

-

duration

The timeout

then Z1 can commit

is set at

guarantees

x 1 precedes

in which

case

a precedence

gm, in which

a cycle and can abort

case

z 1.

fault

due to non-termination

3.3

demonstrate

nates with

erwise live computation

set, then there

and

In any protocol

are

no uncommitted forks in its causal past, and xl can be committed. This is the case at point A in Figure 3: R1 has an empty guard set, and so the left thread terminates with an empty guard set and commits.

a cycle and aborts.

Liveness

at least one dependency which goes backwards in real time—that is, one resulting from a fork. If S1 termiguard

Z1 commits,

Figure 7 illustrates the laat case. We begin the same way, but process X later learns from process Z that Z1 precedes Z1. Thus process X detects

is

If the left thread terminates with no value fault, we check for a time fault by examining the commit guard set at the termination point. Every time fault includes

commit

Later

also.

that

predicate xl aborts in case S1 diverges. If the timeout expires, the left thread continues, but predicate x 1 is

an empty

receive

Figure 6 illustrates the first case. In this example, both process X and process Z have been optimistically parallelized. The return message to process X depends upon

process

of X.

aborted, and a value assumed.

S1’s commit

an earlier guess 21 in process Z. Process X informs other processes which depend upon guess x 1 that the

of implementation-defined

the beginning

that

Z1, (2) gm will abort, or (3) we will

Z1 guess precedes the Z1 guess. A timeout

of X is

that the event gm is in the past of S1, and that S1 can commit only if gm commits. But we do not yet know whether xl can commit. We will notify other processes that gm precedes xl by sending a new type of control message called a precedence message. Eventually, one of three things will happen: (1) gm will commit, in which

C3{}



thread

5.

guard does not include xl, but it does include some other guard gm from another process. NOW we know

*

RI{ }

B, and the right

be rolled

?..,

T

I

at Z before

propagates it to the left thread of X at point C. As a result, Z1 is in the causal past of point D. But since

xl)

i

4: because C3 arrived

C2, xl is in the guard set of R2. This causes xl to be added to the guard set of Y at point B, and RI then

..~ .,.

Xl} ‘, ... .... .. ..-.

B-

D in Figure

at point

R2{

R (Xl}

A (xl)

R3{Xl }

b

~.m . ... ... ... .. ..i...-’-”-{( D: .. . . ----

~

. .

}

.. : : ~**,,,......””**”’””* : : :

)

● ..m. mm. . . . . . . ..m...

:.......................

If the guard set at the end of the left thread is nonempty, there may be a time fault. Recall that the commit guard set includes all the uncommitted guards in the causal paat. If xl belongs to that set, then we know there is a time fault, because this means that S1’s execution causally follows the S2 thread. This is the case

that

employing

aborts,

it is not possible by rolling

it is necessary to deadlock back infinitely

to

an othoften.

Our algorithm is live provided: (1) we assume a finite number of processes in the system; (2) we impose a limit L specifying the maximum number of times the same

160

y

x ‘1

{}1

z

w

‘1

{)]

1(1

1{}

xl )

Figure

6: Successful

and returns.

Parallelization

At point

sent to X and Y, which its computation.

of Two Threads.

One-way

C the message PRECEDENCE(Z1 ,{z1}) both

commit

their

computations.

message sends (itfl is sent to Z.

Finally,

z.

(}

‘1

)

I

M1{z1)

r

:Zl) C*{}

: ,

•Z1 }

:

1!

+ :

M2{ X1)

● :

R2{

E

&--

is

then commits

()

... .. #

.“

i : : : :

to W, which

{)1

,.. : : : :

are used aa well as calls

D the message COMMIT(Z1)

X sends COMMIT($l)

x

B (Zl

and itf2)

At point

.“ ,.”

x1}

D {xl]

: ,.“

c

Fg

Fimre 7: Aborted ParaHelization of Two Threads. At point C {z1} --i {xl} and at point F {xl} left threads send PRECEDENCE messages to one another, and discover the cycle z 1 + Z1 + z 1. X to Z and W, and Z sends ABORT(Z1 ) to X and Y. As a result, Y rolls back to point A, W rolls and X and Z both abort their right-hand threacls. The cycle includes M2, R2, Ml, RI, and both

161

+ {z1}. The two sends ABORT(31 ) back to point ~, dashed lines.

computation this limit

will

be re-executed dependency in finite

be re-executed

is exceeded,

that

optimistically;

particular

pessimistically;

(3) commit,

messages are guaranteed

plies. If it is the receipt of a message, then the message is from a partial computation with a maximum of

when

computation

will

abort,

k optimistic computation

and

to be propagated

decisions. But we know that this partial is live, so we will produce the same results

as the pessimistic

time.

algorithm

by retrying

the computa-

tion from the read up to L times, and then blocking the receive operation, accepting only committed messages. Theorem

1 Subject

to the above conditions,

an opti-



mistic parallelization of a distributed system will yield the same pantial tTaces as the pessimistic computation.

We first show that icate is committed Every

commit

fork.

The

timeout

guard

left

completes

will

or will

with

pletes without fault.

predicate

thread

period,

4

every individual commit guard predor aborted in a finite period of time: xi is associated

either

time

a value fault, a value fault,

complete

out.

before

If it times

ai is aborted. then there

with

4.1

a or

denoted

may be a time

processes

is a time fault, there exists a chain .zk . . . + such that Xi d ~j + Y, Z,...

Predicate

Zi will be aborted

ym will have already

fore the fork

~m+k, or ym ~

then

eventually

Zi will

y~+k.

receive

Xi.

successfully

4.1.1

be-

depend

upon

message.

If

y~+k.

Therefore

or else commit

a finite

number

Process

are The

be denoted

Xn; the guess that

no time

fault

and no value fault.

Index

messages will

retains

a current

of its relative

thread

state number

indez,

a pair

con-

n, and an inter-

val number Interval which is incremented every time a message is received which introduces a new dependency.

the

The process stores its largest as MaxThTead.

4.1.2 Similarly, if there is no time fault, then either antecedent will abort, causing X to be aborted nite time,

with

State

Each thread sisting

messages will propagate through a chain which never includes the same process more than once. Since there are finitely many processes, this chain must eventually terminate.

through

identifiers

such as X and Y.

If y~ haa aborted,

an abort

y~ ~ Yna+k, then any dependency message for y~ received by process Y can be immediately propagated to the processes which

notation:

case letters,

gave rise to that fork will also be denoted Zn. The guess is that the corresponding left thread will complete

in the

aborted

by upper

nth fork of process x will

of

chain have terminated or timed out and messages have traveled a number of hops equal to the length of this chain. If any process appears more than once, say y~ then either

Structures

We use the following

If it com-

If there

and y~+~,

Data

the

out

after all left threads

Implementation

Commit

Guard

assigned

thread

identifier

Set

some in fiEach

be propagated

thread

it depends

of processes and will eventually

maintains upon.

a set Guard

For the initial

reach process X.

Guard

We now show liveness of the transformation by induction on the number n of antecedent optimistic decisions prior to any particular optimistic decision of a computation:

for all processes upon instance, {x3, x5, y9}.

of guesses that

thread

of a process,

= 0.

The commit

guard set contains which

all uncommitted

guesses

the process depends.

For

decision must be a fork. in a finite time. If it has

A thread may depend upon many guesses by the same process, particularly if an optimization like call streaming is applied repeatedly. We would like to take advan-

we simply refuse to not committed after L repetitions, fork, guaranteeing that there will be no abort.

tage of the fact that a dependence upon X5 implies a dependence upon all previous committed threads of z,

Suppose that our transformation is live for n < k. Then if the k + 18* decision is a fork, the above reasoning ap-

and only store one guess per process in Guard. However, since Z3 may abort before or after 35 has been forked, this would make it impossible to decide how to

If n = O, then this optimistic It will either commit or abort

162

respond

to the message ABORT (Z3).

4.1.5

The solution is to represent the guess identifiers as lpairs, consisting of an incarnation number along with a thread identifier. The incarnation number is incremented every

as a pair

Xi,n where

and n the index

guess of each of these

number

incarnation

staA table records the index associated

a given event.

For example,

whether

of the thread.

the

aborted,

history

records

or is unknown.

Since most guesses are assumed to commit, thk should be implemented aa a sparse vector with missing elements

An with

aasumed to be commits.

The incarnation start tathe set of antecedents of if incarnation

processes,

it has committed,

i is the

incarnation

the start of each incarnation. ble can be used to determine

Histories

Each process maintains commit information about each of the processes with which it communicates. For each

time the process aborts one of its own threads, and the thread index is reset to the index of the abort thread. Each guess is represented

Commit

simplicity. history

2 of process

We write

entry

We will

History[z3]

treat

it as a vector

to denote

for

the commit

for guess 3 of process x.

X begins at event 3, then the guess X2,4 is known to be preceded by Xl,l, X1,2, and X2,3, but not by X1,3. Receipt of C2,3 can also be taken as an implicit abort of z 1,3. Because the incarnation start table recordk the abort

history

Events

of each process, only the most recent guess

from each process needs to be maintained guard

4.2

4.2.1

in the commit

Fork

set.

Incarnation numbers and their implementation scribed in [9]; we will describe the algorithm incarnation they should

A foFk operation begins a new thread. A timer is set for the compiler-determined timeout interval. The

are dewithout

numbers for simplicity of explication, be used in any real implementation.

state is copied into each new thread;

but

lnteFval 4.1.3

Rollback

Points

For each guess g E GuaFd, the thread

guess, denoted then

we roll

we retain

became

by RoUbacks[g].

the state

dependent

which

introduced

is just

the dependency

Zn, so the new state index

from

the receive operation

the creating

thread.

upon g.

For

each thread

Graph

we maintain

a commit

graph,

CDG,

GuaFd

as leaf nodes.

which

is a DAG When

knows the set of dependencies it broadcasts to its commit

with

a thread

before, which of out-of-order

will substantially message arrivals

4. Also, since the section of the process

the

of the state for the right-hand

dependency members

terminates

for the following

thread.

of and

thmead,

4.2.2

Send

graph. When a message is sent, the GuaFd is appended to it. The entire CDG could be appended as well. This would allow earlier commit processing by the receiver at the expense of larger messages and more processing per message arrival.

For instance, if GuaFd = {xl, y7} and the control message PREGEDENCE(=l, {Z5}) arrives, then the edge Z5 -+ xl is added to the CDG. If a cycle is detected in the CDG,

the

between the fork and join points is simply waiting for the return of the call, it is not necessary to make a copy

them and they are added by each receiver dependency

vector is copied

s[xn] is assigned

As an implementation issue, note that in the case of call streaming the fork can be performed afteT the call has

such as in Figure

Dependency

Rollback.

value (n, O).

been sent, rather than decrease the likelihood Commit

n of the new

is (n, O).

GuaTd and the guess Xn. The Rollbacks

preceding

Note that if the rollback point is in a previous thread, then this (and all intervening threads) are aborted.

4.1.4

is incremented

number

The History and (7DG are copied from the creating thread. The GuaFd is the union of the creating thread’s

that

If that guess is aborted,

before

MaxThFead

the relative

to determine

index

upon

back to the end of the interval

Rollbacks[g], that

first

is set to O and

by one and becomes thread

at which

then the compiler-

determined predictor function is applied the initial state of the right thread.

then some guess must be aborted.

163

4.2.3

Message

introduce

Arrival

new dependencies. Newguards

When a message arrives at a process, it is first checked that the message is not an orphan. The message is an

then

orphan

ing that

if ~~~@ard

Orphan

: History[g]

= Guar&

new dependencies

mented

– Guardx

have been

a new interval

# 0

introduced,

haa begun.

Interval

and for each g c Newgucmds,

the Guard and the CDG, Index = (Thread, Interval),

= aborted.

If

meanis incre-

g is added

to

and Rollbacks[g] is set to the new state index.

messages are discarded. Note that

From a correctness

point

introduces

of view, a message can be de-

if several messages are available, the fewest new dependencies

the one that

(that

is, the one

livered to an arbitrary thread of the process, but we will often have information available which allows us to

for whkh lNewguardsl is smallest) should be chosen. ThkI minimizes the chance that receiving the message

optimize

will lead to an aborted

the delivery

decision.

As an optimization, if we know message (does not cause the the ified), then we do not need to aborted, we just remove it from

If we have some knowledge that the message was intended for a particular thread, for example if this is the return of a call, we can check that the message does not depend upon some future thread. If it does, then we know that the future thread must be aborted, since a return

which

must be in its past has interacted

with

that

this is purely

an optimization

which

When

If the message passes these checks, to any thread type,

provided

the future

which that

is waiting

then

the message does not

of the thread.

Generally,

cases

we deliver

for a message

When a join committable, known.

it

of this

depend

First

on

the message should

= {Z5, 113}, and the process is running

ends and follows

there it,

is a

the two

occurs, guess x.+k may be immediately immediately abortable, or remain un-

the verifier

function

is sent, and thread

its left

is executed

to check that

n is consistent

with

the

the state

n + k. If the verifier message ABORT(Zn+~ )

n+ k and any dependent

threads

are

terminated. Otherwise, if Guar& = @then Zn+k does not depend upon any uncommitted guesses, and can therefore be committed. The control message COMMIT (~n+k) is sent,

and preferably to Z6. Delivering it to Z4 would cause an abort because the end of X4 precedes the beginning of X5, and X5 would therefore depend upon itself.

thread

n is terminated.

If Guar& # 0, then Zn depends upon guesses whose outcome is unknown. In this case, the message PRECEDENCE(Xn ,Guard) is sent, and thread n waits for other COMMIT, ABORT, and PRECEDENCE messages until it has enough information to decide to commit or

Receive

When a receive is executed one of the messages made available by the message arrival processing described above is received. If no message is available, the thread

abort

blocks. When

X

that was used to begin thread fails ~n+k is aborted: a control

thread plus three forks with guesses X4, X5, and x& then the message can only be delivered to threads X5 and x&

4.2.4

Xn of process

state at the end of thread

probably be delivered to the earliest possible thread. For instance, if a message arrives for process a with its Guard

a thread

thread %+k running that logically threads are said to execute a join.

waiting for the return and abort. For call streaming this optimization is straightforward since returns are easy to calls; in other

Join

allows

early detection of aborts: if we do not check the message, then the caller thread will eventually time out

match up with their corresponding this may be more difficult.

that a message is a read process state to be modput it in Rollbacks; if the CDG.

it. 4.2.5

Note

state.

xn.

So far we have described these control messages as simply being ‘sent”. They could either be sent by broada message m is received

by process

X,

cast or by explicitly

it may

164

sending

them

to processes

which

If an edge added to the CDG creates a cycle, then a time fault haa been detected. All threads in the cycle

are known to depend on the guard in question (this information could be recorded during message send processing). The former should work well in a local-area network quently.

where the threads are created relatively The latter would be more appropriate

are aborted.

infrein a

wide-area network or when the number of threads created is large; the control messages could then be piggybacked on data messages being For simplicity

of explication,

5

exchanged. we assume

that

control

messages are broadcast. The broadcast itself need not be reliable, since the loss of a message will not cause incorrect commits. However, the broadcast must ble live in the sense that if repeated broadcasts are made, a message will eventually be delivered.

Related

Jefferson

[4] first

a model

in which

Receive

individual analogous

Commit

When the message COMMIT(Zn) is updated so that History[xn] moved from

Any

threads

informed,

and CDG.

each Guard

of Xn are also removed also have committed. that

is received, = commit.

are waiting

and may commit

Any

the CDG

from

Receive

the history z~ is re-

to Time

in a Time

to complete

Warp

a join

are

is received,

of Guard

a = min{rlg

virtual

is the

earliest

state

will be sent. that

follow

Receive

order

or else the server

depends

in a cycle in the

since this would

upon

require

the sending

process

process and any other

senders.

In

contrast, the parallel simulation application for which Time Warp was originally designed can use the simula-

A T = Rol&zcks[g]} that

result

roll

relation.

dent of the receiving

interval

would

will

order of We only

to decide upon a receive time for the message indepen-

tion time for events to provide

z~,

and we roll back to that point, possibly aborting some guarded threads, including threads waiting to complete a join.

4.2.8

time

In a distributed or loosely coupled parallel system, processes are compiled and instantiated independently. It is not feasible to impose a total ordering upon the com-

Then c Abortset

each

the history

putations, Xn in the CDG.

simulation,

There is nothing of a process and

call a server then the server must process the calls in the

= abort. Each thread is if x~ c CDG, itmust be rolled

the members

or-

as well.

roll back if our execution

thread

denote

distributed

back. In our approach, there is only a partial events, which is determined during execution.

message for that

to

are totally

Most importantly however is that Time Warp imposes a single, totally ordered global virtual time throughout the system. Every event must be assigned both a send time and a receive time by the application. If two clients

as a result.

When the message ABORT(Z~) is updated so that HistoTy[z.]

Let Abortset

Warp

process runs sequentially. to our guessing the future

happens-before

back, and an abort

of time-faults

predecessors

since they must

Abort

then checked in turn:

the concept

the events of a system

executing it in parallel with the present. However, there is no reason in principle why this could not be applied

assigned

4.2.7

applied

dered. Like Jefferson’s Time Warp implementation, our approach is based on optimism and rollback. The two approaches differ in two ways. Firstly,

4.2.6

Work

the total

order.

The ParaTran system [11] transparently parallelizes Scheme code using Time Warp. It replaces variables by stacks to support rollback, and automatically generates the timestamps required by Time Warp based upon the sequential

Precedence

dependencies

of the program.

As in Time

When the message PREcEDENCE(z~,Gua?’d) is received, we set History [zfi] = unknown. The CDG of each

Warp, there is a global total ordering of events as opposed to our partial ordering. The ParaTran methodology applies to a single sequential program, not a network of multiple sequential processes. In addition, there

thread is updated: for each g E Guard, the edge g d is added if either g or z~ is a node of the CDG.

is no provision for increasing mistically guessed values.

x~

165

parallelism

by using opti-

UNIX

Concurrency control techniques, such as optimistic concurrency control[6], also guess that confllcts do not occur and abort However,

computations

when conflicts

there are two differences

ness requirement

for optimistic

development.

Useniz

Conference

In Proceedings (July

of the Summer

1986).

are detected.

between

[2] ALLEN,

the correct-

parallelization

lation

and the

R., AND KENNEDY, of Fortran

!l’hansactions

correctness requirement for concurrency control: (1) In concurrency control, all transactions must be serializable, whereas in optimistic parallelization, each individ-

[3] HOARE, C. Prentice-Hall 1985.

Automatic

to vector

on Programming

tems 9, 4 (October

ual process is separately serializable but the system as a whole is partially ordered; (2) In concurrency control, any serializable ordering is legal, whereas in optimistic parallelization only orderings which are consistent with

K.

programs

trans-

form.

Languages

ACM

and Sys-

1987).

Communicating International,

Sequential Processes. Englewood Cliffs, N.J.,

Virtual time. Transactions [4] JEFFERSON, D. R. on Programming Languages and Systems 7, 3 (July

S1; Sz are legal.

1985), 404-425. [5] KUCK,

6

Conclusions

Because

our

[6] KUNG,

are transparent,

Trans.

and familiar

statements,

which

ming languages explicitly

programming

idioms,

we can

managing

low-level

The

particular

application

of our relevant

works in which network latency ingly large fraction of the total tributed

our algorithm

B.,

graphs and compiler

AND

ROBINSON,

J.

for

concurrency

control.

Syst. 6, 2 (June clocks,

T.

On op-

ACM

1981), 213-226.

and the ordering

system.

Commun.

of

ACM

1978), 558-565.

mes: A Language tice Hall,

January

for Distributed

Computing.

Pren-

1991.

[9] STROM, R. E., AND YEMINI, recovery in distributed systems.

for

to

high-speed

call

will become an increastime to complete a dis-

has the transparency

S. A.

Optimistic

ACM Transactions

Systems 3, 3 (August

1985), 204–226.

[10] STROM, R. E., AND YEMINI, S. A. Synthesizing distributed and parallel programs through optimistic transformations. In Current Advances

net-

in Distributed Computing and Communications, Y. Yemini, Ed. Computer Science Press, Rockville,

computation.

In summary,

LEASURE,

Program-

primitives

algorithm to

D.,

[8] STROM, R. E., BACON, D. F., GOLDBERG, A., LOWRY, A., YELLIN, D., AND YEMINI, S. A. 17e~-

concurrency.

is particularly

T.,

Database

on Computer streaming

M. Dependence

methods

21, 7 (July

such as call

are easy to reason about.

need not contain

PADUA,

[7] LAMPORT, L. Time, events in a distributed

achieve the performance benefits of complex procotols such as call streaming without requiring programmers to rewrite code when moving from a centralized to a distributed system. We allow programmers to use straightforward

H.

timistic

S1 terminates.

transformations

R.,

optimization. In Proceedings of the 8th ACM Conference on Principles of Programming Languages (January 1981), ACM.

S1 II Sz. It is not necessary to prove statically that S1 and S2 do not interfere; our algorithm allows them to run in parallel and if a conflict is detected, aborts S2 it after

KUHN,

AND WOLFE,

We have given an algorithm which is capable of transparently transforming a serial program S1; Sz into

and re-runs

D.,

MD,

prop-

erties of conventional parallelizing optimization. However it is applicable to systems of distributed processes that are either not subject to static analysis or which are not considered parallelizable by static analysis.

1987, pp. 234-256.

M. [11] TINKER, P., AND KATZ, Parallel tion of sequential Scheme with ParaTran. and Functional Programming Conference

execuIn Lisp (1988),

pp. 28–39. [12] UNITED STATES DEPARTMENT OF DEFENSE. Reference Manual for the Ada Programming Language,

References [1] ACETTA, GOLUB, YOUNG,

ANSI/MIL-STD-1815-1983 M., D., M.

BARON,

R,,

BOLOSKY,

RASHID, R., TEVANIAN, A., Mach: A new kernel foundation

W., AND for

166

cd., February

1983.