Performance Bene ts of Optimistic Programming: A Measure of HOPE Hanan L. Lut yya and Michael A. Bauer
Crispin Cowan
Department of Computer Science and Engineering Oregon Graduate Institute P.O. Box 91000 Portland, OR 97291-1000
Computer Science Department University of Western Ontario London, Ontario N6A 5B7
[email protected] [email protected]
[email protected]
February 2, 1995 Abstract
Optimism is a powerful technique for avoiding latency by increasing concurrency. By optimistically assuming the results of some computation, other computations can be executed in parallel, even when they depend on the assumed result. Optimistic techniques can be particularly bene cial to parallel and distributed systems because of the critical impact of inter-node communications latency. This paper describes how optimism can be used to enhance the performance of distributed programs by avoiding remote communications delay. We then present a new programming model that automates many of the diculties of using optimistic techniques in a general programming environment, and describe a prototype implementation. Finally, we present performance measurements showing how optimism improved the performance of a test application in this environment. Keywords: optimism, concurrency, parallelism, distributed computing, rollback, implementation, performance.
1 Introduction Optimism is a powerful technique for avoiding latency by increasing concurrency. By optimistically assuming the results of some computation, other computations can be executed in
Supported in part by the National Sciences and Engineering Research Council of Canada (NSERC).
1
parallel, even when they depend on the assumed result. This paper describes an environment providing automatic assistance for writing such optimistic programs, and characterizes the performance impact of a sample distributed program that uses these techniques. Optimistic techniques can be particularly bene cial to parallel and distributed systems. Inter-node communications latency is critical to performance because it limits the degree to which an application can be parallelized. Optimism can help mask inter-node communications latency by making optimistic assumptions about the behavior of remote nodes. Avoiding communications latency is just a special case of the general technique of using optimistic assumptions to avoid latency by increasing concurrency. Optimism increases concurrency by making an assumption about a future state, and verifying the assumption in parallel with computations based on the optimistic assumption. The classical example is optimistic concurrency control: optimistically assume that locks will be granted, proceed with the transaction, and verify that the locks were granted at the end of the transaction [21]. If the database is distributed, the latency of accessing the locks increases, but the chance of being granted locks does not change. In this paper, we will show how to avoid the latency of a remote procedure call, such as printf to a remote console, by optimistically assuming that the call behaved as expected. These examples illustrate why distributed systems particularly bene t from optimism: moving a computation to a remote node increases latency, but does not change the predictability of the computation. Any assumption can be made, provided a reliable method exists to verify that the assumption was correct. If the assumption is discovered to be correct, then latency has been avoided and performance has improved. However, if the assumption is incorrect, then all computations that used the assumption must be rolled back and re-executed using correct data. Optimism has been used in various areas to enhance performance [7, 15, 26, 28]. However, optimism is mostly embedded inside systems, and not exposed to the applications programmer. Optimism is not often used in applications because optimistic programs are dicult to write and require ad hoc techniques to implement. When an optimistic assumption is made, all of its causal descendants (i.e. all computations resulting from the optimistic assumption) must be tracked, and rolled back if the assumption proves false; a process we call dependency tracking. Verifying that an optimistic algorithm correctly tracks and rolls back all dependents of an optimistic assumption is tedious, at best, without automatic assistance. We believe that research into optimistic algorithms has been hindered by the lack of 2
adequate automatic assistance that allows the application programmer to make optimistic assumptions and not worry about the details of the dependency tracking, checkpointing, and rollback. This paper presents HOPE (Hopefully Optimistic Programming Environment): a programming model for expressing optimism. Previously, we have de ned the HOPE programming model and its applicability [7, 10, 23]. We have constructed a prototype HOPE system [8], de ned a formal semantics for HOPE [11], and proved that some important properties of the prototype implementation are consistent with the formal semantics [9]. This paper presents the performance results of using HOPE's optimistic primitives to avoid computational latency, speci cally communications latency. The rest of this paper is organized as follows. Section 2 describes optimism in more detail. Section 3 describes the HOPE programming model and presents an example. Section 4 describes the prototype implementation, and Section 5 describes some performance properties of the prototype. Finally, section 6 presents our conclusions and future research.
2 Optimistic Programming Optimistic programs are those that will allow sequences of actions that interfere in some runs to be executed concurrently based on a guess (or optimistic assumption). Optimistic programs have been used in a variety of applications such as fault tolerance [26, 19], replication [7, 15, 20, 24, 28], concurrency control [16, 18, 21, 30], and discrete event simulation [2, 13, 17, 25, 29]. In these applications optimism was used to decrease response time to enhance performance. Optimism has not been generally exploited because of the diculty in writing optimistic algorithms. Writing optimistic algorithms are dicult, time-consuming, and ad hoc, since an optimistic programmer must handle the following issues. First, checkpoints and rollback which are dicult and non-portable. Second, the optimistic programmer must keep track of all actions that must be rolled back if the assumption is wrong, including remote processes that have been sent messages. This requires that processes that make optimistic assumptions must keep track of all remote processes that messages were sent to. This is called dependency tracking. Third, the optimistic programmer must worry about transitivity i.e. what if a computation that proceeds based on an optimistic assumption (optimistic computation) is used to decide whether another assumption is valid. If this optimistic 3
computation is rolled back then the validation of the assumption must be undone.
2.1 Previous Work Supporting Optimism We believe that research into optimistic algorithms has been hindered by the lack of adequate automatic assistance that allows the application programmer to make optimistic assumptions and not worry about the details of the dependency tracking, checkpointing, and rollback. Previous work in supporting optimistic programming includes [3, 4, 18, 22, 27]. However, previous work has either restricted the type of optimistic assumption that can be made, or restricted the scope of optimistic computation [9]. In [27], computation based on an optimistic assumption is limited to the scope of an if or while statement. In [3, 4], computation based on an optimistic assumption is limited to the scope of a previously de ned encapsulation. All encapsulations must be de ned ahead of this time. This means that dependency tracking is not necessary, but it also means that the range of computation based on an optimistic assumption is statically bound. In Time Warp [18], the amount of computation based on a optimistic assumption is not statically bound, but only one kind of optimistic assumption can be made.
2.2 HOPE Features Any optimistic assumption can be made, and any method can be used to verify an optimistic assumption, including a method selected at run-time. HOPE is novel because we believe it to be the rst system in which both the kind and scope of an optimistic assumption is unconstrained. HOPE provides primitives that abstract the basic elements of optimism: specifying an optimistic assumption, identifying the assumption, and verifying whether or not the assumption is correct. HOPE supports transitivity, in that all primitives can be speculatively executed. HOPE is also novel because it provides an assumption identi er as a separate entity in the computational model. Previous optimistic systems have made optimistic computations directly dependent on other computations. Optimistic assumptions can be armed or denied in parallel with ongoing computations that depend on the optimistic assumption. The primitives are general, in that any optimistic assumption can be made, and any user-programmed criteria can be used in deciding whether the optimistic assumption was correct. The veri cation criteria can also be selected at run 4
time. Furthermore, armation and denial of optimistic assumptions, as well as making further optimistic assumptions, can all be performed by computations that are themselves optimistic, i.e. the primitives can be applied transitively. Finally, will show that despite the overhead required to implement HOPE, we are able to decrease the latency of a remote procedure call.
3 The HOPE Optimistic Programming Model HOPE is a set of primitives for expressing optimism, but is not a complete programming language. Rather, it is programming model for optimism, embodied as a set of primitives designed to be embedded in some other programming language, similar to Linda [5]. There are very few restrictions on the kinds of distributed systems in which HOPE can be embedded. HOPE can be embedded in any system providing concurrent processes that communicate with messages. For our purposes, a distributed program, P, is composed of communicating sequential processes, 1 2 , that execute operations that cause events that change the state of a process. A computation is a sequence of consecutive states that have occurred in the execution of a process. Rollback returns a process to a previous state in its computation and discards the computation subsequent to that state. An optimistic assumption is an assertion about a future state that has yet to be veri ed. An optimistic computation (or speculative computation) is a computation that proceeds based on an optimistic assumption and is said to be dependent on that assumption. If the optimistic assumption is found to be true, then the optimistic computation is retained, otherwise it is rolled back. HOPE consists of one data type and four primitives, as follows: AID x x is an assumption identi er or AID, used to identify particular optimistic assumptions. guess(x) The executing process makes an assumption identi ed by x. guess speculatively returns True immediately, and returns False if rolled back. arm(x) The executing process asserts that the optimistic assumption identi ed by x is con rmed as true. deny(x) The executing process asserts that the optimistic assumption identi ed by x is found to be false. p ; p ; :::; pn
5
free of(x)
The executing process asserts that the current computation is not, and never will be, dependent on the assumption identi ed by x.
The AID is a signi cant novel feature of HOPE. An AID is a reference to an optimistic assumption. Using the primitives described here, dependence, precedence, and con rmation of an assumption can all be handled separately. guess(x) appears to the programmer to be a boolean function that returns True if the assumption identi ed by x is correct, and returns False if x's assumption is found to be incorrect. The AID is an abstraction that represents the optimistic assumption, and can be used to arm or deny the optimistic computation. guess(x) will return True immediately, regardless of the status of the assumption. Speculative computation begins at this point, with the process \dependent" on x. Based on the returned value, the process then proceeds based on the optimistic assumption being true. If x's assumption is later discovered to be false, the process is rolled back to where it called guess(x), and False is returned instead of True. Having been informed through the return code that the assumption was incorrect, the process can then take the necessary steps to deal with the awed assumption. Idiomatically, guess(x) is embedded in an if statement. The \true" branch of the if statement contains the optimistic algorithm, and the \false" branch of the if statement contains the pessimistic algorithm. aid init(x) is used to initialize x ahead of time, so that a checking mechanism can be set up to verify x's assumption.1 arm(x) asserts that the assumption associated with the AID x is correct. Similarly, deny(x) asserts that the assumption associated with x is incorrect. If arm(x) is executed anywhere in the system, all the speculative computations executed from guess(x) onward are retained. If deny(x) is executed anywhere in the system, the computations from guess(x) onward, including any causal descendants in other processes, are discarded and rolled back, and execution re-starts from guess(x) with a return code of False instead of True. There is no restriction on how much computation can be executed before an optimistic assumption is con rmed. There is no restriction on which process in the program may con rm an optimistic assumption. Only one arm or deny primitive may be applied to a given assumption identi er, because multiple arm or deny primitives are redundant, and con icting arm and deny primitives have no meaning. Speculative processes can execute Although guess is applicable in modeling non-deterministic algorithms, it is a subjunctive statement, not a non-deterministic statement. 1
6
Worker Process = line = call print("Total if (line > PageSize) f call newpage();
=
, total);
is "
S1
=
S2
=
=
=
g
RPC
=
RPC
=
=
=
call print("Summary ..."); S3 RPC ... end process Figure 1: Before Call Streaming Transformation =
=
=
=
=
=
arm and deny primitives, and the system will transitively apply the assertions, i.e., if a speculative process is made de nite, then all arm primitives it has executed will have the same e ect as de nite arm primitives. In summary, a guess(x) eventually either results in the execution of an arm(x) and guess returns True, or deny(x) and guess returns False. In addition to explicit guess primitives, processes can also become dependent on AIDs by
exchanging messages. When a speculative process sends a message, the message is \tagged" with the set of AIDs that the sender currently depends on. When the message is received, the receiver implicitly applies a guess primitive to each of the AIDs in the message's tag. Execution of the free of(x) statement means that the executing task has no causal dependencies on any event in the speculative executions dependent on x. If any such dependency is ever detected, then x is denied. Asserting free of(x) ensures an execution in which the asserting process is causally free of all events dependent on the AID x. Subsection 3.1 illustrates the meaning of the HOPE primitives by presenting an optimistic program and its pessimistic equivalent. The pessimistic program in this example speci es the meaning of the optimistic constructs being illustrated.
3.1 Example A program increases its concurrency by making an optimistic assumption about its future state and verifying the assumption in parallel with computations based on the optimistic assumption [10]. If a computation proceeds based on an optimistic assumption and that assumption is shown to be incorrect, then all computations predicated on that assumption must be rolled back to correct for the incorrect assumption. If, during the optimistic computation, process sends a message to process then 's subsequent computation becomes pi
pj
7
pj
Worker Process aid t PartPage;
=
=
PartPage = aid init(); send(WorryWart, PartPage, total); if (guess(PartPage)) f do nothing =
g
g
=
else f
call newpage();
S2
=
S2
=
S3
=
=
call print("Summary ..."); ... end process. =
=
=
RPC
=
RPC
=
=
=
=
WorryWart Process(PartPage, total) aid t PartPage;
=
=
receive(PartPage, total); line = call print("Total is ", total); free of(PartPage); if (line PageSize) f arm(PartPage);
=
S1
=
=
RPC
=
