Parlists { a Generalization of Powerlists Submitted to Euro-Par 97 Jacob Kornerup Department of Computer Sciences, The University of Texas at Austin, Austin, TX 78712;
[email protected]
Abstract. The powerlist notation has been very successful in specifying
a number of parallel algorithms in a very elegant fashion. The major criticism of the notation was the restriction that input lengths were limited to powers of two. In this paper we present parlists, an extension of the powerlist notation to lists of arbitrary positive lengths.
1 Introduction The powerlist notation [10] has proven to be a major step forward in describing parallel algorithms succinctly. It allows the programmer to work at a high level of abstraction, by avoiding indexing notations, leading towards ecient implementations on parallel architectures [7]. The powerlist data structure is a list whose length is a power of two. In the powerlist notation it is possible to elegantly specify algorithms such as the Discrete Fast Fourier Transform without resorting to \index gymnastics" [10]. For such algorithms the restriction to lists of lengths a power of two is not serious; they are often presented this way in the literature. However, for most algorithms the restriction is unnatural. To remedy this, Jayadev Misra [11] generalized the powerlist notation to lists of arbitrary length by adding constructs from linear list theory. In this paper we present our completion of his theory1 and work through a number of examples, some of which were also done in the powerlist notation, and some new examples that could not be covered elegantly in the powerlist notation. This new data structure is named \parlist", which is short for parallel list.
2 Parlist Theory A parlist is a non-empty list, whose elements are all of the same type, either scalars from the same base type, or (recursively) parlists that enjoy the same property. Two parlists are similar if they have the same length and their elements are similar; two scalars are similar when they come from the same base type. We categorize parlists according to their length. The shortest parlist has length 1, it is called a singleton ; hx i denotes the singleton containing the scalar x. 1
The theory presented in [11] was incomplete. The author of this paper has completed the theory and worked out the examples presented in this paper. This paper is submitted with permission from Jayadev Misra.
A non-singleton parlist v can be deconstructed into a single element and a parlist whose length is one less than that of v, using both the . (\cons") and the / (\snoc") operator: v = a .p ^ v = q/ b (1) where a,b and the elements of p and q are similar to the elements of v, and p and q are similar parlists. In (1) a is the rst element of v and b is the last element of v. A parlist, p, of even length has the property that it can be deconstructed using both the ./ (\zip") and the j (\tie") operator: p = u ./ v ^ p = r j s (2) where u is a parlist containing the elements of p at even positions, and v is the parlist containing the elements of p at odd positions. Similarly, r is the parlist containing the rst half of p and s is the second half of p; the parlists r; s; u and v are all similar. We formalize the involved types and lengths by introducing a type system. The type function takes two arguments, a type and a positive integer and returns the type that contains all parlists with elements of the given type and length equal to the given number. Using we can give the signature for the parlist operators ( is a type and n is in , the positive natural numbers) : :1 (3) h i : ?! . : : :n ?! : :(n + 1) (4) / : : :n ?! : :(n + 1) (5) j : : :n : :n ?! : :(2 n) (6) ./ : : :n : :n ?! : :(2 n) (7) Note that powerlists is a subtype of parlists, corresponding to the lists whose length is a power of two ( : :(2k )). We overload the name , by having it denote the type of all parlists (corresponding to : :n for all and n) and naming the algebra we de ne below. We further re ne the type , by introducing the subtype that corresponds to all parlists whose elements are taken from . Finally, we partition the type into the subtypes . , . and . , de ned respectively as : :1; : :(2k) and : :(2k +1), where k ranges over . We will only write expressions that have a correct type as de ned above; e.g. when we write p ./ q it is understood that p and q are similar parlists, i.e. both members of : :n for some and n; when we write a .p it follows that a is similar to the elements of p. We use the proof format and notation presented by Dijkstra and Scholten [3], this includes writing function application using an in x dot: f:x . To minimize the use of parenthesis, we give di erent binding powers to the operators (most will be de ned later in the paper) as prescribed by the table below, where the operators are grouped in decreasing order from left to right, and operators in the same group have equal binding power: ParList
ParList
X
X
Pos
ParList X
X
ParList X
ParList X
ParList X
X
ParList X
ParList X
ParList X
ParList X
ParList X
ParList X
ParList X
ParList X ParList
ParList X
X
ParList
ParList.X
X
ParList.X
Singleton X EvenParList X
ParList X
ParList X
Pos
ParList X
X
ParList X
OddParList X
:
!
. / ./
j +
?
mod = ) ^ _
2.1 Axioms In the following we extend the axioms of the powerlist theory [10] to an axiomatization of the algebra. The algebra has ve constructors: h i; j; ./; . and /. They are all isomorphisms on their respective domains, with the following laws as consequence, where p; q; u; v 2 :n ^ a; b; c 2 : ha i = hb i a = b (8) pjq=ujvp=u ^ q=v (9) p ./ q = u ./ v p = u ^ q = v (10) a .p = b .q a = b ^ p = q (11) p/ a = q/ b a = b ^ p = q (12) ParList
ParList
ParList.X
X
(8t : t 2 :1 : (9 a :: t = ha i)) (13) (8t : t 2 :(2 n) : (9 u; v :: t = u j v)) (14) (8t : t 2 :(2 n) : (9 u; v :: t = u ./ v)) (15) (8t : t 2 :(n + 1) : (9 a; p :: t = a .p)) (16) (8t : t 2 :(n + 1) : (9 b; q :: t = q/ b)) (17) The following axioms are from the powerlist theory: ha i ./ hb i = ha i j hb i (18) (p j q) ./ (u j v) = (p ./ u) j (q ./ v) (19) The remaining axioms extends the powerlist algebra to de ne the full algebra. a . (p j q) ./ (u j v) / b = a . (u ./ p) j (v ./ q) / b (20) a . hb i = ha i j hb i (21) ha i / b = ha i j hb i (22) a . (p/ b) = (a .p) / b (23) a . (p ./ q) = (u ./ p) / b a .q = u/ b (24) a . (p j q) = (u j v) / c (9 b :: a .p = u/ b ^ b .q = v/ c) (25) Note the symmetry between ./ and j in the axioms that come from the powerlist algebra. Without an operational model the roles of ./ and j can be interchanged in this theory. This is not the case when we add the axioms that de ne the full algebra. If we interpret . and / as prepending and appending an element to a then the contrast between (24) and (25) and between (20) and (19) precisely capture the operational di erence between ./ and j. From (24) and (17) we can derive the following lemma. ParList.X
ParList.X
ParList.X
ParList.X
ParList.X
ParList
ParList
ParList
Lemma 1 (8a; p; q :: (9 b; u; v :: a . (p ./ q) = (u ./ v) / b ^ (8b; u; v :: (9 a; p; q :: a . (p ./ q) = (u ./ v) / b ^ Proof of (26) ((27) is similar)
a .q = u/ b a .q = u/ b
^ p = v)) (26) ^ p = v)) (27)
true
f axiom (17) g (8a; q :: (9 b; u :: a .q = u/ b)) f axiom (24) g (8a; p; q :: (9 b; u :: a . (p ./ q) = (u ./ p) / b ^ a .q = u/ b)) f one-point rule and trading g (8a; p; q :: (9 b; u; v :: a . (p ./ q) = (u ./ v) / b ^ a .q = u/ b ^ p = v)) End of Proof Let be a binary operator, de ned on a scalar type , i.e. : ?! lift to operate on with the following laws: ha i hb i = ha bi (a .p) (b .q) = (a b) . (p q) (p ./ q) (u ./ v) = (p u) ./ (q v) Y
Y
Y
Y
. We
ParList
(28) (29) (30) As alternatives to (29) and (30) we could have chosen (31) and (32) as they are interchangeable: (p/ a) (q/ b) = (p q) / (a b) (31) (p j q) (u j v) = (p u) j (q v) (32) It is a worthwhile exercise to prove that (31) and (32) follows from (28), (29) and (30).
2.2 Functions in
ParList
As mentioned above, the type can be divided into three disjoint subtypes: . , . and . . We use this partitioning when de ning functions over . . A function is de ned using three cases, one for each subtype, de ned using pattern-matching on the argument. The table below prescribe which constructors are allowed to pattern-match an argument in a subtype: Subtype Allowed Constructors . hi . ./ j . . / ParList.X
Singleton X
EvenParList X
OddParList X
ParList X
Singleton X
EvenParList X OddParList X
These rules are introduced to exploit parallelism as much as possible. When the argument is of type . , the computation should be expressed using a balanced divide-and-conquer strategy. Arguments of type . should be treated as an alignment step, introduced by necessity. As an example, we de ne the function rev : : :n ?! : :n that reverses its argument. rev :ha i = ha i (33) rev :(p ./ q) = rev :q ./ rev :p (34) rev :(a .p) = rev :p / a (35) Note that the choice of ./ and . as constructors was arbitrary. A de nition using j and/or / in their place yields the same function. In the de nition of rev , (34) expresses that each recursive case is independent and can be evaluated in parallel. The step described by (35) corresponds to a sequential \alignment" step, necessary before a balanced recursive step can be performed. In the case of rev the \alignment" step does not have to be sequential; depending on the parallel architecture (and the concrete implementation of ) rev can be evaluated in constant time. This would be the case on a CREW PRAM with the straightforward implementation of . A familiar property of rev is that it is its own inverse, which we prove below. rev :(rev :p) = p (36) EvenParList X
OddParList X
ParList X
ParList X
ParList
ParList
Proof of (36), base case: rev :(rev :ha i) = f rev (33) g rev :ha i = f rev (33) g ha i Inductive odd case: rev :(rev :(a . (p ./ q))) = f rev (35) g rev :(rev :(p ./ q) / a) = f rev (34) g rev :((rev :q ./ rev :p) / a) = f See (37) below g rev :(b . (u ./ v))
Inductive even case: rev :(rev :(p ./ q)) = f rev (34) g rev :(rev :q ./ rev :p) = f rev (34) g rev :(rev :p) ./ rev :(rev :q) = f induction (36) twice g p ./ q
Combining both sides of the odd case: a . (p ./ q) = (rev :v ./ rev :u) / b f Axiom (24) g a .q = rev :v/ b ^ p = rev :u f rev (35) g a .q = rev :(b .v) ^ p = rev :u f See (37) below g a .q = rev :(rev :q/ a) ^ p = rev :(rev :p)
= f rev (35) g rev :(u ./ v) / b = f rev (34) g (rev :v ./ rev :u) / b
f rev (35), induction (36) g a .q = rev :(rev :(a .q))
f induction (36) g a .q = a .q
f predicate calculus g true
End of Proof
In the above we used Lemma 1 and axiom (24) to establish (9 b; u; v :: b .(u ./ v) = (rev :q ./ rev :p)/ a ^ b .v = rev :q/ a ^ u = rev :p) (37)
2.3 Broadcast Sum
We turn to the de nition of the function sum : : :n ?! : :n, that returns a list where each element is the sum of all the elements of the argument list (a broadcast sum). Here is a type with the property that ( ; +) is a semigroup. It is necessary to de ne the functions lst : ?! , which returns the last element of a list, and [a+] : : :n ?! : :n, which returns the list where a has been added to each element of the argument list. sum:ha i = a (38) sum:(a .p) = (a + lst:t) . [a+]:t; where t = sum:p (39) sum:(p ./ q) = t ./ t; where t = sum:(p + q) (40) lst:ha i = a (41) lst:(p/ b) = b (42) lst:(p j q) = lst:q (43) [a+]:hb i = ha + bi (44) [a+]:(b .p) = (a + b) . [a+]:p (45) [a+]:(p j q) = [a+]:p j [a+]:q (46) When sum is evaluated with an argument of length 2n ? 1; n 1 there are n ? 1 deconstructions using . and n ? 1 deconstructions using ./ . Each deconstruction takes one parallel time step, in order to perform the sum. The total number of parallel steps thus becomes 2n ? 2. In contrast, if the argument is of length 2n , only n parallel steps are needed. Adding a sucient number of dummy elements (i.e. identity elements of +) to a list makes it into a powerlist. Thus, functions like sum can be evaluated in parallel in fewer steps than with the original list. ParList Y
ParList Y
Y
Y
ParList.X
ParList Y
2.4 Reusing Powerlist Proofs in the
ParList
X
ParList Y
Algebra
One of the advantages of the algebra is that it is a conservative extension of the powerlist algebra. As a consequence any result proven about a function ParList
de ned in the powerlist algebra holds for those parlists that are also powerlists, i.e. whose length is a power of two. Moreover, when powerlist function de nitions are extended with an odd case they become functions. Inductive proofs of properties done in the powerlist algebra can be reused in the proof of the same property for the extended function in the algebra. Depending on the structure of the powerlist proof, the only remaining proof obligation may be to prove the odd case. Take as an example the function rev de ned in the powerlist algebra by (33) and (34). A proof of (36) in the powerlist algebra consisting of the base and even cases is sucient to prove (36) in the powerlist algebra. When (35) is added to make rev a function, the odd case is the only missing part of the proof; the two others can be reused. A requirement is that the reused proof does not use properties that are speci c to powerlists, e.g. properties like length:p is even ) length:p is a power of 2. ParList
ParList
ParList
2.5 Pre x Sum Pre x sum is a fundamental parallel algorithm; it is used in many algorithms as a building block, e.g. carry lookahead addition (see section 3). Let be a data type with the property that ( ; +; 0) is a monoid. The pre x sum ps : : :n ?! : :n of a list p can be de ned [10] as the (unique) solution to the equation (in u): u = (0!u) + p (47) where the operator ! : : :n ?! : :n \pushes" a scalar into the list from the left and the rightmost element of the list is lost. ! has a higher binding power than that of . and / ; it is de ned as follows: a!hb i = hb i (48) a!(p/ b) = a .p (49) a!(p ./ q) = a!q ./ p (50) Exploring the de ning equation for pre x sum (47), we can derive a scheme for computing the pre x sum, due to Ladner & Fischer [9]. This was done for powerlists by Misra [10] presenting the base and even cases. We explore the odd case. By introducing q and b such that ps:(a .p) = q/ b we get: Y
Y
ParList Y
X
q/ b
ParList Y
ParList X
= f De ning equation for ps (47) g 0!(q/ b) + p/ a = f !(49) g 0 .q + p/ a = f Lemma 2 (54), below g
ParList X
0!q/ lst:q + p/ a = f Axiom (29) g (0!q + p) / (lst:q + a) Summarizing: q/ b = (0!q + p) / (lst:q + a) f Axiom (12) g q = 0!q + p ^ b = lst:q + a f De ning equation for ps (47), Leibnitz Rule g q = ps:p ^ b = lst:(ps:p) + a From the above along with Misra's de nition (51) and (52), we get the following de nition of Ladner and Fischer's algorithm: ps:ha i = ha i ps:(p ./ q) = 0!t + p ./ t; where t = ps:(p + q) ps:(p/ a) = ps:p / lst:(ps:p) + a
(51) (52) (53)
In the proof above we used Lemma 2; its proof and the formulation of a dual lemma is omitted for the sake of brevity.
Lemma 2 8a; p : a 2 ^ p 2 X
:
ParList X
:
a .p = a!p / lst:p
(54)
3 Adder circuits In [1] Will Adams presents powerlist descriptions for two arithmetic circuits that perform addition on natural numbers: the ripple carry adder and the carry lookahead adder. The ripple carry adder performs addition as it is rst taught in grade school; it is an inheritly sequential method, yielding a linear time method in the number of bits to be added. The carry lookahead adder uses a pre x sum calculation to propagate carries, yielding a method that is logarithmic in the number of bits to be added, in a setting where sucient parallelism available. Adams [1] proved that the ripple carry circuit correctly implements addition and that the carry lookahead and the ripple carry circuits are the same function. This result was achieved in the powerlist algebra. In the following we derive the carry lookahead adder and extend the equivalence result to the algebra. As mentioned in subsection 2.4, it is possible to reuse proof of properties of powerlist functions, when the function de nition is extended with a case that covers odd length lists. Thus, it is only necessary to specify and verify the odd case, assuming that the existing proof of the even case does not rely on properties that are speci c to powerlists. ParList
The ripple carry adder [1] has the type rc : f0,1g ParList:f0,1g:n ParList:f0,1g:n ?! ParList:f0,1g:n f0,1g
where the rst argument is the carry-in bit and the second and third argument are the two lists of bits that are to be added. The rst component of the result is the carry-out bit of the computation and the second component is a list containing the result of the addition. The following equations de ne rc, where (55) and (56) are taken from [1]: rc:b:hx i:hy i = (h(x + y + b) mod 2i; (x + y + b) 2)
rc:b:(p j q):(r j s) = (t; d)
t=ujv (u; c) = rc:b:p:r (v; d) = rc:c:q:s rc:c:(p/ a):(q/ b) = (u/ y; x) where x = (a + b + d) 2 y = (a + b + d) mod 2 (d; u) = rc:c:p:q
where
(55) (56) (57)
The carry lookahead adder has the following type cl : f0,1,g ParList:f0,1,g:n ParList:f0,1,g:n ?! ParList:f0,1,g:n f0,1,g
where corresponds to a \propagate" action for the carry-in value to a position. To specify the carry lookahead adder, Adams introduces the following associative scalar operators x if x = y : f g f g ?! f g xy = (58) if x 6= y y if y 6= (59) ? : f g f g ?! f g x?y = x if y = x if y 6= : f g f g ?! f g x y = (60) :y if y = :0 = 1 where :1 = 0 : = 0,1,
0,1,
0,1,
0,1,
0,1,
0,1,
0,1,
0,1,
0,1,
In [1] the carry lookahead adder is de ned by cl:b:p:q = (t; d)
where t = s r d = lst:s ? lst:r r = pq s = ps:(b!r)
(61)
where ps is computed using the associative operator ?. Expanding the odd case of the de nition of cl we get: cl:c:(p/ x):(q/ y) = (a; w) (62) where w = u v a = lst:u ? lst:v v = (p/ x) (q/ y) u = ps:(b!v) Comparing this with the quantities de ned by cl:b:p:q (61) we get v
= f (62) g p/ x q/ y
= f is scalar g (p q) / (x y) = f (61) g r/ (x y)
a
= f cl (62) g lst:u ? lst:v
u
= f (62) g ps:(b!v) = f calculation on the left g ps:(b!(r/ (x y))) = f ! (49) g ps:(b .r) = f lemma 2 (54) g ps:((b!r) / lst:r) = f ps (53) g ps:(b!r) / (lst:(ps:(b!r)) ? lst:r) = f (61) g s/ (lst:s ? lst:r) w
= f cl (62) g u v
= f calculations above g = f calculations above g lst:(s/ (lst:s ? lst:r)) ? lst:(r/ (x y)) (s/ (lst:s ? lst:r)) (r/ (x y)) = f lst (42) g = f Axiom (31) g lst:s ? lst:r ? (x y) (s r) / ((lst:s ? lst:r) (x y)) = f cl (61) g = f cl (61) g d ? (x y) t/ (d (x y)) in summary we have cl:c:(p/ x):(q/ y) = (t/ (d (x y)); d ? (x y)) (63) where cl:b:p:q = (t; d)
we can now prove the missing case in the proof of the equivalence of the ripple carry and carry lookahead adders.
Proof
rc:c:(p/ a):(q/ b) = cl:c:(p/ a):(q/ b) f rc (57) and cl (63) g
(s/ ((a + b + d) mod 2); (a + b + d) 2) = (t/ (e (x y)); e ? (x y)) ^ (s; d) = rc:c:p:q ^ (t; e) = cl:c:p:q f by induction (s; d) = (t; e) g (s/ ((a + b + d) mod 2); (a + b + d) 2) = (s/ (d (x y)); d ? (x y)) f equality on pairs g (a + b + d) 2 = d ? (x y) ^ s/ ((a + b + d) mod 2) = s/ (d (x y)) f Axiom (12) g (a + b + d) 2 = d ? (x y) ^ s = s ^ (a + b + d) mod 2 = d (x y) f (64) and (65) see below g true
End of Proof
In the last hint we used the following identities established in [1]: d ? (x y) = (a + b + d) 2 d (x y) = (a + b + d) mod 2
(64) (65)
4 Related Work & Conclusion This work is built on top of the work done on powerlists. Misra presented the theory and a number of examples [10]. Adams derived and veri ed addition circuits [1] (as mentioned in section 3) and multiplication circuits [2]. Kornerup presented a mapping strategy for powerlist onto hypercubic architectures [7] and derived the Odd-even sort in the powerlist notation [8]. The powerlist theory itself and many of Adam's results have been mechanically veri ed by Kapur and Subramaniam [6, 5] using the inductive theorem prover Rewrite Rule Laboratory. Gamboa [4] has veri ed many fundamental results about powerlists using the ACL2 theorem prover. His work focuses on veri cation of sorting algorithms. Mou and Hudak [13, 12] presented Divacon, a very general notation for describing divide-and-conquer algorithms in a functional manner. The Divacon notation is meant to capture the entire class of divide-and-conquer algorithms. Because of this generality it is dicult to prove the kinds of properties that have been done in the powerlist and notation. appears to be an appropriate generalization of the powerlist notation. The powerlist examples presented above had straightforward extensions to ParList
ParList
the algebra. The set of shared axioms makes it possible to reuse proofs of properties of the corresponding powerlist functions when proving the same properties of functions. One of the weaknesses of the above presentation is the lack of a precise method for specifying the cost, or the running time, of a given function in a speci c implementation. We have done some preliminary work in the context of powerlists that appears to generalize to the algebra; however, that work is not ready for publication. ParList
ParList
ParList
5 Acknowledgments The basic ideas behind the extensions presented in this paper are due to my advisor Jayadev Misra; he shared them with me and encouraged me to develop the theory and to write this paper. ParList
References 1. Will E. Adams. Verifying adder circuits using powerlists. Technical Report CSTR-94-02, University of Texas at Austin, Department of Computer Sciences, March 1994. 2. Will E. Adams. Multiplication circuits in powerlists. Unpublished manuscript, 1995. 3. Edsger W. Dijkstra and Carel Sholten. Predicate calculus and program semantics. Springer Verlag, 1990. 4. Ruben A. Gamboa. Defthms about zip and tie: Reasoning about powerlists in ACL2. Technical Report CS-TR-97-02, The University of Texas at Austin, Department of Computer Sciences, January 23 1997. 5. D. Kapur and M. Mechanically verifying a family of multiplier circuits. Lecture Notes in Computer Science, 1102:135{??, 1996. 6. D. Kapur and M. Subramaniam. Automated reasoning about parallel algorithms using powerlists. In Vangalur S. Alagar and M. Nivat, editors, AMAST '95, volume 936 of LNCS, pages 416{?? Springer-Verlag, 1995. 7. Jacob Kornerup. Mapping a functional notation for parallel programs onto hypercubes. Information Processing Letters, 53:153{158, 1995. 8. Jacob Kornerup. Odd-even sort in powerlists. To appear in Information Processing Letters, 1997. 9. Richard E. Ladner and Michael J. Fischer. Parallel pre x computation. Journal of the ACM, 27(4):831{838, October 1980. 10. Jayadev Misra. Powerlist: A structure for parallel recursion. ACM Transactions on Programming Languages and Systems, 16(6):1737{1767, November 1994. 11. Jayadev Misra. Generalized powerlists. Unpublished manuscript, May 1996. 12. Zhijing G. Mou. A Formal Model for Divide-and-Conque and Its Parallel Realization. PhD thesis, Department of Computer Science, Yale University, May 1990. 13. Zhijing G. Mou and Paul Hudak. An algebraic model for divide-and-conquer and its parallelism. The Journal of Supercomputing, 2(3):257{278, November 1988. This article was processed using the LATEX macro package with LLNCS style
