) ij
a4
m2
a5
d6
6
ld0
a1
m3
7
st7
a4
m2
a5
d6
8 9
ld0
a1
m3
10
st7
a4
m2
a5
d6
11
19
st7
vr0 ld0 m2 vr2
a5 vr5
a1
−2
m3
vr1
vr3
0
a4 vr4
d6 vr6
add: mult: div: load: store:
a1,a4,a5 m2, m3 d6 ld0 st7
st7
Figure 4.6: Skip and delta factors for stage scheduling (Example 4.4). Example 4.4 Figures 4.6a and 4.6b, respectively, show the dependence graph and a
stage schedule with minimumintegral MaxLive for the kernel y[i] = (x[i]4+x[i]+a)/ (x[i]+b) which will be used to illustrate our algorithm below. By postponing the schedule of the add1 and add5 operations from times 3 and 2 to times 9 and 8, respectively, the register requirements decrease from 15 to 13. As we will see in the next section, the schedule presented in Figure 4.6b also achieves minimum fractional MaxLive, thus resulting in the minimum register requirements for this kernel, MRTschedule, and set of functional unit latencies. 102
We can now use Equation (4.4) to compute all the si;j values. The resulting skip factors are shown in Figure 4.6a. Using these skip factors and computing the values according to Equation (4.7), as shown in Figure 4.6a for counter-clockwise traversal of underlying cycles u1 and u2, the constraints for this kernel correspond to the following set of underlying-cycle constraints, from Equation (4.6):
p0;2 + p2;3 + p3;4 ? p1;4 ? p0;1 = ?2 p0;1 + p1;4 + p4;6 ? p5;6 ? p0;5 = 0
Constraint u1: Constraint u2:
(4.8)
It is sucient to introduce one constraint per elementary cycle of the underlying graph, because the constraints of any arbitrary (possibly complex) cycle is simply a linear combination of the constraints of its elementary cycles. For example, the constraint associated with the outer cycle of Figure 4.6 as computed from Equation (4.6) with = 1 from Equation (4.7): p0;2 + p2;3 + p3;4 + p4;6 ? p5;6 ? p0;5 = ?2 can be obtained by simply adding the constraints for underlying cycles u1 and u2. Therefore, satisfying constraints u1 and u2 will necessarily satisfy the constraint associated with the outer cycle. Although any pi;j values that satisfy these constraints result in a valid stage schedule, we are interested in the solution that minimizes integral MaxLive. In general, the integral part of a virtual register lifetime along a register data ow edge (i; j ) is directly proportional to the sum of the minimum and additional skip factors and the dependence distance along that edge. As a result, we can express the integral part of the lifetime from operation i to operation j , !i;j iterations later, as:
inti;j = si;j + pi;j + !i;j
(4.9)
The integral part of the lifetime associated with vri corresponds to the maximum integral part among all the outgoing register data ow edges of operation i:
inti =
max (! + si;j + pi;j ) 8j s.t. (i;j )2Ereg i;j 103
(4.10)
Using Function (4.10), we compute the integral MaxLive of the kernel of Example 4.4 as the sum of the lifetimes over all virtual registers: max(p0;1 + 1; p0;2; p0;5) + p1;4 + p2;3 + p3;4 + p4;6 + p5;6 + p6;7 + 5
(4.11)
We can now reduce the problem of nding a stage schedule that results in the minimum integral MaxLive to a well-known class of problems, solved by a linear programming (LP) solver [47]. Note, however, that (4.11) is not acceptable as the input to an LP-solver, because the objective function cannot contain any max functions. However, since we are minimizing the objective function, we can remove the max function by using some additional inequalities, called max constraints:
pi;j + si;j + !i;j bi
8(i; j ) 2 Ereg
(4.12)
Finally, we can remove the constant term in the objective function. The LP-solver input for the kernel and MRT presented in Figure 4.6 is therefore:
Minimize: Constraint u1: Constraint u2: Max Constraints:
b 0 + b1 + b2 + b3 + b4 + b5 + b6 p0;2 + p2;3 + p3;4 ? p0;1 ? p1;4 = ?2 p0;1 + p1;4 + p4;6 ? p0;5 ? p5;6 = 0 p0;1 + 1 b0; p0;2 b0; p0;5 b0 p1;4 b1; p2;3 b2; p3;4 b3 p4;6 b4; p5;6 b5; p6;7 b6
(4.13)
The result of the LP-solver is shown in Figure 4.6c, i.e. p0;1 = p0;5 = 2 and all other pi;j = 0. We can verify that this solution satis es the constraints of Problem (4.13) and yields b0 = 3. Since, except for b0, all the bi in the objective function are 0, this solution has integral MaxLive = b0 + 5 = 8 registers. The corresponding stage schedule is shown in Figure 4.6b. 104
Algorithm 4.1 (MinBu Stage Scheduler) A schedule with the minimum inte-
gral MaxLive for a kernel with a general dependence graph, MRT, and set of functional unit latencies is found as follows:
1. Compute all si;j using Equation (4.4) and search for all elementary cycles in the underlying graph of the dependence graph GfV; Esched [ Ereg g [74]. 2. If the underlying dependence graph is acyclic, the solution that produces the minimum integral MaxLive is obtained by setting the values of all pi;j to zero. 3. Otherwise, build an underlying-cycle constraint for each elementary cycle of the underlying dependence graph using Equations (4.6) and (4.7). Add the max constraints using Inequality (4.12) and derive the integral MaxLive objective function by summing the b variables over i. 4. Solve the system of constraints to minimize the integral MaxLive by using an LPsolver. The solution de nes the pi;j values that result in the minimum integral MaxLive. Note that the solver may fail to nd any solutions because of some critical recurrence cycles in the dependence graph. In this case, the given MRT has no solution and a new MRT-schedule must be provided, possibly with a di erent II .
We prove the correctness of this algorithm with two theorems. The rst theorem validates the correctness of the solution for general dependence graphs, and the second theorem validates the solution and the linear solution time for the case of an acyclic underlying dependence graph.
Theorem 4.1 The solution of the minimum integral MaxLive LP-problem as de ned
in Algorithm 4.1 results in the minimum integral MaxLive for a kernel with a general dependence graph, MRT, and set of latencies. The solution is integer valued, if an LP-solver nds an optimal solution.
Proof. We rst demonstrate that a stage schedule is valid if and only if it satis es the nonegative pi;j constraints and the underlying-cycle constraints. Since each si;j 105
is de ned in Equation (4.4) to be the smallest integer value that minimally satis es scheduling constraint (4.3), this scheduling constraint is satis ed if and only if pi;j 0. Thus, enforcing nonegative pi;j values for each (i; j ) 2 Esched satis es each scheduling constraint individually. In addition to enforcing the scheduling dependences, a valid schedule must also enforce the assignment constraints, which insure that each operation is assigned to a unique time (see Section 5.3.1 for example). Because we represent a schedule in our formulation by the time di erences along the edges of the dependence graph, enforcing the assignment constraints is equivalent to insuring that every path leading to a vertex results in precisely the same time. If there are two distinct paths leading to a vertex, the union of the edges in these two paths must create one or more underlying cycles. Because the underlying cycle constraints (4.6) precisely enforce that the cumulative distance along the closed path of each underlying cycle is 0, the two distinct paths along an underlying cycle result in the same time di erence between their two end points. Thus, enforcing the assignment constraints is equivalent to enforcing the underlying cycle constraints. Note that satisfying the underlying cycle constraints only for the elementary underlying cycles is sucient, since any nonelementary cycle can be simply expressed as a linear combination of several elementary cycles. Consequently, a stage schedule is valid if and only if it satis es the pi;j 0 and the underlying-cycle constraints. Moreover, since the underlying-cycle constraints, the max constraints, and the objective function (minimize integral MaxLive) are all linear, we can use an LP-solver to nd a schedule that minimizes integral MaxLive. Furthermore, since the solution (namely the set of pi;j values) must be (nonnegative) integer valued, we must show that if an LP-solver nds a optimal solution, the pi;j in that solution will be integer valued. This part of the proof is provided in Appendix A.
2
Theorem 4.2 For a dependence graph with an acyclic underlying dependence graph, the minimum integral MaxLive for a given kernel, MRT, and set of latencies is found in a time that is linear in the number of edges in the dependence graph. The minimum integral MaxLive is found by simply setting all pi;j values to zero.
106
Proof. Consider applying the general form of Algorithm 4.1 (Steps 1, 3, and 4)
to a dependence graph with an acyclic underlying dependence graph. There are no underlying cycles, and hence no underlying-cycle constraints, in the linear program problem formulation, e.g. see (4.13). The max constraints all have the form pi;j bi + ci;j where bi and pi;j are to be found and ci;j is a constant. The objective is to minimize a linear combination of the bi, all of which have nonnegative coecients. Consequently the solution cost is monotonically nondecreasing in the bi. Setting all pi;j = 0, their minimum possible value, also allows the bi to have their minimum feasible value and minimizes the objective function. Therefore, Steps 3 and 4 take no time and Step 2 of Algorithm 4.1 can be used instead. The solution time for Algorithm 4.1 is thus reduced to the time for Step 1. The si;j are each computed in constant time using Equation (4.4). Since there is one si;j parameter per edge in the dependence graph, the solution time is linear in the number of edges. 2 A corollary of Theorem 4.2 states that setting the pi;j to 0 for each edge in an acyclic part of an underlying dependence graph results in the minimum integral part for those edges; furthermore, increasing the pi;j of any of these edges cannot decrease integral MaxLive. This corollary can be shown by applying the proof of Theorem 4.2 for the acyclic parts. We conclude this section by contrasting the complexity of our stage scheduling model summarized in Figure 4.7 with the modulo scheduling model presented by Govindarajan et al [44], where both models minimize integral MaxLive for a machine with nite resources, but we assume a given MRT and Govindarajan et al do not. Note that since their algorithm produces an MRT as well, it considers a larger scheduling space and may thus result in schedules with smaller integer MaxLive. We address algorithms that explore the entire modulo scheduling space in Chapter 5. The most important di erence between these two algorithms that minimize integer MaxLive is that the stage scheduling model can be directly solved with an LP-solver instead of an integer linear programming solver which is usually implemented by solving an exponential number of LP-problems. The stage scheduling model also has 107
Minimize:
Bu =
X i2V
bi
Subject to: Underlying Cycles:
X (i;j )2u
signi;j (u) pi;j = ?
X (i;j )2u
signi;j (u) si;j ? u
8u 2 UG Max:
bi ? pi;j !i;j + si;j
8(i; j ) 2 Ereg
Lower Bounds:
bi 0; integer
8i 2 V
pi;j 0; integer
8(i; j ) 2 Esched [ Ereg
De nitions:
rdisti;j = (rowj ? rowi ) mod II
X signi;j (u) rdisti;j u = II1 &
(i;j )2u
' l i;j ? rdisti;j si;j = ? !i;j II
Figure 4.7: Stage scheduling model for minimum integral MaxLive (minimum Bu ers).
108
a vastly smaller number of variables and constraints. For example, all the resource constraints are resolved prior to stage scheduling during the MRT-scheduling phase, and therefore are ignored in the stage scheduling model. Also, most of the dependence constraints present in the modulo scheduling model disappear in the stage scheduling model as most of them are resolved statically by selecting appropriate minimum skip factors (si;j ). The only remaining dependence constraints are the underlying cycle constraints. Similarly, most of the constraints required to compute the register requirements associated with a solution are also computed statically in the stage scheduling model. The only pi;j that actually must be accounted for are the ones where operation i is in an underlying cycle; the corollary of Theorem 4.2 insures that all other pi;j will be 0 in an optimum stage schedule. As a result, the stage scheduling model results in solution times that are signi cantly smaller than those of the modulo scheduling model, and thus the stage scheduler is an interesting tool for analyzing large loops, even though it considers only the given MRT.
4.3.2 Scheduling for Minimum MaxLive In this section, we extend the stage scheduling algorithm of the previous section to consider the fractional as well as the integral parts of the register requirements. We rst quantify the fractional MaxLive associated with a kernel, an MRT, and a stage schedule. Then we investigate the interaction between the fractional MaxLive and the search for an optimal stage schedule. We conclude the section by presenting an algorithm that minimizes the total MaxLive among all stage schedules for a given MRT. Consider a register edge (i; j ) between operations i and j scheduled in rowi and rowj , respectively. We characterize the fractional part of the lifetime associated with register edge (i; j ) in row r by fracr;i;j , where fracr;i;j is 1 if the fractional part of the lifetime spans row r, and 0 otherwise. In general, fracr;i;j is 1 if row r is encountered in the MRT when going forward (with wraparound) from rowi to rowj , i.e. from the row of def-operation i to the row of use-operation j , inclusively. This 109
de nition implies that fracr;i;j is 1 if the modulo distance from rowi to r is no larger than the modulo distance from rowi to rowj , i.e. if dist(rowi; r) rdisti;j , using the de nition of the dist function and the rdisti;j terms shown in Equation (4.1). Using the following generic function X : 8 > < 1 if condition is true (4.14) X (condition) = >: 0 otherwise we may express the fractional part of the lifetime generated by register edge (i; j ) in row r as:
fracr;i;j = X (dist(rowi; r) rdisti;j )
(4.15)
Note that Equation (4.15) is based on our virtual register model, which assumes that a virtual register is reserved when its def-operation is scheduled and freed one cycle after its last use-operation is scheduled. Since, in general, the register requirements of a virtual register are determined only by its longest lifetime, i.e. by the edge from its def-operation to its last use-operation, we introduce two following de nitions:
lasti ldisti
operation that last uses vri distance between the row of operation i to the row of operation lasti, possibly in the next instance of the MRT.
Using our formulation, lasti and ldisti are determined as follows:
lasti = k s.t. (i; k) 2 Ereg and rdisti;k + (!i;k + si;k + pi;k ) II = max rdisti;j + (!i;j + si;j + pi;j ) II (4.16) 8j s.t. (i;j )2E reg
ldisti = dist(rowi ; rowlasti )
(4.17) 110
The sum of the fractional parts in row r, referred to as fracr , is then obtained by summing the fractional part of each operation, i, to its last use-operation, lasti, namely:
fracr =
X
X (dist(rowi; r) ldisti)
(i;lasti)2Ereg
(4.18)
where X (dist(rowi ; r) ldisti) corresponds to fracr;i;lasti . Because fractional MaxLive is determined by the row with the most fractional parts, it is computed as follows: fractional MaxLive =
max fracr
r=0;:::;II ?1
(4.19)
Total MaxLive is then simply the sum of integral and fractional MaxLive, namely:
X
RMRT =
(i;lasti)2Ereg
max r=0;::: ;II ?1
X
(i;lasti)2Ereg
(!i;lasti + si;lasti + pi;lasti ) +
X (dist(rowi; r) ldisti)
(4.20)
where the rst and second terms correspond, respectively, to the integer and fractional MaxLive associated with the given MRT and stage schedule. We can now formalize the interaction between the fractional and the integral parts of the register requirements as follows. When searching for a stage schedule with minimum integral MaxLive, the solver determines the pi;j values, which in turn determine fractional MaxLive, as indicated by Equations (4.16), (4.17), (4.18), and (4.19). Since fractional MaxLive is not taken into account when minimizing integral MaxLive, it is possible that there is another minimum integral MaxLive stage-schedule with lower fractional MaxLive, or even another stage-schedule with higher integral MaxLive but with lower fractional and total MaxLive. Consider, for example, a minimum integral MaxLive stage-schedule that does not result in minimum fractional MaxLive. Since the schedule does not achieve fractional MaxLive, we know that there must be at least one virtual register, vri, for which another use-operation, x, would lower fractional MaxLive if it were the last 111
use-operation. By introducing additional constraints in the linear program, we may arti cially force operation x to be the last use-operation of vri, i.e. we can force lasti = x. By solving this augmented problem, i.e. by searching for a stage schedule with minimum integral MaxLive among the subset of all stage schedules that have lasti = x, we may nd out if fractional MaxLive can be decreased without increasing integral MaxLive. Note that, in general, there may be one or more use-operations of one or more virtual registers that can decrease fractional MaxLive. In these cases, we may have to investigate combinations of forced last use-operations, that can possibly achieve a lower MaxLive. Note that even though the fractional MaxLive can be reduced by at most one by altering the contribution from a single virtual register, it is possible to achieve lower total MaxLive values even if integral MaxLive is increased, because the increase in integral MaxLive may be smaller than the decrease in fractional MaxLive caused by changing the last use-operations of several virtual registers. We may now formulate our general approach to minimizing total MaxLive. First, we search for a minimum integral MaxLive stage-schedule among all stage schedules that share a given MRT, using Algorithm 4.1. The resulting schedule, referred to as the base solution, achieves minimum integral MaxLive but may not, in general, result in minimum fractional MaxLive. The second step is to identify the combinations of lasti operations that decrease fractional MaxLive below that of the base solution. In the remainder of this section, we refer to such combinations of lasti operations as forcing combinations, and the lasti operations as their forcing candidates. We assume in this section that every possible combination is investigated; in Section 4.3.3, however, we will present inclusion properties that reduce the number of forcing combinations that must be investigated to guarantee an optimal solution. Finally, we investigate each forcing combination in turn, by augmenting the linear program that was used to compute the base solution with additional constraints associated with its forcing candidates, and applying Algorithm 4.1 to the augmented problem formulation. The additional constraints, referred to as forcing constraints are de ned as follows. Consider a forcing candidate lasti = x that may result in a reduced fractional 112
MaxLive if operation x is scheduled as the last use-operation of vri. We could simply implement these constraints by forcing every other use-operation of vri to be scheduled no later than operation x. While these constraints would clearly work, they may be overly constraining because some of these other use-operations may not increase the fractional parts beyond that of operation x, even if they were scheduled last. Thus, we can formulate forcing constraints that consider only the use-operations that e ectively increase the fractional part of a virtual register lifetime beyond that of operation x, i.e. force each operation z such that (i; z) 2 Ereg with rdisti;z > rdisti;x not to be the last use-operation of virtual register i. We may thus formulate the forcing constraints associated with the forcing candidate lasti = x as:
!i;z + si;z + pi;z < !i;x + si;x + pi;x 8 z s.t. (i; z) 2 Ereg and rdisti;z > rdisti;x
(4.21)
To illustrate this process, consider Example 4.4 again. As shown in Figures 4.8a and 4.8b, three operations use vr0, add1, mult2, and add5 scheduled in rows 0, 1, and 2, respectively. The fractional parts associated with Example 4.4 are shown in Figure 4.8c with three di erent cases for vr0 depending on which of the three operations is the last use of vr0. As indicated before, the rst step is to search for a schedule with minimum integral MaxLive; in this case, we initially obtained a schedule (referred to as the base solution) with an integral MaxLive of 8 and a fractional MaxLive of 6, because add1 was scheduled II cycles earlier than shown in Figure 4.4b and thus add5 was the last use-operation of vr0. The second step is to nd the forcing combinations, if any, that decrease the fractional MaxLive of the base solution. Since there is only one virtual register with multiple use-operations, with 3 use-operations in this example, there are precisely 3 possible last use-operations. Of these three, since the fractional MaxLive of 6 can only be reduced if add1 is forced as the last use-operation of vr0, only last0 = add1 results in a lower fractional MaxLive than achieved by the base solution. To investigate the stage schedules that result from forcing last0 to be the candidate add1, we introduce the forcing constraints 113
d) Dependence graph with additional constraints
a) Original dependence graph vr0 ld0
vr0 ld0
m2
a5
vr2
a1
a5
m2
vr5
vr2
vr5
a1
vr1
vr1
m3
m3
vr3
vr3 a4
a4
vr4
vr4 d6
d6
vr6
vr6
st7
st7 c) Fractional parts
b) MRT (II = 3)
vr0 row:0
ld0
a1
m3
1
st7
a4
m2
a5
d6
2
a1 last−use of vr0
vr0
vr0
vr1
m2 last−use of vr0
vr2
vr3
vr4
vr5
vr6
a5 last−use of vr0
Figure 4.8: Fractional part of the lifetimes (Example 4.4). produced by Inequality (4.21). Since the 2 other use-operations of vr0 are both associated with larger fractional parts, we must introduce 2 forcing constraints, one for the mult2 and one for the add5 operation. These constraints are depicted in the dependence graph of Figure 4.8d by the additional (bold) edges. Solving the augmented problem formulation for a stage schedule with minimum integral MaxLive also results in a schedule with an integral MaxLive of 8 in this example, thus achieving a lower fractional MaxLive without increasing integral MaxLive. The resulting stage schedule is the one shown in Figure 4.6b.
114
Algorithm 4.2 (MinReg Stage-Scheduler) A schedule with the minimum MaxLive
for a kernel with a general dependence graph, MRT, and set of functional unit latencies is found as follows:
1. Use Algorithm 4.1 to compute the minimum integral MaxLive. We refer to this resulting stage schedule as the base solution. If Algorithm 4.1 fails to produce a stage schedule, which may happen because of some critical recurrence cycles in the dependence graph, a new MRT-schedule must be provided, possibly with a di erent II . 2. If the underlying dependence graph is acyclic, the base solution results in the minimum MaxLive. 3. Otherwise, compute the total MaxLive associated with the base solution using Equation (4.20) and determine the forcing combinations that decrease the fractional MaxLive of the base solution. While all possible forcing combinations can be exhaustively computed and used, Algorithm 4.3 in Section 4.3.3 indicates how to use inclusion properties to reduce the number of forcing combinations that need to be investigated. 4. Use Algorithm 4.1 repeatedly to compute the minimum integral MaxLive for the base system augmented by the forcing constraints associated with each forcing candidate of the current forcing combination. Forcing constraints are generated using Inequality (4.21). By processing each forcing combination in order of increasing fractional MaxLive, the rst solution that achieves the integral MaxLive obtained by the base solution is optimal. If no such solution is found then continue evaluating forcing combinations until the decrease in fractional MaxLive below the base solution for all remaining unevaluated combination is no greater than the reduction in total MaxLive obtained by the best solution yet found. Then this best solution found is optimal.
115
We prove the correctness of this algorithm with two theorems. The rst theorem validates the correctness of the solution for general dependence graphs, and the second for acyclic underlying graphs.
Theorem 4.3 Algorithm 4.2 produces a solution with the minimum MaxLive for a
kernel with a general dependence graph, MRT, and set of latencies. Furthermore, any solution found will be integer valued, if an LP-solver nds an optimal solution.
Proof. Since we test all the forcing combinations that may reduce the fractional
MaxLive, and since Algorithm 4.1 produces the minimal integral MaxLive for each of the problem augmented by the forcing conditions, the solutions of Algorithm 4.2 and the original unaugmented solution must include the minimum achievable total MaxLive for this MRT. By ordering the forcing combinations by increasing order of fractional MaxLive, we may stop the search as soon as a solution with reduced fractional Maxlive and the minimum integral MaxLive obtained by the base solution is found. This solution is optimal since we rst checked all forcing combinations with lower fractional MaxLive and since we cannot, by de nition, reduce the integral MaxLive below that of the base solution. If no such solution is found, the nal stopping criterion in Step 4 of the algorithm stops considering new combinations when it is impossible for any of them to result in a better solution than the best solution found so far. Moreover, the additional forcing constraints de ned by Inequality (4.21) can be represented in the original dependence graph by additional scheduling edges. Therefore, from Theorem 4.1, any solution produced by an LP-solver for each use of Algorithm 4.1 is guaranteed to be integer valued, if an LP-solver nds an optimal solution.
2
While there is potentially an exponential number of forcing combinations to investigate, this number tends in practice to be relatively small for several reasons. First, the number of forcing combinations is dependent on the number of operations lying in underlying cycles, which is usually only a fraction of the total number of operations. Second, we have to consider only those forcing combinations that potentially reduce 116
the fractional MaxLive. For example, we do not need to consider forcing combinations unless they may impact the value of the max function in Equation (4.20). Finally, we can use some inclusion properties to reduce the total number of forcing combinations investigated. More details are provided in Section 4.3.3.
Theorem 4.4 For dependence graphs with acyclic underlying graphs, the solution that minimizes integral MaxLive also minimizes the total MaxLive for a given kernel, MRT, and set of latencies.
Proof. In general, the solution that minimizes integral MaxLive may not be a solu-
tion with the minimal fractional MaxLive. Recall, however, that the Algorithm 4.1 solutions for acyclic underlying graphs have all pi;j = 0. Therefore, reducing a fractional lifetime for some vri requires setting at least one pi;j to 1 or more (i.e. setting pi;j to at least the value that minimally forces operation j to be the last use-operation of vri, as computed in Equation (4.16)), thereby increasing the integer lifetime for vri by at least 1. Therefore, reducing the fractional register requirement for vri cannot decrease the MaxLive of this virtual register. As the lifetime of each vr is independent of the lifetime of any other vr for an acyclic underlying graph, it is never advantageous to force a di erent last use-operation unless the total lifetime of the associated vr is decreased. Thus, we cannot reduce the register requirements of the base solution if the underlying graph is acyclic. 2 A corollary of Theorem 4.4 states that Theorem 4.4 can also be applied to acyclic parts of the underlying dependence graph (i.e. to a multiuse virtual register lifetime none of whose def-use edges are in an underlying cycle), because in acyclic parts of the underlying graph, increasing such a lifetime cannot decrease MaxLive, as shown in the proof of Theorem 4.4. This result is in contrast to the cyclic part of the underlying graph, where Theorem 4.4 does not apply since increasing the length of one lifetime may decrease the length of another, and thus may possibly decrease MaxLive.
117
4.3.3 Determining Forcing Combinations In the previous section, we reduced the value of fractional MaxLive associated with a stage schedule by forcing the last use-operations of multiuse variables with at least one def-use edge in an underlying cycle. In this section, we proceed in the reverse fashion: we determine a target value for fractional MaxLive and investigate the resulting forcing candidates and forcing combinations that achieve this target fractional MaxLive. a) MRT (II = 3)
b) Fractional parts vr0
row:0
ld0
a1
m3
1
st7
a4
m2
a5
d6
2
lastuse: a1
vr0
vr0
m2
a5
potential fractional parts
vr1
vr2
vr3
vr4
vr5
vr6
minimum fractional parts
Figure 4.9: Fractional parts of the lifetimes (Example 4.5). Since the last use-operations are unknown in this section, we distinguish two components of the fractional parts associated with a virtual register. The rst component corresponds to the minimum fractional parts and spans (inclusively) from the row of the def-operation to the row of the use-operation that is rst encountered in the MRT when going forward, with wraparound. This component is illustrated by the black bars in Figure 4.9b. By counting the black bars in each row, we can obtain a lower bound on the fractional MaxLive, 5 in this example. The second component corresponds to the potential fractional part and spans (exclusively) from the rows of the use-operation that was rst encountered to the row of the use-operation that is last encountered in the MRT when going forward, with wraparound. This component is illustrated by the grey bars in Figure 4.9b. Depending on which operation is scheduled last, these gray bars may a ect the overall fractional MaxLive. In this example, fractional MaxLive increases from 5 to 6 if either of the operations m1 or a3 is the 118
last use of vr0. Therefore, forcing operation a2 to be the last use ensures a fractional MaxLive of 5. Using Equation (4.18), we describe the number of fractional live registers for each row of the MRT as a system of equations, in which the minimal fractional parts are reduced to a constant term. We obtain for Example 4.5 the following fracr equations for each of the MRT rows:
frac0 = 5 frac1 = 5 + X (1 ldist0) frac2 = 4 + X (2 ldist0)
(4.22)
which results from the fact that there are 5 minimum fractional parts in row 0, 5 minimum fractional parts and one potential fractional part in row 1, and 4 minimum fractional parts and one potential part in row 2. By looking at this system of equations, we can deduce that the value of fractional MaxLive is 6 unless if X (1 ldist0) = 0, in which case MaxLive is 5. Using the de nition of X , in Equation (4.14), forcing X (1 ldist0) to 0 corresponds to forcing 1 ldist0 to be false, namely: 1 > ldist0 = dist(row0; rowlast0 ) Thus, forcing X (1 ldist0) to 0 constrains the last use of vr0 to be in row 0 since, otherwise, ldist0 is strictly larger than 0. As a result, we have just found one forcing candidate, namely last0 =a2, since it is a use-operation of vr0 that is assigned to row 0; therefore, forcing a2 to be the last use results in a fractional MaxLive of 5. From the above description, we can see that the fewer equations in the system, the fewer the forcing combinations that have to be investigated by an LP-solver. Therefore, simplifying the system of equations is crucial to obtaining an ecient algorithm. We list here some rules that can be used to prune the system of equations without compromising the optimality of the solution. All rules will use a stylized version of 119
Equation (4.22):
fracr = cr + X1 + : : : + Xnr
(4.23)
where the constant cr represents the number of minimal fractional parts in row r and the X terms represent the nr distinct conditions for the potential fractional parts in row r. If only solutions with fractional MaxLive T are interesting, then the following set of constraints apply:
fracr = cr + X1 + : : : + Xnr T
8r 2 [0; II )
(4.24)
However those that are trivially satis ed or covered may be pruned, as shown by the following rules.
Rule 4.1 (Partial Evaluation of X Terms) Consider a multiuse virtual register vri for which none of its def-use edges belong to an underlying cycle of the dependence graph. In this case, we can statically evaluate each of the X (dist(rowi ; r) ldisti) terms associated with vri , because the corollary of Theorem 4.4 indicates that reducing the fractional part of a multiuse virtual register lifetime none of whose def-use edges are in an underlying cycle necessarily increases its integral part. Statically evaluating X terms can be eliminated from all fracr expressions, with a corresponding adjustment in cr .
Rule 4.2 (Noncritical Row) Consider an equation fracx and a target fractional
MaxLive T :
fracx = cx + X1 + : : : + Xnx T Equation fracx can be safely eliminated if cx + nx T holds.
120
Proof. Since even in the worst case scenario, where all X1 : : : Xnx evaluate to 1, the number of fractional parts in row x would still be no larger than the target fractional MaxLive, T . Thus this constraint is trivially satis ed. 2
In our example, with a target fractional MaxLive of 5, this rule results in the elimination of equations frac0 and frac2 in Equation (4.22).
Rule 4.3 (Included Row) Consider two equations fracx and fracy where the X terms of equation fracx are a subset of the X terms in equation fracy , namely: fracx = cx + X1 + : : : + Xnx fracy = cy + X1 + : : : + Xnx + Xnx +1 + : : : + Xny Equation fracy can be safely removed if cy + ny cx + nx holds, since equation fracx is more constraining.
Proof. We demonstrate here that the number of fractional parts of the lifetime in
row y is guaranteed to be smaller than the number in row x, and therefore enforcing equation fracx is sucient. Assuming the worst case scenario where all the ny ? nx terms Xnx+1 + : : : + Xny evaluate to 1, we can write:
fracy cy + X1+; : : : ; +Xnx + (ny ? nx) Using the condition cy + ny cx + nx, we can in turn bound the left hand side of the previous inequality as follows:
fracy cy + X1 + : : : + Xnx + (ny ? nx) cx + X1 + : : : + Xnx = fracx Therefore, satisfying fracx necessarily satis es fracy . 2 We can now describe an algorithm that computes all forcing combinations that result in lower fractional MaxLive. This algorithm speci es Step 3 of Algorithm 4.2 in more detail. 121
Algorithm 4.3 The set of forcing combinations that results in a target fractional
MaxLive of T or less are computed as follows:
3.1 Build the fracr equations for each row of the MRT using Equation (4.18). 3.2 Perform the partial evaluation of all possible terms using Rule 4.1 and then eliminate noncritical and included fracr equations using Rules 4.2 and 4.2, for the target fractional MaxLive, T . 3.3 Build the set of all forcing combinations that satis es the target fractional MaxLive as follows. We initialize the current forcing combinations to be empty. We pick a fracr equation and select the minimum number of X terms that must be 0 to satisfy the target fractional MaxLive, T . We record the forcing candidates associated with each of the X terms that are set to 0 in the current forcing combination. We then proceed similarly with each fracr equation in turn. Eventually, when each fracr equation has been satis ed, we have obtained the rst forcing combination, which we store. We then backtrack, unselect a X term, remove the forcing candidates corresponding to this unselected X term from the current forcing combination, select a di erent X term, and add the forcing candidates corresponding to the newly selected X term to form another forcing combination, and store it. Proceed in this fashion until all combinations of X in all fracr equations have been exhausted.
Note that Algorithm 4.3 is rst used with the minimum possible T , and once each forcing combination for a given T has been investigated, we increment T by 1 and repeat until an optimal solution has been found by Algorithm 4.2.
122
4.4 Stage Scheduling Heuristics We now investigate fast and ecient heuristics for stage scheduling. The heuristics reduce the register requirements of a modulo schedule by reassigning some operations to di erent stages, i.e. by shifting operations by multiples of II cycles. The input for these heuristics is a valid initial modulo schedule, and the output is a valid modulo schedule with reduced register requirements. First, we formalize the input to the stage scheduling heuristics and present the running example used to illustrate our techniques in Section 4.4.1. In Section 4.4.2, we describe the basic stage scheduling transforms that are used by the stage scheduling heuristics to modify the stage in which operations are scheduled. We then present the four stage scheduling heuristics considered in this chapter, in Sections 4.4.3, 4.4.4, 4.4.5, and 4.4.6. We summarize in Section 4.4.7 by illustrating the e ect of these heuristics, and combinations of them, on our running example.
4.4.1 Input to Stage Scheduling Heuristics The initial modulo schedule is given by its initiation interval II and the time at which each operation of the rst loop iteration is scheduled. Because the stage scheduling heuristics operate on the dependence graph annotated by the additional skip factors (i.e. the pi;j factors), we rst illustrate how to derive the pi;j factors associated with the initial modulo schedule. We derive here the initial pi;j associated with edge (i; j ), given II and the scheduling time of operations i and j in the initial schedule, timei and timej , respectively. Recall that the time interval along edge (i; j ) is de ned in Equation (4.2) as follows:
timej ? timei = rdisti;j + (si;j + pi;j ) II By isolating pi;j and substituting the right hand side of Equation (4.4) for si;j , we
123
obtain:
pi;j
& ' time l j ? timei ? rdisti;j i;j ? rdisti;j = ? + !i;j II II
Since ?dxe = b?xc, we may transform the above equation to: $ % rdist time i;j ? li;j j ? timei ? rdisti;j + pi;j = + !i;j II II Note that the rst term of the right hand side of this equation is guaranteed to be integer valued since rdisti;j is de ned as (rowj ? rowi ) mod II , in Equation (4.1). Thus, the (timej ? timei ? rdisti;j )=II term can be safely introduced inside the oor function of the second term. After canceling the rdisti;j terms, we obtain the following equation for the initial pi;j associated with edge (i; j ), given the scheduling time of operations i and j in the initial schedule: % $ time j ? timei ? li;j + !i;j (4.25) pi;j = II The input to the stage scheduling heuristics simply corresponds to the dependence graph of the loop annotated by the initial pi;j values, as computed by Equation (4.25).
Example 4.5 We will illustrate our techniques using the following kernel as a
running example: z[i] = SQRT(x[i]/ (1/x[i]+x[i]+a)) +(y[i]+a) *(y[i]+b) *(y[i]+c), computed as presented in Figure 4.10a. This example, although constructed to illustrate our heuristics concisely, is representative of several kernels found in our benchmark suite. A modulo schedule with II = 6 for our target machine is presented in Figure 4.10b. This schedule is a good example of a schedule produced by a register insensitive modulo scheduler: it results in a schedule with high throughput and provides a good stage schedule along the critical path (add0, load1, div3, add4, div5, sqrt6, add7, store8) so as to minimize the schedule length of the loop in the typical fashion. However, operations not on the critical path tend to be scheduled too early, e.g. operation add9 is scheduled at time 5 when it could be scheduled as late as time 29. This e ect is due to the fact that current modulo schedulers favor 124
scheduling operations early to prevent needlessly increasing either II or the schedule length. Stage scheduling will thus make a signi cant di erence unless the number of stages or the number of operations not on the critical path are too small. Trying different MRTs for a given loop iteration is also likely to become increasingly important as II increases. The dependence graph in Figure 4.10a has three self-loops on the address arithmetic nodes (add0, add9, and add10) and four elementary underlying cycles in the underlying graph G = fV; Esched [ Ereg g. The dependence graph for Example 4.5 is representative of the ones found in our benchmark suite, as on average more than 50% of the operations of a loop iteration belong to an underlying cycle. The values of nonzero pi;j are shown as label on their respective edges. Because we describe a stage schedule using time-di erences associated with edges instead of absolute times associated with operations, we must ensure that the underlying cycle constraint, de ned in Equation (4.6), holds for each elementary underlying cycle. Since we are given a valid modulo schedule, the constant term de ned in Equation (4.7) can be quickly computed as the sum of the pi;j from the given modulo schedule, where a pi;j is taken as negative when its edge is traversed in the reverse direction. These constant terms are shown in Figure 4.10a inside the curved dashed arrows.
4.4.2 Stage Scheduling Transforms The rst transform modi es the additional skip factors associated with cut edges, where a cut edge is an edge in the dependence graph that, if removed, cuts the graph into two disconnected parts. Note that an edge is a cut edge if and only if it does not lie in a cycle of the underlying dependence graph. We proved in Theorem4.2 that the additional skip factors of a dependence graph without underlying cycles can be individually minimized by setting all their values to zero. We apply this theorem here to the parts of the underlying dependence graph that are acyclic. 125
a) Dependence graph with additional skip factors
b) Schedule (II=6) Time:
a9
a12
a10
a0
l11
l1
a13
a2
a14
1
0
d3
−1
−1
a4
m15
d5
m16 r6
0: 1: 2: 3: 4: 5: * 10: 11: * 15: * 19: * 29: 30:
Operations a0, a10; l1; a2, d3, l11; a12, a13; a14; a9; a4; d5; m15; r6, m16; a7; s8;
long latency operations mult: m15, m16 (4 cycles latency) div: d3,d5 (8 cycles latency) sqrt: r6 (10 cycles latency)
a7
s8
time register and scheduling edge (bold: edge in underlying cycle)
short latency operations add: a0,a2,a4,a7, a9,a10,a12, a13,a14 (1 cycle latency) load: l1,l11 (1 cycle latency) store: s8 (1 cycle latency)
non−zero additional skip factor cumulative additional skip factor along an underlying cycle
Figure 4.10: Initial stage-schedule with additional skip factors (Example 4.5).
126
Theorem 4.5 (Acyclic Transform) Setting to zero the additional skip factor of a
cut edge of the underlying dependence graph results in a valid stage schedule.
Proof. To guarantee that the acyclic transform results in valid stage schedules, we
must show that each pi;j remains nonnegative and that the sum of the pi;j along each underlying cycle of the dependence graph remains unchanged. By de nition, a cut edge cannot be part of any underlying cycle as, otherwise, the removal of that edge would not result in an unconnected underlying graph. As a result, this transform does not modify the value of the additional skip factors along any underlying cycles. Consequently, the sum of the pi;j along each underlying cycle remains constant and each pi;j remains nonnegative. 2 a) Initial schedule
b) Modified schedule u1
u1 j1
j2
j1
j2
i u2
k1
i
k2
u3
u2
k3
u4
k1
k2
u3
k3
u4
underlying cycle additional skip factor register and scheduling edge
Figure 4.11: Up-propagation transform. The second transform shifts some of the additional skip factors associated with the output edges of a vertex to the input edges of that vertex. Figure 4.11 illustrates this transform, where operation i is scheduled three stages later, i.e. three units of additional skip factor are propagated up, from the output edges to the input edges of operation i. 127
Theorem 4.6 (Up-Propagation Transform) Reducing the additional skip factor of each output edge of a vertex by the minimum additional skip factor among these edges and increasing the additional skip factor of each input edge by the same amount results in a valid stage schedule.
Proof. By decreasing the pi;j of each output edge of vertex i by the minimum pi;j
among these edges, the pi;j values are guaranteed to remain nonnegative. Since the pi;j of each input edge is increased by the same amount as the decrease of each output edges, these pi;j values remain nonnegative and the sum of the pi;j along each underlying cycle is guaranteed to remain constant. 2 A similar transform, down-propagation, shifts the additional skip factor down instead of up, and is obtained by interchanging the words input and output in the description of the up-propagation transform.
4.4.3 Acyclic Heuristic (AC) The rst heuristic reduces the register requirements associated with edges that belong to acyclic parts of the underlying dependence graph. This heuristic has a linear computational complexity in the number of edges and results in valid stage schedules with no larger register requirements. First, the heuristic detects all cut edges, using a linear-time algorithm that enumerates the biconnected components of an undirected graph [1, pp. 179{187], i.e. the underlying cycles of the dependence graph. Cut edges are then simply the edges linking vertices from distinct biconnected components. Note that self-loop edges can safely be ignored during the stage scheduling process, because they do not introduce any scheduling constraints for the stage scheduler2. The heuristic then performs the constant-time acyclic transform on each of the cut edges. Since acyclic transforms only decrease the pi;j , and since the register 2 The scheduling constraints generated by self-loop edges simply yield a lower bound on II and, since the stage scheduling heuristics do not change II , these constraints (and edges) can be safely ignored during stage scheduling.
128
requirements associated with a stage schedule is the sum of the skip factors of the last use-operations plus other unrelated terms, the register requirements cannot increase. This heuristic highlights the advantage of describing a stage schedule using skip factors associated with edges instead of absolute scheduling times associated with operations. In Figure 4.10a there are 2 cut edges with nonzero pi;j . Setting p16;7, the additional skip factor between operations mult16 and add7, to 0 e ectively delays mult16 and all its predecessors by II cycles. This simple local change thus reschedules operations add10, load11, add12, add13, add14, mult15, and mult16 one stage later. Similarly, setting p9;8 to 0 delays add9 by four stages. Because the additional skip factor of a cut edge can always be set to 0, without a ecting any of the underlying cycles and regardless of the rest of the dependence graph, the AC heuristic will be used rst, and subsequent heuristics will ignore both cut edges and self-loop edges. All the following heuristics will thus operate only on the nontrivial cyclic parts of the underlying graph. In Figure 4.10a, there are two nontrivial biconnected components, one containing the operations load1, add2, div3, add4, and div5, and the other containing the operations load11, add12, add13, add14, mult15, and mult16. The edges of these components are shown in bold in Figure 4.10a.
4.4.4 Sink/Source Heuristic (SS) The second heuristic addresses single vertices that are scheduled too early or too late. It considers only sink vertices, i.e. vertices with no output edges, and source vertices, i.e. vertices with no input edges. This heuristic has a linear computational complexity in the number of edges and results in valid stage schedules with no larger register requirements. First, the heuristic detects all sink and source vertices, ignoring cut edges and selfloop edges. Then, the heuristic applies the up-propagation transform to all source vertices, visiting each edge at most once. Since a source vertex has no input edges, the up-propagation transform only decreases the additional skip factors of its output 129
edges. Thus, the register requirements cannot increase. Similarly, the heuristic applies down-propagation to all the sink vertices. In Figure 4.10a, load1 and load11 are source vertices and div5 and mult16 are sink vertices. However, as the minimum pi;j in each of the four cases is 0, the SS heuristic does not cause any change in this example.
4.4.5 Up Heuristic (UP) We now present a technique that enhances the impact of the SS heuristic by propagating the additional skip factors upward in the dependence graph. This heuristic has a linear computational complexity in the number of edges, and results in valid stage schedules. Note that this heuristic may increase the register requirements of a schedule, although that rarely occurs in our benchmark3. a) Initial Schedule
b) Improved Schedule
l11 l11 a12
a13
a14 a12
a13
m15
m15
a14
m16
m16
time
register and scheduling edge additional skip factor
Figure 4.12: UP propagation heuristic. We illustrate this heuristic by investigating one of the biconnected components of our running example, shown in Figure 4.12a. As previously mentioned, the SS In our benchmark suite of 1327 loops, the UP heuristic increased the register requirements of only three loops, and those by no more than 2. 3
130
heuristic cannot reduce the additional skip factors as both the minimum pi;j among the output edges of the source operation, load11, and the minimum pi;j among the input edges of the sink operation, mult16, are 0. However, p12;15, p13;15, and p14;16 can each be reduced by 1 without modifying the sum of the pi;j along the two underlying cycles and without resulting in negative pi;j values. This heuristic proceeds by applying the up-propagation transform (ignoring selfloop edges and cut edges) to the vertices of the dependence graph, in reverse topological order. In Figure 4.12a, the transform can be applied to operations add12, add13, and add14, and then to load11. Figure 4.12b shows the additional skip factors after completion of this heuristic. For strongly connected components, it may be advantageous to use this heuristic several times. A similar heuristic may be constructed using the down-propagation transform with the normal topological order.
4.4.6 Register Sink/Source Heuristic (RSS) This last heuristic is based on removing redundant register edges in the dependence graph, which helps make improvements of a stage schedule more apparent. This heuristic has a quadratic computational complexity in the number of register edges, as shown in Section 3.5, and results in valid stage schedules with no larger register requirements. Figure 4.13 illustrates the importance of removing redundant register edges, using the second biconnected component of Figure 4.10a. Figure 4.13a illustrates the dependence graph for this component before the elimination of redundant register edges. In this graph, the cumulative additional skip factor of the leftmost underlying cycle must be equal to 1. For example, either p1;2 or p2;4 may be 1, with the other pi;j = 0. Without careful inspection, one may conclude that either solution results in the same register requirements, as reducing one lifetime increases the other, since both p1;2 and p2;4 are associated with scheduling and register edges. However, by using the redundant edge removal technique presented in Section 3.5, we discover that the register edges (load1, add2) and (load1, div3) are redundant, 131
a) Initial schedule
b) Improved schedule
l1
l1
a2
d3
d3 a2
a4
a4
d5
time
d5
scheduling edge register edge additional skip factor
Figure 4.13: Register source heuristic example. as the register requirements generated by the register edge (load1, div5) cover the register requirements of these two edges. It is then apparent that minimizing p2;4, rather than p1;2, results in lower register requirement, since p1;2 is no longer associated with a register edge, as shown in Figure 4.13b. First, the heuristic removes all redundant edges, which has a quadratic computational complexity in the number of edges if the MinDist relation is provided. Computing the MinDist relation [49] corresponds to an all-pairs longest-path problem which has a cubic computational complexity in the number of operations (in each strongly connected component). However, this relation may be readily available, as some modulo schedulers use this relation to determine the minimum initiation interval. Then, the RSS heuristic applies the up-propagation transform to each reg-source vertex, i.e. vertices without input register edges (ignoring self-loop edges and cut edges). Although the up-propagation transform may increase the additional skip factor along the input edges, we know that none of these edges contributes to the overall register requirements. Thus, the register requirements cannot increase. We then apply the down-propagation transform to the reg-sink vertices, i.e. those without output register edges. 132
b) Improved schedule (II=6)
a) Dependence graph with improved additional skip factors
Time:
a0
+ 2*II l1 a10
d3 l11 a2
+ II a12
RSS,UP
a13
UP
a4
m15
a14
d5
+ 3*II
AC
+ II
0: 1: 2: * 8: * 11: 12: * 14: 15: 16: * 19: * 21: 22: * 25: * 29: 30:
Operations a0; l1; d3; a2; d5; a10; l11; a12,a13; a14; r6; m15; a4; m16; a7,a9; s8;
r6
m16
a9
a7
+ 4*II AC
register and scheduling edge (bold: edge in underlying cycle)
s8
scheduling edge only (bold: edge in underlying cycle)
time
non−zero additional skip factor
Figure 4.14: Improved stage-schedule (Example 4.5).
133
4.4.7 Combining Heuristics To summarize the bene ts of the stage scheduling heuristics presented in Section 4.4, we consider the schedules of Example 4.5 presented in Figures 4.10b and 4.14b. The new stage schedule in Figure 4.14 delays one operation by 4 stages (add9), one operation by 3 stages (add14), four operations by 2 stages (add10, load11, add12, add13), and three operations by 1 stage (add2, mult15, mult16), reducing the register requirements by 9, i.e. a reduction of 4 for vr9, 2 for vr14, 1 for vr2, vr12, vr13, and vr16, and an increase of 1 for vr11. Figure 4.14a also shows which stage scheduling heuristics produce each modi cation in the dependence graph. Combining the stage scheduling heuristics is advantageous, as several of these address distinct parts of the dependence graph. For example, using the AC heuristic in conjunction with the other stage scheduling heuristics is very advantageous as the AC heuristic focuses on the acyclic parts of the underlying dependence graph whereas the other heuristics focus on the cyclic parts of the underlying dependence graph. Using both the UP and the RSS heuristic is advantageous, as they address distinct features of underlying cycles. Also, in graphs with strongly connected components, it may be advantageous to use the UP heuristic several times, as the propagation of the pi;j values may require more than one pass before being reduced in the presence of strongly connected components. Note also that since the SS heuristic is fully covered by the RSS heuristic, there is no gain in using the SS heuristic if the RSS heuristic is used.
4.5 Schedule-Length Sensitive Stage Scheduling Since the stage scheduling approach preserves the initiation interval, it reduces the register requirements without lowering the steady-state throughput of a modulo schedule. However, the overall performance of a modulo schedule is also function of its transient performance, which is directly proportional to the schedule length (SL) of an iteration. And since the stage scheduling algorithms presented in Sections 4.3 and 4.4 modify the stages of the operations, these algorithms may increase SL, and 134
Schedule−length insensitive stage−scheduling
Schedule−length sensitive stage−scheduling
a) Initial schedule
c) Initial schedule
b) Modified schedule
d) Modified schedule
start
l0
l4
l0
l0
start
l4
l0
m1
d5
m1
m1
d5
l4
m1
l4
a2
d5
s6
s3
a2
a2
d5
s3
s6
s3
a2
s6
s3
stop
s6
stop
time operations
pseudo operations
scheduling edge
non−zero additional skip factor
register and scheduling edge
Figure 4.15: Stage scheduling and schedule length (Example 4.6). thus degrade the overall performance of the schedule. In this section, we present a technique that alters the original dependence graph such that stage-scheduling algorithms may not increase the schedule length of an iteration when searching for a schedule with reduced register requirements.
Example 4.6 We illustrate the e ect of stage scheduling on the schedule length
using the following example: z[i] = 3*x[i]+y[i] and w[i] = 1/y[i], computed as presented in Figure 4.15. A modulo schedule for this example is shown in Figure 4.15a, where each operation is scheduled as early as possible to minimize the schedule length. This schedule does not minimize the register requirements, as the value de ned by the load4 operation is alive for three stages before being used by the add2 operation, i.e. p4;2 = 3. Since there is no underlying cycle in the dependence graph, a schedule 135
with minimum register requirements is simply obtained by setting all the additional skip factors (pi;j ) to 0, as shown in Figure 4.15b. It is clear from this gure that while this transformation minimizes the register requirements, it may also increase the schedule length of the modulo schedule. The main idea in achieving schedule-length sensitive stage-scheduling is to introduce additional scheduling constraints into the dependence graph such that the schedule length may not be increased by the scheduling algorithm without violating some scheduling constraints. To express these additional constraints easily, we introduce a start and a stop vertex which are, respectively, a predecessor and successor to every other vertex in the dependence graph. The additional scheduling constraints are then simply formulated by an additional scheduling edge from the stop vertex to the start vertex associated with the latency lstop,start = ?(SL ? 1), where SL is the desired schedule length. In general, the start and stop vertices must each be connected to every other vertex of the dependence graph; however, super uous scheduling edges can be removed using Theorem 3.2. The latencies associated with these additional edges vary depending on the de nition of the schedule length. We consider here the scheduling length as the distance from the cycle in which the earliest operation is scheduled to the cycle in which the computation of the last value is completed (i.e. its schedule cycle + its latency). Thus, for each operation i, the latency of edge (start; i) is set to 0 and the latency of edge (i; stop) is set to the latency of operation i. The modi ed dependence graph associated with Example 4.6 is shown in Figure 4.15c. Note that each operation is now part of an underlying cycle. In Figure 4.15c, we may delay the store6 operation by one stage without increasing the schedule length, since p6;stop = 1. This fact can be exploited by a stage-scheduler to reduce the lifetime along edge (4; 2) by one stage, i.e. by II cycles, as shown in Figure 4.15d. Note that this solution has the same SL as the initial schedule in Figure 4.15a, but that the modi ed schedule in Figure 4.15b achieves a greater reduction in register requirements by increasing SL. 136
4.6 Measurements In this section we investigate the register requirements of the integer and oating point register les for a benchmark of 1327 loops obtained from the Perfect Club [13], SPEC-89 [91], and the Livermore Fortran Kernels [65] compiled for the Cydra 5 as described in Section 2.7. We use here the Iterative Modulo Scheduler [78] to produce high quality MRT-schedules, which are used as input to both the optimal stage scheduling algorithm and the stage scheduling heuristics. Thus, in this section, every schedule of a loop shares the same MRT and di ers only by the stage in which each operation is placed. The performance of the following stage scheduling algorithms is investigated here.
MinReg Stage Scheduler. This stage scheduler minimizes MaxLive, the maximum number of live values at any single time in the loop schedule, over all valid modulo schedules that share a given MRT. The resulting schedule has the lowest achievable register requirements for the given machine, loop, and MRT. For dependence graphs with underlying cycles, the complexity of this scheduler may require the solving of an exponential number of LP-problems.
MinBu Stage Scheduler. This stage scheduler minimizes the integer MaxLive over all valid modulo schedules that share a given MRT. The resulting schedule has the lowest achievable bu er requirements for the given machine, loop, and MRT. We present here the register requirements associated with these schedules in our comparisons. For dependence graphs with underlying cycles, the complexity of this scheduler requires the solving of a single LP-problem.
Stage Scheduling Heuristics. The stage scheduling heuristics used in this section
are all based on the four basic heuristics presented in Section 4.4, namely Acyclic (AC), Sink/Source (SS), Up (UP), and Register Sink/Source (RSS) heuristics. When combined, the name of the stage scheduling heuristic simply lists the basic heuristics in the order in which they are used. For example, the stage scheduling heuristic AC+3*UP consists of the one call to the Acyclic heuristic followed by three calls to the Up heuristic. 137
The register requirements of the schedules produced by the above algorithms are investigated for loops with unlimited schedule length in Section 4.6.1 and with bounded schedule length in Section 4.6.2. We then investigate the e ect of increasing II on the register requirements in Section 4.6.3. To facilitate comparison with previous work, the MRT-schedules used in Sections 4.6.1 and 4.6.2 correspond to the schedules computed by Rau and used in [78]. In Section 4.6.3 however, we use our implementation of the Iterative Modulo Scheduler to compute the required MRTschedules, since we also need MRT-schedules with larger initiation intervals.
4.6.1 Schedules with Unlimited Schedule Length We rst investigate the register requirements of stage schedules with unlimited schedule length. The characteristics of the input data dependence graphs and MRTschedules for the 1327 loop benchmark are shown Table 4.2. Measurements: II II=MII operations operations in underlying cycles underlying cycles register edges: without redundant edges scheduling edges: without redundant edges
min 1:00 1:00 2:00 0:00 0:00 1:00 1:00 1:00 1:00
freq median average max 28:6% 3:00 11:51 165:00 96:0% 1:00 1:01 1:50 0:4% 10:00 17:54 161:00 42:4% 4:00 10:18 158:00 55:4% 0:00 3:01 66:00 0:4% 12:00 22:30 220:00 0:4% 11:00 21:35 220:00 0:4% 14:00 25:79 345:00 0:4% 12:00 22:47 232:00
Table 4.2: Characteristics of the 1327-loop benchmark suite for schedules with unlimited schedule length. The data most relevant to the computational complexity of the optimal stage scheduling algorithm is the number of operations that belong to some nontrivial underlying cycle, the number of register edges, and the initiation interval. Table 4.2 indicates that, in this regard, the 1327-loop benchmark contains a wide range of loops, including loops with up to 158 operations in underlying cycles, 220 register edges, and 138
an II of up to 165 cycles. The high quality of the MRT-schedules is indicated by the fact that 96.0% of the loops scheduled by the Iterative Modulo Scheduler [78] achieve MII . Notice the fraction of operations in underlying cycles. If the workload is \equal number of iteration" from each loop in the benchmark, 57.9% of the operations belong to some underlying cycle. If the workload is \equal number of nodes" from each loop, 37.4% of the operations belong to an underlying cycle. The di erence between these two percentages occurs because smaller loops have a lower percentage of their operations in underlying cycles. If operations with self-loops are also considered as underlying cycles, these percentages become 76.3 and 62.5%, respectively. Table 4.2 also indicates the e ectiveness of the dependence graph transformation presented in Section 3.5: when redundant register and scheduling edges are eliminated from the dependence graphs, the average number of edges decrease by 4.3 and 12.9%, respectively. When searching for optimal stage schedules with unlimited schedule length, MinBu Stage-Scheduler nds a schedule with minimum integral MaxLive for all 1327 loops. Similarly, MinReg Stage-Scheduler nds a schedule with minimum MaxLive for 1296 of the 1327 loops; for the remaining 31 loops, the scheduler exceeded an upper limit of 512 augmented problems solved in Algorithm 4.2, Step 4. Our measurements include these loops as well, for which we report the schedules with the lowest register requirements among all stage-schedules considered. Our rst measurement investigates the register requirements achieved by the MinReg Stage-Scheduler, which results in the lowest register requirements over all valid modulo schedules that share a common MRT, and compares the results to a registerinsensitive scheduling algorithm and a schedule independent lower bound. This measurement provides insight regarding the range of decrease in register requirements that can be achieved by the stage scheduling approach. For a machine with any given number of registers, Figure 4.16 presents the fraction of the loops in the benchmark suite that can be scheduled without spilling and without increasing II . In this graph, the X-axis represents MaxLive and the Y-axis represents 139
1 0.9 Fraction of loops scheduled
0.8 0.7 0.6 0.5 0.4 0.3
Schedule-Independent Lower Bound
0.2
MinReg Stage-Scheduler
0.1
Iterative Modulo Scheduler
0 0
16
32
48
64
80 96 MaxLive
112
128
144
160
Figure 4.16: Register requirements in the 1327-loop benchmark suite. the fraction of loops scheduled. Thus, for example, if 32 registers were available, the MinReg Stage-Scheduler could schedule 58% of the loops without spilling and without increasing II . Alternatively, to schedule 80% of the loops successfully with the chosen II and no spilling, the MinReg Stage-Scheduler needs at least 45 registers. The second curve in Figure 4.16, labeled \Schedule Independent Lower Bound," corresponds to MinAvg in [49] and is a lower bound on the register requirements formed by assuming a machine with unlimited issue width and unlimited functional resources. The lower bound is obtained by computing the MinDist relation and by assuming that no register lifetime spans more than the minimum scheduling distance (i.e. MinDist) between its def and its use-operations. Notice the gap between the MinReg Stage Scheduler curve and the Lower Bound curve, which we believe is caused by two factors. First, the MinReg Stage Scheduler searches for a schedule with the lowest register requirements only among the modulo schedules that share a given MRT. Therefore, when the MRT given to the stage 140
1 0.9 Fraction of loops scheduled
0.8 0.7 0.6 0.5 0.4
Best Heuristic
0.3
MinBuff Stage-Scheduler
0.2
Simple Heuristic
0.1
Iterative Modulo Scheduler
0 0
4
8
12
16
20 24 28 32 36 Additional MaxLive
40
44
48
52
Figure 4.17: Additional register requirements relative to the MinReg Stage-Scheduler. scheduler is suboptimal, the stage scheduler will nd the fewest registers required to support that MRT, which may be signi cantly larger than the absolute minimum register requirements over all MRTs for that loop4. The second factor is that the schedule-independent lower bound may be signi cantly too optimistic due to the complexity of some of the loops in the benchmark suite and the fact that this lower bound assumes unlimited issue width and unlimited functional resources. The third curve presents the register requirements of the Iterative Modulo Scheduler, which presently does not attempt to minimize the register requirements. The gap between these last two curves indicates the degree of improvement that stage scheduling heuristics can potentially achieve, i.e. they should do better than the Experimental evidence on a subset of this benchmark suite, consisting of 920 loops with up to 41 operations and II up to 118, for which an optimal modulo schedule was sought over all MRTs (see Chapter 5) showed, however, that the di erence between the local and absolute minimum was surprisingly small. This result may not hold for larger loops. 4
141
Iterative Modulo scheduler and cannot do better than the MinReg Stage-Scheduler. Our second measurement illustrates the additional registers needed when scheduling algorithms other than the MinReg Stage Scheduler are used. This measurement allows us to precisely evaluate the performance, in register requirements, of the various stage scheduling heuristics investigated in this chapter. From top to bottom, the curves in Figure 4.17 present the additional registers needed by the Best Heuristic5, MinBu Stage-Scheduler, a Simple Heuristic6, and the Iterative Modulo Scheduler. The rst interesting observation is that the Best Heuristic has lower additional register requirements than the MinBu Stage-Scheduler. As indicated in Figure 4.17, the Best Heuristic nds a schedule with no additional register requirements in 95.9%, versus 92.5% for the MinBu Stage-Scheduler; also, the Best Heuristic schedules 98% of the loops with 1 additional register, versus 3 for the MinBu Stage-Scheduler. This observation can be explained by the fact that, since the MinBu Stage-Scheduler only minimizes integral MaxLive, it arbitrarily picks one of the stage schedules with minimum integral MaxLive, disregarding fractional MaxLive. As indicated by the measurements in Figure 4.17, this arbitrary pick results in schedules with higher register requirements than the schedules found by the Best Heuristic. The second observation is that even a simple heuristic signi cantly decreases the register requirements, below those of a register-insensitive scheduler. In Figure 4.17, the Simple Heuristic nds a stage schedule with no additional register requirements in 80.0% of the loops, and schedules 95% of the loops with no more than 9 additional registers. The Iterative Modulo Scheduler results in a stage schedule with no additional register requirements in 42.0% of the loops, a surprisingly high number for a scheduler that does not even attempt to minimize the register requirements. This result can be explained in part by the fact that this scheduler attempts to minimize the length of a schedule, which generally results in a stage schedule with low register requirements along the critical path of that schedule. However, the Iterative Modulo The best heuristic (AC+3*UP+RSS) uses the AC heuristic, followed by three passes of the UP heuristic, followed by the RSS heuristic. Additional passes of the UP heuristic only marginally decrease the register requirements. 6 The simple heuristic (AC+SS) uses the AC heuristic followed by the SS heuristic. 5
142
1 0.9 Fraction of loops scheduled
0.8 0.7 Heuristic AC+3*UP+RSS
0.6
Heuristic AC+3*UP
0.5
Heuristic AC+UP
0.4
Heuristic AC+RSS 0.3
Heuristic AC+SS
0.2
Heuristic AC
0.1
Iterative Modulo Scheduler
0 0
1
2
3
4
5 6 7 8 Additional MaxLive
9
10
11
12
Figure 4.18: Additional register requirements of stage scheduling heuristics. Scheduler needs up to 23 additional registers to successfully schedule 95% of the loops, indicating that more than 5% of the loops have a signi cant number of operations that are not on the critical path and are scheduled too early, or too late. Our third measurement investigates in more detail the additional register requirements of several stage scheduling heuristics. To facilitate the reading of Figure 4.18, the heuristics are listed in the legend in order of decreasing performance. The bottom curve corresponds to the performance of the Iterative Modulo Scheduler. The next curve corresponds to the AC heuristic, which nds a stage schedule with no additional register requirements in 73.9% of the loops, and schedules 95% of the loops with no more than 10 additional registers. This signi cant decrease in register requirements due to the AC heuristic is solely obtained by moving the unmodi ed schedules of the biconnected components of the underlying dependence graph to be as close to one another as possible. To decrease the register requirements of the modulo scheduled loops further, the schedule within each biconnected component must also be 143
1 0.9 Fraction of loops scheduled
0.8 0.7 0.6 0.5 0.4 Schedule-Independent Lower-Bound
0.3
Iterative Modulo Scheduler
0.2
Heuristic AC+3*UP+RSS 0.1
MinReg Stage-Scheduler
0 0
10
20
30
40 50 Schedule Length
60
70
80
Figure 4.19: Schedule length of one loop iteration in the 1327-loop benchmark suite. considered. The next curve corresponds to the AC+SS heuristic, which also reschedules those vertices in the dependence graph that have no input edges or no output edges (ignoring self-loop and cut edges). The AC+SS heuristic results in lower register requirements than the AC heuristic in the range from 1 to 10 additional registers and is nearly indistinguishable after that. The next curve corresponds to the AC+RSS heuristic, which performs better than the two previous heuristics in the range from 1 to 10 additional registers. This heuristic requires the removal of redundant register edges, which is accomplished in quadratic time, provided that the MinDist relation is available in a lookup table. We notice that the AC, AC+SS, and AC+RSS heuristics each schedule approximately 95% of the loops with no more than 10 additional registers. To improve the register requirements further, we must more aggressively address the biconnected components using heuristics based on the up-propagation transform. The last three mostly overlapped curves of Figure 4.18 correspond to the per144
formance of the AC+UP, AC+3*UP, and AC+3*UP+RSS heuristics. These three heuristics perform signi cantly better, nding an optimal stage schedule for 92.9%, 95.0%, and 95.9% of the loops and scheduling 98% of the loops with no more than 3, 1, and 1 additional registers, respectively. Although these three curves mostly overlap, the AC+3*UP+RSS heuristic does perform slightly better, with additional register requirements of 94 registers, instead of 110 (AC+3*UP) and 241 (AC+UP), summed over all 1327 loops of the benchmark suite. Our next experiment investigates the impact of the stage scheduling algorithms on the schedule length (SL), which determines the transient performance of the modulo schedules. The distribution of SL is shown in Figure 4.19 for the Iterative Modulo Scheduler, the MinReg Stage-Scheduler, and the AC+3*UP+RSS heuristic. The graph also includes a lower bound on the schedule length of a schedule, labeled \Schedule-Independent Lower Bound," that corresponds to the MinDist relation between the start and stop vertices, as de ned in Section 4.5. Comparing the schedule length of the MinReg Stage-Scheduler to the Iterative Modulo Scheduler, the stage scheduler reduces SL by 1 cycle for 2 out of 1327 loops, i.e. for 0.2% of the loops. It results in the same SL in 63.7% of the loops, increases SL by 1 cycle in 12.0%, by 2 in 5.8%, by 3 to 10 cycles in 10.2%, and by up to 155 cycles in the remaining 9.1% of the loops, resulting in a median of 0 and an average of 2.72 additional cycles. The average increase of 2.06 additional cycles in schedule length for the AC+3*UP+RSS is slightly smaller. A better metric for the increase in schedule length is the ratio of SL to the schedule length lower bound. This ratio is given in Table 4.3 for the Iterative Modulo Scheduler and each stage scheduling algorithm. Compared to the Iterative Modulo Scheduler, the MinReg Stage-Scheduler increases the average schedule length ratio by 5.6%, while decreasing the average register requirements by 19.8%. Similarly, the stage scheduling heuristics increase the average schedule length ratio by up to 4.6%, while decreasing the average register requirements by up to 19.6%. Ultimately, the best performance metric is the total execution time associated with a schedule, which is a function of II , SL, and pro ling information (i.e. the 145
Measurements: Iterative Modulo Scheduler: MaxLive Schedule Length (ratio) Execution Time (ratio) MinReg Stage Scheduler: MaxLive Schedule Length (ratio) Execution Time (ratio) AC+3*UP+RSS Heuristic: MaxLive Schedule Length (ratio) Execution Time (ratio) AC+3*UP Heuristic: MaxLive Schedule Length (ratio) Execution Time (ratio) AC+UP Heuristic: MaxLive Schedule Length (ratio) Execution Time (ratio) AC+RSS Heuristic: MaxLive Schedule Length (ratio) Execution Time (ratio) AC+SS Heuristic: MaxLive Schedule Length (ratio) Execution Time (ratio) AC Heuristic: MaxLive Schedule Length (ratio) Execution Time (ratio)
min freq median average
max
2:00 0:1% 1:00 45:3% 1:00 52:9%
38:00 1:03 1:00
34:36 166:00 1:08 2:47 1:03 1:48
2:00 0:1% 1:00 38:1% 1:00 48:1%
29:00 1:05 1:00
27:54 133:00 1:15 2:96 1:06 1:69
2:00 0:1% 1:00 38:4% 1:00 48:2%
29:00 1:04 1:00
27:61 133:00 1:13 2:96 1:06 1:69
2:00 0:1% 1:00 38:4% 1:00 48:2%
29:00 1:04 1:00
27:62 133:00 1:13 2:96 1:06 1:68
2:00 0:1% 1:00 38:5% 1:00 48:2%
29:00 1:04 1:00
27:72 133:00 1:13 2:96 1:06 1:68
2:00 0:1% 1:00 38:6% 1:00 48:2%
29:00 1:04 1:00
28:33 135:00 1:13 2:96 1:06 1:68
2:00 0:1% 1:00 38:6% 1:00 48:2%
29:00 1:04 1:00
28:73 135:00 1:13 2:96 1:06 1:68
2:00 0:1% 1:00 38:9% 1:00 48:2%
29:00 1:04 1:00
29:15 135:00 1:12 2:96 1:06 1:68
Table 4.3: Characteristics of the stage scheduling algorithms with unlimited schedule length.
146
total number of times the loop is entered and executed. We investigate here the ratio of the execution time to the execution time lower bound for the 597 loops for which the pro ling data was available to us. The execution times are computed using Equation (3.1), using II and SL to compute the actual execution time of the schedules, and MII and the schedule length lower bound to compute the execution time lower bound. These ratios, shown in Table 4.3, indicate that using the stage scheduling algorithms (with unlimited schedule length) instead of the Iterative Modulo Scheduler results in an increase in the average execution time ratio of 2.9%. Algorithms: Redundant edges: average maximum std dev.
AC+SS
AC+3* MinBu MinReg UP+RSS yes no yes no yes no 0.1 ms 0.4 ms 141.8 ms 112.2 ms 43.9 sec 23.5 sec 3.3 ms 12.7 ms 18691.7 ms 17558.4 ms 7180.0 sec 6495.5 sec 0.1 ms 0.6 ms 950.9 ms 816.6 ms 374.0 sec 268.4 sec
Table 4.4: Run time for the stage-scheduling algorithms on a Sun Sparc-20 workstation. Runtime statistics gathered by executing several stage scheduling algorithms on a SUN Sparc-20 workstation are given in Table 4.4. Notice the decrease in runtime when redundant edges are eliminated from the dependence graph. For example, the MinReg Stage-Scheduler completes in 53.5% of time, on average, when redundant edges are eliminated using the technique described in Section 3.5. Note also that the average run time of both heuristics is 2 to 3 orders of magnitude faster than the average run time of the MinBu Stage-Scheduler and 4 to 5 orders of magnitude faster than the MinReg Stage-Scheduler.
4.6.2 Schedules with Bounded Schedule Length We now investigate the register requirements of stage schedules when the schedule length of a loop iteration is bounded by the schedule length initially obtained by the Iterative Modulo Scheduler. The results presented here are thus indicative of the 147
decrease in register requirements achieved while maintaining, or possibly increasing, the performance of the resulting schedules, since II is kept constant and SL cannot increase. Measurements: II II=MII operations operations in underlying cycles underlying cycles register edges: without redundant edges scheduling edges: without redundant edges
min freq median average max 1:00 28:6% 3:00 11:51 165:00 1:00 96:0% 1:00 1:01 1:50 4:00 0:4% 12:00 19:54 163:00 4:00 0:4% 12:00 19:54 163:00 1:00 3:8% 5:00 11:10 116:00 1:00 0:4% 12:00 22:30 220:00 1:00 0:4% 11:00 21:30 220:00 4:00 0:4% 33:00 59:88 601:00 4:00 0:4% 19:00 32:83 287:00
Table 4.5: Characteristics of the 1327-loop benchmark for schedules with bounded schedule length. The characteristics of the input data dependence graphs (augmented by the start and stop vertices) and MRT-schedules for the 1327 loop benchmark are shown in Table 4.5. The major di erence with respect to the characteristics of schedules with unlimited schedule length shown in Table 4.2, is that each operation is now part of some underlying cycle. Other di erences are the increased number of underlying cycles, 11.10 versus 3.01 cycles, and the increased number of scheduling edges, 32.83 versus 22.47 nonredundant edges, on average. Again, eliminating the redundant register and scheduling edges from the dependence graph is e ective, decreasing the average number of edges by 4.5 and 45.1%, respectively. In this section, the MinBu Stage-Scheduler nds a schedule with minimum integral MaxLive for all 1327 loops, and the MinReg Stage-Scheduler nds a schedule with minimum MaxLive for 1246 of the 1327 loops. For the remaining 88 loops, we report in our measurements the schedules with the lowest register requirements among all stage-schedules considered. It is not surprising that the MinReg Stage-Scheduler nds fewer optimal schedules when the schedule length is bounded since the solver is 148
very sensitive to the number of underlying cycles. Note also that since each vertex is part of an underlying cycle, the AC heuristic is not used here as it impacts only the acyclic parts of the dependence graph. Similarly, the SS heuristic is not used here since it does not apply to vertices in a strongly connected component, where each vertex has at least one input and one output edge. The rst measurement investigates the increase in register requirements that is incurred when algorithms with bounded schedule length are used instead of the optimal stage scheduling algorithm with unlimited schedule length. Figure 4.20 illustrates the additional register requirements of the bounded MinReg Stage Scheduler and the bounded stage scheduling heuristics, relative to the MinReg Stage Scheduler with unlimited schedule length. The top curve of Figure 4.20 corresponds to the bounded MinReg Stage-Scheduler. This curve is interesting since it compares the optimal scheduling algorithm with bounded SL with the same algorithm using the unlimited SL. This top curve is thus indicative of the additional register requirements that are strictly due to the bound on SL. As shown in the graph, the bounded optimal stage scheduler nds a schedule with the same number of registers for 81.2% of the loops and requires up to 1 additional register to schedule 95% of the loops. The other curves in Figure 4.20 correspond to the stage scheduling heuristics with bounded schedule length, and the Iterative Modulo Scheduler, which produced that schedule length, shown for comparison. These heuristics still result in a signi cant decrease in register requirements below Iterative Modulo Scheduling. For example, 95% of the loops can be scheduled with 2, 3, and 7 additional registers for the 3*UP+RSS, 3*UP, and RSS bounded scheduling heuristic. Our next measurement evaluates more directly the e ectiveness of the bounded heuristics, by comparing their register requirements to the bounded MinReg StageScheduler. This comparison is provided in Figure 4.21. Our rst observation is that the bounded heuristics achieve a signi cant fraction of the decrease in register requirements that is achieved by the bounded optimal stage scheduler, although they are not as close as the same scale as the unbounded heuristics are to the unbounded 149
1 0.9 Fraction of loops scheduled
0.8 0.7 0.6 MinReg Stage-Scheduling (bounded)
0.5
Heuristic 3*UP+RSS (bounded)
0.4
Heuristic 3*UP (bounded)
0.3
Heuristic UP (bounded) 0.2
Heuristic RSS (bounded)
0.1
Iterative Modulo Scheduling
0 0
1
2
3
4
5 6 7 8 Additional MaxLive
9
10
11
12
Figure 4.20: Additional register requirements with respect to the MinReg Stage-Scheduler with unlimited schedule length). 1 0.9 Fraction of loops scheduled
0.8 0.7 0.6 0.5
Heuristic 3*UP+RSS (bounded)
0.4
Heuristic 3*UP (bounded)
0.3
Heuristic UP (bounded)
0.2
Heuristic RSS (bounded)
0.1
Iterative Modulo Scheduler
0 0
1
2
3
4
5 6 7 8 Additional MaxLive
9
10
11
12
Figure 4.21: Additional register requirements with respect to MinReg Stage-Scheduler with bounded schedule length.) 150
Measurements: Iterative Modulo Scheduler: MaxLive Schedule Length (ratio) Execution Time (ratio) Bounded MinReg Stage Scheduler: MaxLive Schedule Length (ratio) Execution Time (ratio) Bounded 3*UP+RSS Heuristic: MaxLive Schedule Length (ratio) Execution Time (ratio) Bounded 3*UP Heuristic: MaxLive Schedule Length (ratio) Execution Time (ratio) Bounded UP Heuristic: MaxLive Schedule Length (ratio) Execution Time (ratio) Bounded RSS Heuristic: MaxLive Schedule Length (ratio) Execution Time (ratio)
min freq median average
max
2:00 0:1% 1:00 45:3% 1:00 52:9%
38:00 1:03 1:00
34:36 166:00 1:08 2:47 1:03 1:48
2:00 0:1% 1:00 45:3% 1:00 52:9%
29:00 1:03 1:00
27:82 133:00 1:08 2:47 1:03 1:48
2:00 0:1% 1:00 45:3% 1:00 52:9%
38:00 1:03 1:00
28:03 166:00 1:08 2:47 1:03 1:48
2:00 0:1% 1:00 45:3% 1:00 52:9%
29:00 1:03 1:00
28:24 148:00 1:08 2:47 1:03 1:48
2:00 0:1% 1:00 45:3% 1:00 52:9%
29:00 1:03 1:00
28:32 148:00 1:08 2:47 1:03 1:48
2:00 0:1% 1:00 45:3% 1:00 52:9%
29:00 1:03 1:00
28:90 166:00 1:08 2:47 1:03 1:48
Table 4.6: Characteristics of the stage scheduling algorithms with bounded schedule length.
151
optimal stage scheduler. For example, the best bounded heuristic nds a schedule with no additional registers for 91.6% of the loops, versus 95.9% for the best unbounded heuristic. This e ect can be explained, in part, by the fact that two of the heuristics that are guaranteed not to increase the register requirements, i.e. AC and SS, do not apply to dependence graphs that are strongly connected. The second observation is that the RSS heuristic is much more e ective for bounded heuristics than it is for the heuristics with unlimited SL. For example, although 91.6% of the loops are scheduled without additional registers with the 3*UP+RSS heuristic, only 87.7% are with only the 3*UP heuristic. This di erence contrasts with the small improvement, 95.9 versus 95.0%, when AC+3*UP+RSS is used instead of AC+3*UP for the heuristics with unlimited SL. The characteristics of the stage scheduling heuristics with bounded schedule length are summarized in Table 4.6. Notice that the schedule length and the execution time ratios are constant for every algorithm. As a result, the bounded MinReg StageScheduler and the 3*UP+RSS heuristic achieve a decrease in register requirements of 19.0 and 18.4%, respectively, with no decrease in performance.
4.6.3 Schedules with Increased Initiation Interval Our next experiments investigate the reduction in register requirements that is achieved when increasing II . We use here our implementation of the Iterative Modulo Scheduler in order to obtain MRT-schedules with increased II . We rst present in Figures 4.22 and 4.23 the register requirements of the Iterative Modulo Scheduler and the MinReg Stage-Scheduler, respectively, when the initial value of II found by the Iterative Modulo Scheduler is increased by 0, 1, 2, 3, 4, 5, 10, and 21 additional cycles. For a xed number of registers, i.e. a vertical line in Figures 4.22 and 4.23, we may investigate the fraction of the loops that can be scheduled when increasing II by some amount. In Figure 4.23 for example, 58% of the loops can be scheduled by the MinReg Stage-Scheduler without increasing II on a machine with 32 registers. This percentage increases to 73% if II is increased by 152
1 0.9 Fraction of loops scheduled
0.8 II+21
0.7
II+10
0.6
II+5
0.5
I+4 0.4
II+3
0.3
II+2
0.2
II+1
0.1
II
0 0
16 32 48 MaxLive (for Iterative Modulo Scheduler)
64
Figure 4.22: Increasing II to reduce the register requirements using the Iterative Modulo Scheduler. 1 0.9 Fraction of loops scheduled
0.8 II+21
0.7
II+10
0.6
II+5
0.5
II+4 0.4
II+3
0.3
II+2
0.2
II+1
0.1
II
0 0
16 32 48 MaxLive (for MinReg Stage-Scheduler)
64
Figure 4.23: Increasing II to reduce the register requirements using the MinReg Stage-Scheduler. 153
no more than 1 cycle, 78% with no more than 2 cycles, 88% with no more than 5 cycles, and up to 93% with no more than 21 additional cycles. We may also interpret these graphs by investigating the increase in II that may be required to schedule a xed fraction of the loops for varying numbers of available registers. In Figure 4.23 for example, II must be increased by up to 21 cycles to schedule 75% of the loops if only 16 registers are available. However, an II increase of up to 5 cycles is sucient with 24 registers, by 2 cycles with 32 registers, and no II increase is needed with 42 registers. Note also that increasing II decreases the register requirements most signi cantly for schedules that initially require between 16 and 32 registers when the MinReg Scheduler is used, and between 16 and 48 registers when the Iterative Modulo Scheduler is used. Our next experiment investigates the average initiation interval that is achieved on machines with nite register les, assuming that no spill code is used. When the register requirements of a schedule exceed the size of the register le, a new MRTschedule and stage-schedule is sought with a larger II , until its register requirements t the size of the register le or more than 21 attempts have been made, i.e. the search for a schedule was abandoned if II was increased by up to 21 cycles without resulting in a schedule that ts within the register le size. The resulting initiation intervals represent an upper bound on the performance degradation due to nite register le since, in practice, spill code would also be introduced to reduce the register pressure. Since the 1327-loop benchmark suite contains a wide range of loops, we divided the benchmark into 4 disjoint subsets. The rst subset, Benchmark-16, corresponds to loops with low register requirements and includes the 992 loops that can be scheduled by the Iterative Modulo Scheduler with no more than 16 registers when increasing II by up to 21 cycles. The second subset, Benchmark-32, corresponds to loops with medium register requirements and includes the 218 loops not in Benchmark16 that can be scheduled with no more than 32 registers and 21 additional cycles. Benchmark-48 and Benchmark-64 are de ned similarly, resulting in 87 and 23 loops with no more than 48 and 64 registers, respectively. The 7 remaining loops of the 154
1327-loop benchmark require up to 114 registers (when II is increased by 21 cycles) and are not considered here.
reg 16 reg 32 reg 48 reg 64 reg < 1 Algo:
Benchmark-16 with 992 loops 13:07 12:14 12:12 12:11 9:70 9:34 9:34 9:34 9:20 9:01 9:01 9:01 8:94 8:92 8:92 8:92 8:92 8:92 8:92 8:92 IMS BH UH MR
Benchmark-32 with 218 loops | 22:63 19:53 19:14 19:11 16:85 16:06 16:04 16:03 15:81 15:65 15:65 15:65 15:56 15:56 15:56 15:56 IMS BH UH MR
reg 16 reg 32 reg 48 reg 64 reg < 1 Algo:
Benchmark-48 Benchmark-64 with 87 loops with 23 loops | | | | 34:03 28:63 28:16 28:07 | 26:87 25:33 25:28 25:26 36:13 31:43 31:04 31:04 24:93 24:93 24:93 24:93 27:70 27:70 27:70 27:70 IMS BH UH MR IMS BH UH MR
Table 4.7: Average II for machines with nite register les (where IMS is the Iterative Modulo Scheduler, BH is the bounded AC+3*UP+RSS heuristic, UH is the unbounded AC+3*UP+RSS heuristic, and MR is the unbounded MinReg Stage Scheduler). Table 4.7 presents the average II for the Iterative Modulo Scheduler (IMS), the bounded AC+3*UP+RSS heuristic (BH for Bounded Heuristic), the AC+3*UP+RSS heuristic with unlimited schedule length (UH for Unbounded Heuristic), and the MinReg Stage Scheduler (MR) within the four disjoint benchmark subsets considered, and showing how the achieved average II decreases in each subset as the register limit increases. Bold entries correspond to cases where a stage scheduler decreases the average II by more than 10%, compared to the register-insensitive algorithm. We may conclude from Table 4.7 that register sensitive scheduling algorithms have a signi cant impact on the initiation interval when no spill code is introduced and when 155
the register requirements of a loop are critical. For example, the average II decreases by 15.6%, from 22.63 to 19.11 cycles, when the MinReg Stage Scheduler is used instead of the Iterative Modulo Scheduler for the loops in Benchmark-32 on a machine with 32 registers. When the register requirements are not critical, e.g. scheduling loops from Benchmark-32 on a machine with 64 registers, register sensitive algorithms do not impact the average II , as expected. Note also that the average II achieved by the BH and UH heuristics are very close to the II of the optimal algorithm, within 2 and 0.5% for the bounded and unbounded heuristic, respectively.
reg 16 reg 32 reg 48 reg 64 reg < 1 Algo:
Benchmark-16 474 of 992 loops 1:90 1:73 1:73 1:73 1:22 1:14 1:14 1:14 1:13 1:07 1:08 1:78 1:04 1:03 1:04 1:04 1:03 1:03 1:03 1:03 IMS BH UH MR
Benchmark-32 70 of 218 loops | 1:72 1:56 1:57 1:57 1:22 1:17 1:18 1:18 1:09 1:07 1:09 1:09 1:05 1:05 1:05 1:05 IMS BH UH MR
reg 16 reg 32 reg 48 reg 64 reg < 1 Algo:
Benchmark-48 Benchmark-64 40 of 87 loops 8 of 23 loops | | | | 1:63 1:32 1:32 1:33 | 1:19 1:06 1:07 1:08 1:46 1:29 1:33 1:33 1:04 1:04 1:04 1:04 1:10 1:10 1:10 1:10 IMS BH UH MR IMS BH UH MR
Table 4.8: Average execution time ratio for machines with nite register les (where IMS is the Iterative Modulo Scheduler, BH is the bounded AC+3*UP+RSS heuristic, UH is the unbounded AC+3*UP+RSS heuristic, and MR is the unbounded MinReg Stage Scheduler). Table 4.8 presents the execution time ratios achieved by the algorithms for the 597 loops for which pro ling information was available. Interestingly, the bounded stage scheduling heuristic performs generally better (and never worst) than the unbounded 156
52 MinReg Stage-Scheduler
50 Average Schedule Length
3*UP+RSS (bounded)
48
Iterative Modulo Scheduler
46
AC+3*UP+RSS
44
Schedule-Independent Lower Bound
42 40 38 36 34 0
2
4
6
8
10 12 Additional II
14
16
18
20
Figure 4.24: Impact of increasing II on the schedule length. heuristic and the unbounded MinReg algorithms, as its slightly larger average II is more than o set by its lower SL due to bounding. Incidentally, increasing the initiation interval does moderately a ect the schedule length of the modulo schedules, as indicated by Figure 4.24. Note the consistent behavior of the unbounded scheduling algorithms, i.e. the average schedule length rst decreases, reaches a minimum with an additional II of 1 or 2 cycles, and then increases. This e ect appears to be an artifact of the unbounded scheduling algorithms, as the lower bound of the schedule-independent schedule-length, which corresponds to the minimum distance between the start and stop pseudo-operation introduced in Section 4.5, does not indicate a similar e ect, and the Iterative Modulo Scheduler and the bounded heuristic tend to keep reducing SL until they level o .
157
4.7 Summary Modulo scheduling is an ecient technique for exploiting instruction level parallelism, resulting in high performance code, but increased register requirements. In this chapter, we have investigated optimal algorithms and heuristics for stage scheduling which shifts the operations of a schedule by integer multiples of II cycles. The rst contribution of this chapter is an optimal stage scheduling algorithm, the MinReg Stage-Scheduler, which nds a schedule with minimum MaxLive over all valid modulo schedules that share a given MRT. This algorithm proceeds in linear time (in the number of edges) for loops whose dependence graphs have an acyclic underlying graph. Otherwise, the algorithm repetitively uses a linear-programming formulation to handle general dependence graphs with unrestricted loop-carried dependences and common subexpressions. The MinReg Stage-Scheduler uses a novel linear-programming formulation which focuses solely on those operations that are in an underlying cycle of the dependence graph. Because of this ecient representation, and by exploiting the special properties of the stage scheduling problem, a schedule with minimum MaxLive is found for 1296 of the 1327 loops found in a benchmark suite from the Perfect Club, SPEC-89, and the Livermore Fortran Kernels as compiled for the Cydra 5. When applying this algorithm to the MRT-schedules produced by the register-insensitive Iterative Modulo Scheduler employed in our measurements, the average register requirements decrease by 19.8%. This chapter also contributes ecient stage scheduling heuristics that reduce the register requirements of a modulo schedule with much lower computational complexity. One of the proposed heuristics, referred to as the Acyclic heuristic (AC), produces in schedules in linear time that achieve the minimum MaxLive for dependence graphs that have acyclic underlying graphs. Super uous register requirements in the cyclic parts of the underlying graphs are speci cally addressed by the Register Sink/Source (RSS) and Up (UP) heuristics. By combining these basic heuristics, the AC+3*UP+RSS heuristic on average achieves 99.0% of the decrease in register 158
requirements obtained by the MinReg Stage-Scheduler in the 1327-loop benchmark suite, with a worst case complexity that is quadratic in the number of edges, provided that the MinDist relation is given as input. The AC+3*UP on average achieves 98.8% of the optimal decrease in register requirements with a complexity that is linear in the number of edges. Because the initiation interval of the given modulo schedules is preserved by the stage scheduling approach, the steady-state performance of a schedule is una ected by the stage schedulers. However, the stage scheduling algorithms may have an impact on the schedule length of a schedule, and thus alter the transient performance of its schedules. Experimental evidence gathered from the 1327 loop benchmark indicates that the average schedule length ratio increases by 5.6% for MinReg Stage-Scheduler, and by up to 4.6% for the stage scheduling heuristics. This increase in schedule length results, in turn, in an average execution time ratio that is 2.9% higher, for each of the stage scheduling algorithms over the 597 loops for which pro ling data was available to us. To prevent an increase in execution time, we also investigated the performance of the stage scheduling algorithms that search only for schedules with no larger schedule length than the schedule given as input. Experimental evidence from the 1327-loop benchmark shows that the stage schedulers with this bounded schedule length still achieve a large fraction of the possible decrease in register requirements. For example, the MinReg Stage-Scheduler with bounded schedule length achieves 95.9% of the decrease in register requirements that is achieved by the MinReg Stage-Scheduler with unlimited schedule length. The bounded 3*UP+RSS stage scheduling heuristic achieves 92.8% of the decrease in register requirements with a quadratic worst time complexity, and the bounded 3*UP heuristic achieves 89.7% of the decrease with a linear time complexity. To summarize, the stage scheduling approach proposed in this chapter allows a compiler to rst search for an ecient modulo schedule, focusing exclusively on nding a schedule with the smallest feasible II and schedule length. Then a stage scheduling pass signi cantly reduces the register requirements without impact on the 159
performance of the schedule, when a bounded stage scheduling algorithm is used. The bounded 3*UP+RSS heuristic is clearly the best stage schedule heuristic, decreasing the average register requirements by 18.4% compared to a register-insensitive modulo scheduler, and maintaining both the steady-state and transient performance of the original schedules. This heuristic should de nitely be implemented when compiling for machines in which the register requirements of modulo scheduled loops are critical.
160
CHAPTER 5 Modulo Scheduling Algorithms In the previous chapter, we focused on a modulo scheduling approach that distinguished between the MRT-scheduling and the stage-scheduling steps. While solving these two steps separately enabled us to develop ecient stage-scheduling algorithms, the generated schedules may not, in general, have a maximum throughput (II chosen by a modulo scheduling heuristic is not always the minimum possible) or minimum register requirements (since only one MRT is considered). This chapter presents an optimal register-sensitive modulo scheduling algorithm that jointly searches for an MRT-schedule and a stage-schedule, nding a schedule with the highest steady state throughput over all modulo schedules (minimum II ), and the minimum register requirements among all such schedules. Our scheduling method can satisfy any arbitrary dependence constraints for machines with nite resources and arbitrary reservation tables, and it minimizes MaxLive, the maximum number of live values at any single cycle of the loop schedule. When computing the register requirements of a schedule containing predicated operations, we assume that each predicate value is independent of the others. Further reducing the register requirements of schedules by investigating the relations among predicate values is postponed to Chapter 6. The technique used in this chapter is based on an integer linear programming formulation and it capitalizes on recent advances in integer linear programming for scheduling problems. The contributions of this chapter are threefold. First, we con161
tribute a formulation of the resource constraints for machines with nite resources and arbitrary reservation tables. This formulation enables us, for example, to precisely handle the resource requirements of the Cydra 5 [11], a machine with complex resource requirements. Second, we contribute a precise formulation of the register requirements of a modulo schedule. By using this precise model, instead of minimizing some approximation of the actual register requirements, we nd schedules with lower register requirements for a signi cant fraction of the 1327 loops in our benchmark suite (between 13.0% and 21.4%, depending on the approximation). Third, we propose a more ecient formulation of the scheduling constraints, which enables us to increase the number of loops successfully scheduled for minimum MaxLive from 781 to 920 loops. In this chapter, the related work is presented in Section 5.1 and the impact of modulo scheduling on the register requirements is illustrated in Section 5.2. A modulo scheduling algorithm for minimum register requirements is developed in Section 5.3 and this algorithm is extended to nd schedules with maximum throughput in Section 5.4. We demonstrate in Section 5.5 that the search space of register-sensitive modulo scheduling is bounded. Our experimental data is presented in Section 5.6 and our conclusions in Section 5.7.
5.1 Related Work Searching for modulo schedules with the highest steady-state throughput over the entire space of modulo schedules was rst formulated using an integer linear programming formulation by Hsu [48]. This formulation was obtained for arbitrary dependence graphs on machines with nite resources and simple resource reservation tables. Ning and Gao proposed an algorithm based on a linear programming formulation that results in a schedule with the highest steady-state throughput over all modulo schedules, and minimum average lifetime among such schedules for a machine with an unlimited number of resources [70]. In their work, and elsewhere [44][93], the register 162
requirements of a schedule are approximated by conceptual FIFO bu ers that are reserved for an interval of time that is a multiple of II . Their scheduling algorithm handles loop iterations with arbitrary dependence graphs. Ning and Gao's results were extended for machines with nite resources and fully-pipelined functional units by Govindarajan et al [44] using an algorithm based on an integer linear programming formulation that directly minimizes the bu er requirements of a schedule. Altman et al [7] have independently developed a formulation for machines with complex resource requirements. Their formulation simultaneously enforces the nite resource constraints and performs the mapping of operations to speci c functional units. For example [7], in a loop with 3 divide operations on a machine with 2 divide functional units, the mapping problem determines at compile time which divide operation is executed on which divide functional unit. Their approach formulates the joint nite resource and mapping problem as a coloring problem, using N 2 binary variables, N integer variables, and a number of constraints on the order of N 2 II jQj, where N is the number of operations in a loop iteration and Q is the set of machine resources. Their approach is very useful when scheduling for machines for which a mapping can not be trivially found, i.e. for machines that do not satisfy the mapping-free property de ned in Section 5.3.2. Otherwise, there is a class of mapping-free machines for which a mapping can be trivially found by unrolling the resulting schedule some nite number of times, and using a technique similar to modulo variable expansion [55]. For these machines, their approach does not require unrolling but introduces additional constraints on the scheduling space. Thus, their approach may result in schedules with more compact code (since no unrolling is required to nd a feasible mapping) and potentially lower steady-state performance (due to the additional constraints on the scheduling space that can be avoided by unrolling the resulting schedule). For machines where a trivial mapping can be found without unrolling, a formulation like the one proposed in this chapter should be used, since it is more concise (as it introduces no additional variables and uses II jQj constraints), more ecient (since its constraints satisfy the structured property de ned in Section 5.3.4), and 163
handles the resource requirements of machines with arbitrary reservation tables. Altman [6] has independently derived a model that minimizes the register requirements without requiring loop unrolling by jointly scheduling operations and allocating lifetimes to physical registers. This result is obtained by performing as many copy operations as necessary for lifetimes that span several stages. His formulation is based on a coloring problem that uses a large number of additional variables and constraints. In this chapter, we handle the register requirements by using a formulation that may require unrolling the loop, but does not require any additional copy operation. The proposed formulation is concise (because it introduces only the minimum number of variables required to model the lifetimes of each operation in each cycle of the MRT) and is ecient (since the two types of constraints that are used to model the register requirements are structured). Ning and Gao's polynomial-time algorithm [70] has also been used by Dupont de Dinechin [29] to nd modulo schedules with low register requirements. He proposed a scheduling heuristic that solves a linear programming model to compute the ranges of cycles in which to schedule each operation, nds a contention free cycle for one operation, xes the scheduling time of that operation in the linear programming formulation, and continues with the next operation. In this work, he also proposed an ecient technique to compute RecMII , the minimum initiation interval due to recurrences in the dependence graph, as well as a lower bound on the initiation interval due to the register requirements. Although this algorithm uses a linear programming solver, it does not necessarily result in schedule with maximum throughput or minimum register requirements, since it uses a heuristic to nd a contention free schedule for a machine with nite resources.
164
5.2 Modulo Scheduling and Register Requirements In this section, we illustrate the impact of stage scheduling on the register requirements with an example for the target processor summarized in Table 3.1 with three general-purpose functional units and the register model described in Section 3.2 with an additional reserved time of one cycle.
Example 5.1 This example illustrates the impact of modulo scheduling on the reg-
ister requirements of a modulo-scheduled loop. This kernel is: y[i] = a and is computed as shown in the dependence graph of Figure 5.1a.
x[i]2- x[i]-
The scheduler places each operation of an iteration so that both resource constraints and dependence constraints are ful lled. Figure 5.1b illustrates a schedule with an II of 2 for the kernel of Example 5.1 on the target machine. In this schedule, the load, mult, add, sub and store operations of the iteration starting at time 0 are respectively scheduled at time 0, 1, 1, 5, and 6. The modulo reservation table (MRT) associated with a schedule is obtained by collapsing the schedule for an iteration to a table of II rows, using wraparound. Figure 5.1c illustrates the MRT associated with the schedule of Figure 5.1b. The resource constraints of a modulo schedule are satis ed if and only if the packing of the operations within the II rows of the MRT does not exceed the resources of the machine. For our target machine, the resource constraints allow up to 3 operations of any kind to be issued in each row of the MRT. The MRT can also model specialized resources by allowing only certain types of operations in each column. The virtual register lifetimes associated with this iteration are presented in Figure 5.1d. Recall that the black bars correspond to the initial portion of the lifetime which minimally satis es the latency of the de ning functional unit. The dependence constraints are satis ed if and only if no operation is placed during the portion of the lifetime that overlaps with the black bar of a virtual register that it uses. 165
a) Dependence graph
ld
b) Schedule (II=2)
Time: 0
ld
1
*
d) Lifetimes vr0 vr1 ld
+
vr2
vr3
+
*
2
+
*
3 4 −
5
−
− st
st
6 7
st c) MRT
e) Register requirements vr0
Time: 0,2,4,6
ld
1,3,5,7
*
vr1
vr2
vr3
st +
−
Figure 5.1: Modulo schedule with add scheduled early (Example 5.1). a) Schedule (II=2)
Time: 0
ld
1
*
c) Lifetimes vr0 ld
vr1
vr2
Latency portion of the lifetime * Additional lifetime
+
+
2
vr3
Schedule of one iteration
3 4 −
5
− st
st
6 7
b) MRT
d) Register requirements vr0
Time:0,2,4,6
ld
+
1,3,5,7
*
−
vr1
vr2
vr3
Integral part
st Fractional part
Figure 5.2: Modulo schedule with add scheduled late (Example 5.1). 166
Formal Problem De nition. Among all modulo schedules that employ the smallest
feasible initiation interval and satisfy all resource constraints, nd one that minimizes MaxLive. The schedule presented in Figure 5.1b is a schedule that results in the minimum bu er requirements or integral MaxLive for this kernel and target machine. However, minimizing bu er requirements or integral MaxLive does not in general result in a schedule with minimal MaxLive, because the fractional MaxLive may be excessive. For example, Figure 5.2 presents a schedule for the same kernel, resulting in the same integral MaxLive but resulting in a lower total MaxLive. The schedule of Figure 5.2a di ers only by the schedule time of the add operation, which is delayed by one cycle from time 1 to time 2. The register requirements associated with this schedule are shown in Figure 5.2d. Comparing the register requirements of this schedule to those of Figure 5.1e, we see that both have an integral MaxLive of four. However, this schedule results in a lower fractional MaxLive as the fractional parts are more evenly distributed among the II rows. As a result, although the bu er requirements is eight in both cases, MaxLive decreases from eight to seven.
5.3 Scheduling for Minimum MaxLive In this section, we present an integer linear programming formulation that eciently schedules the operations of an innermost loop for machines with nite resource requirements while minimizing the register requirements associated with the schedule. In the rst part of this section, we describe the modulo schedule search space for machines with nite resources and arbitrary reservation tables. We present here three types of constraints that must be satis ed by valid modulo schedules. The rst type corresponds the assignment constraints, which insure that each operation in the loop is assigned to exactly one cycle in the schedule. The formulation of the assignment constraints is presented in Section 5.3.1 using the formulation proposed by Govindarajan et al [44]. The second type represents the resource constraints, which insure that no cycle of the schedule requires more resources than are available in the 167
Machine: (input) Q Mq Resi;q li;j Graph: (input) G = fV; Esched ; Ereg g
!i;j N Schedule: (input) II Results: (output) ar;i
ki vr;i MaxLive
set of resource types. number of resources of type q. set of cycles in which operation i uses resource q. latency between operation i and operation j . dependence graph with scheduling edge set and register data ow edge set. dependence distance between operation i and operation j . number of operations (vertices) in graph G. initiation interval (input constant in the integer programming model). binary matrix where ar;i = 1 if operation i is scheduled in row r. stage number of operation i. number of live virtual registers in row r generated by operation i. maximum number of live virtual registers among all rows.
Table 5.1: Variables for the modulo scheduling model. machine. The formulation of the resource constraints for machines with arbitrary reservation tables is developed in Section 5.3.2 and is original to our work. The third type corresponds the scheduling constraints, which enforce arbitrary scheduling dependences between pairs of operations. A traditional formulation of the scheduling constraints is shown in Section 5.3.3 using the formulation devised by Govindarajan et al [44]. A novel and more structured formulation of the loop scheduling constraints is developed in Section 5.3.4. In the latter part of this section, we describe three objective functions that account for the register requirements or some approximation of the register requirements. The two rst objective functions, presented in Sections 5.3.5 and 5.3.6, compute the average lifetime and the bu er requirements associated with a schedule, respectively, and are similar to objective functions found in previous work [70][44][29][93]. The third 168
objective function computes MaxLive, the maximum number of registers required at any single cycle of the schedule. This objective function is presented in Section 5.3.7 and is also original to our work. We use in this section the notation described in Section 3.5 to represent a loop iteration, its dependence graph, and the machine latencies, using the same notation as described in Section 3.5. In particular, the dependence graph is represented by G = fV , Esched , Ereg g, where the sets of edges Esched and Ereg correspond, respectively, to the scheduling dependences and the register dependences among operations. Moreover, an edge from operation i to operation j , w iterations later, is associated with a latency li;j and a dependence distance !i;j = w. All variables used here are summarized in Table 5.1.
5.3.1 Assignment Constraints Consider a loop with N operations and an initiation interval of II . We represent a schedule for this loop by a II N binary matrix, called A, where ar;i = 1 if and only if operation i is scheduled in row r of the MRT and 0 otherwise. The rst condition that a valid modulo schedule must satisfy is that each operation is scheduled exactly once per iteration: IIX ?1 r=0
8i 2 [0; N )
ar;i = 1
(5.1)
Equation (5.1) de nes all the assignment constraints, i.e. the constraints that assign each operation to exactly one row of the MRT.
5.3.2 Mapping-Free Resource Constraints The second condition that a valid modulo schedule must satisfy is that no cycle of the schedule does consume more resources than are available in the machine. Consider an operation i which uses resource q for one cycle exactly c cycles after being issued (referred to as a usage of resource q in cycle c in Chapter 2). Operation i utilizes 169
resource q in row r if operation i is scheduled in row (r ? c) mod II . Using the binary matrix A, we know that resource q is reserved in row r if a(r?c)modII;i = 1. To account for all the usages of resource q in a given row, we simply need to sum the appropriate a variables for each resource usage of q in each operation. For a machine with Mq machine resources of type q, the resource constraint for row r and resource q simply states that no more than Mq resources of type q can be concurrently reserved among all operations, namely: NX ?1
X
a(r?c)modII;i Mq i=0 c2Resi;q
8q 2 Q; 8r 2 [0; II )
(5.2)
where Q is the set of resource types and c 2 Resi;q indicates that operation i uses a resource of type q exactly c cycles after being issued. Inequality (5.2) precisely models the resource requirements in each cycle, and any assignment, A, that satis es Inequality (5.2) has no resource con icts, provided that the machine satis es the following property.
De nition 5.1 (Mapping-Free Property) A machine is said to be mapping-free
if for every schedule that satis es resource constraint (5.2) there is a mapping from each operation's resource usages to resource instances that is consistent with the hardware requirements of the machine.
We describe here three classes of machines that satisfy this property. First, consider a machine with no more than one instance of each resource, i.e. q 1 for each q 2 Q. Such a machine clearly satis es the mapping-free property because, since there is only one instance to choose from, the mapping from an operation's resource usages to resource instances is trivial. Second, consider a machine where, for each resource q 2 Q with Mq > 1, each operation that uses q can arbitrarily use any instances of q, in each cycle in which the operation reserves one or more instances of q. This machine also satis es the mappingfree property because there are no speci c hardware requirements that restrict the mapping from an operation's resource usages to resource instances. 170
Third, consider a machine for which there is a string constraint that requires certain uses of a resource by some operations that reserve that resource for several consecutive cycles to use the same instance of that resource for that entire string of cycles; however, every other resource usage over all operations can be mapped freely to any instance of the resource used. Note that we are guaranteed to be able to nd a legal mapping from each operation's strings to resource instances by using the algorithm presented in the proof of Theorem 3.1 and unrolling the loop some nite number of times. Note that Theorem 3.1 applies since this mapping problem is isomorphic to the problem of mapping virtual registers (each of which has a lifetime that is a string of consecutive cycles and must be assigned to the same physical register for its entire lifetime) to physical registers. After the strings are mapped, mapping the remaining resource usages is unrestricted and therefore trivial. Thus this machine satis es the mapping-free property as well. Note that, since the de nition of this third machine includes all machines with some mix of fully pipelined and non-pipelined execution units as de ned in [6], they constitute an important and common class of machines. Thus, for these three classes of machines and for any other machines with the mapping-free property, Inequality (5.2) insures that there are no resource con icts and that a legal mapping to resource instances can be found. There are, however, machines that do not satisfy the mapping-free property. For example, consider a machine with multiple instances of resource q (i.e. Mq > 1) that is used by an operation in two (or more) nonconsecutive cycles, without using that resource in the intervening cycles, and where the same instance of q must be used in both these cycles. Since these two usages are coupled, we must not only guarantee that no more than Mq resources are used in each cycle but also ensure that there is a legal mapping from the operation's resource usages of q to instances of q. We can handle these machines in one of the following three ways. One way is to use the alternative operations approach introduced in Chapter 2 and decide before scheduling which operations are mapped to which alternative, and which usages of a problematic resource are mapped to which instance of that resource. Thereby we would essentially 171
be considering each instance of a problematic resource to be a di erent resource so as to create a machine with only a single instance of each problematic resource. Finding an optimal solution may then, however, require solving many scheduling problems that di er in the alternatives chosen. A second way is to \ ll-in" such coupled nonconsecutive cycle usages with usages in the intervening cycles, thereby holding the resource throughout the intervening cycles, rendering it unusable by other operations during this period, and accepting the possible nonoptimal schedules that results. However, note that if each string formed by such added usages is contained within the usage pattern within the corresponding maximal resource, as de ned in Step 2 of Section 2.2, then the resulting schedule will still be optimal. A third way is to use the formulation recently proposed by Altman et al [7] which handles both the scheduling and the mapping problems simultaneously.
5.3.3 Simple Formulation of the Scheduling Constraints The third condition that a valid schedule must satisfy is that each scheduling dependence present in its dependence graph is enforced. We now introduce two derived parameters that characterize the MRT row and the time at which each operation is scheduled:
rowi timei
as:
row number in which operation i is scheduled (from 0 to II ? 1). time at which operation i is scheduled
Using matrix A, we may write the row number in which operation i is scheduled
rowi =
IIX ?1 r=1
r ar;i
(5.3)
A modulo schedule also de nes the stage in which each operation is scheduled. We represent the stage numbers by k, an integer vector of dimension N , where ki is 172
the stage number in which operation i is scheduled. Matrix A and vector k uniquely de ne the cycle in which each operation is scheduled. The schedule time of operation i is equal to:
timei = ki II + rowi
(5.4)
A modulo schedule must enforce all the scheduling dependences of its dependence graph. A dependence between operation i and operation j , !i;j iterations later, is ful lled if operation j is scheduled at least li;j cycles after operation i: (timej + !i;j II ) ? timei li;j
8(i; j ) 2 Esched
(5.5)
Substituting Equations (5.3) and (5.4) into Inequality (5.5) results in the following inequality: IIX ?1 r=1
r (ar;j ? ar;i) + (kj ? ki) II li;j ? !i;j II
8(i; j ) 2 Esched (5.6)
Inequality (5.6) de nes all the scheduling constraints of a modulo schedule for a given initiation interval II with respect to the dependence distances !i;j and dependence latencies li;j of a dependence graph G.
5.3.4 Ecient Formulation of the Scheduling Constraints Solving an integer linear programming model is generally implemented by iteratively solving a linear programming model where additional constraints are introduced (and removed) to force each integer variable to an integer value without omitting from the solution space any optimal (integer) solution [69]. A branch-and-bound algorithm can be used to determine which parts of the search space to consider, and each branchand-bound node is generally evaluated by solving a linear programming model with the original constraints augmented by some additional constraints that force variables to integer values. A key aspect for formulating an ecient integer program is 173
to nd a formulation that results in fewer branch-and-bound nodes, a goal that can be achieved in part by structuring the problem so that the linear programming solver naturally results in an integer solution for as many integer variables as possible. One bene cial property of a problem formulation is de ned here:
De nition 5.2 (Structured Constraints) A constraint is de ned as structured if each variable appears at most once, multiplied by either a +1, 0, or ?1 constant coef-
cient. By extension, a formulation is de ned as structured if each of its constraints are structured.
Note that the assignment and the resource constraints do satisfy this property, whereas the scheduling constraints formulated in Inequality (5.6) do not satisfy this property, since the elements of the k vector are multiplied by II and the elements of the binary A matrix are multiplied by r 2 [0; II ). In this section, we reformulate the scheduling constraints so that the scheduling constraints are structured, thus resulting in a more ecient representation of the modulo scheduling problem. The basic idea for this reformulation is due to Chaudhuri et al [23] which has such a reformulation for straight line (nonloop, nonbranching) code. The adaptation of this idea to modulo schedules for loop code is, however, not straightforward and substantially di erent in detail, as is the proof of the validity of this adaptation. Experimental evidence presented in Section 5.6.1 will indicate the bene t of using the structured formulation; for example, using the constraint presented in this section directly resulted in an average decrease in visited branch-and-bound nodes by a factor of 128, for the formulation that minimizes MaxLive. We derive a structured formulation of the scheduling constraint associated with scheduling edge (i; j ) as follows. First, we assume that we know the scheduling time of operation i (i.e. timei is known and constant) and derive a structured constraint that precisely determines the scheduling times for operation j that satisfy scheduling edge (i; j ). Second, we show how to extend this result without assuming the value of timei to be known and constant. 174
Case with known time
i
Recall that a scheduling constraint associated with scheduling edge (i; j ) enforces the scheduling dependence between operation i and operation j , !i;j iterations later, and was de ned in Inequality (5.5) as timej +!i;j II ?timei li;j . Because each term in the scheduling constraint has an integer value, we may reformulate this inequality as a strict inequality, i.e. timej + !i;j II ? timei > li;j ? 1. We may thus write:
timej + !i;j II > timei + li;j ? 1
(5.7)
The left hand side of Inequality (5.7) corresponds to the times where the value produced by operation i can legally be used by operation j (!i;j iterations later). For conciseness, we refer to this value as timeu (for the time of use) in the remainder of this section. Using the relations timej = kj II + rowj and timeu = ku II + rowu , we may thus de ne the stage and row of timeu as, respectively,
ku = kj + !i;j
rowu = rowj
(5.8)
Similarly, the right hand side of Inequality (5.7) corresponds to the latest time in which the value produced by i is forbidden from use. We refer to this value as timef (for the last forbidden time) in the remainder of this section. Using the relations timei = ki II + rowi and timef = kf II + rowf , we may thus de ne the stage and row of timef as, respectively, $ % row i + li;j ? 1 kf = ki + rowf = (rowi + li;j ? 1) mod II (5.9) II Note that since we assume here that the value of timei is known and constant, by extension, the values of kf and rowf are known and constant as well. The new de nitions are summarized here for later reference.
175
timeu timef
time where the value produced by operation i can legally be used by operation j (!i;j iterations later). latest time in which the value produced by i is forbidden from use.
Using the de nitions from Equations (5.8) and (5.9), we may write the scheduling constraint expressed in Inequality (5.7) as:
ku II + rowu > kf II + rowf
(5.10)
We may transform Inequality (5.10) to isolate the two stage numbers:
? rowu ku ? kf > rowf II
(5.11)
Interestingly, we can show that the row di erence rowf ? rowu has values in the range (?II; II ) since rows, by de nition, have values in the range [0; II ). Consequently, the right hand side of Inequality (5.11) has values in the range (?1; 1). Therefore, we can guarantee that when the integer valued stage di erence ku ? kf is 1 (or larger), the scheduling constraint is satis ed. Similarly, we can guarantee that when the stage di erence ku ? kf is -1 (or smaller), the scheduling constraint is violated. We may thus write: (ku > kf ) ) scheduling constraint is satis ed (ku < kf ) ) scheduling constraint is violated
(5.12) (5.13)
We are thus able to determine the scheduling times of operation j that satisfy the scheduling constraint when ku 6= kf . Otherwise kf = ku , i.e. both the use time of the value produced by operation i and the latest forbidden time for that value occur in the same stage of the schedule, and thus the speci c values of rowu and rowf must be take into account to determine whether timej is feasible. 176
To evaluate this case, let kf = ku and substitute kf for ku in Inequality (5.10), obtaining the following scheduling constraint:
rowu > rowf
(5.14)
Inequality (5.14) states that, since operation i is scheduled in rowi, its value is only available after rowf , and thus if operation j is scheduled in the same stage as timef , operation j must be assigned to a row that is strictly larger than rowf . Reformulating Inequality (5.14) using the binary ar;j variables, we obtain:
z=
row Xf x=0
ax;j = 0
(5.15)
where the value of the sum Equation (5.15) is referred to as z. Because of the assignment constraints, we know that only one ar;j variable is equal to 1, and all other ar;j variables are equal to 0. Thus, we know that z is either 0 or 1, depending on the row in which operation j is scheduled. Consequently, we can clearly see that Equation (5.15) is satis ed if and only if operation j is not assigned to any of the rows in the range [0; rowf ]. As a result, Inequality (5.14) and Equation (5.15) are equivalent. We may thus write: (ku = kf ) & (z = 0) ) scheduling constraint is satis ed (ku = kf ) & (z 6= 0) ) scheduling constraint is violated
(5.16) (5.17)
To summarize our ndings, we have shown that operation j satis es the scheduling edge (i; j ) if either Relation (5.12) is satis ed (i.e. ku > kf ) or Relation (5.16) is satis ed (i.e. ku = kf and z = 0). Otherwise, we have shown by Relations (5.13) and (5.17) that the scheduling constraint is violated. We may now combine the two disjoint Relations (5.12) and (5.16) by formulating the following constraint:
kf ? ku + z 0 177
(5.18)
Inequality (5.18) is equivalent to the union of Relations (5.12) and (5.16) because when ku > kf , Inequality (5.18) holds regardless of the value of z since z 2 [0; 1] and when ku = kf , Inequality (5.18) holds precisely when z = 0. Using the de nitions in Equations (5.8), (5.9), and (5.15) in Inequality (5.18), we obtain the following constraint: $ % (rowi +li;jX ?1)modII row i + li;j ? 1 ki + ax;j 0 (5.19) ? kj ? !i;j + II x=0 Consequently, we have derived a constraint that determines, for a given timei (expressed in terms of ki and rowi ), the scheduling times for operation j (expressed in terms of kj and ax;j ) that satisfy the scheduling edge (i; j ). This constraint is structured in that its variables (i.e. kj and ax;j ) appear only once and are only multiplied by +1, 0, and ?1 constant coecients.
Case with unknown time
i
In this case, we do not assume to know the time at which operation i is scheduled. Our claim is that the following inequality: $ % (r+li;j ? X1)modII r + l i;j ? 1 ki + ax;j 1 ? ar;i ? k ? ! + j i;j II x=0 8r 2 [0; II ) (5.20) is equivalent to Inequality (5.19) when r = rowi and is trivially satis ed otherwise. First, we show that Inequalities (5.19) and (5.20) are equivalent when r = rowi. Since r = rowi is assumed, we can substitute rowi for r in Inequality (5.20), obtaining the same left hand side as in Inequality (5.19). Moreover, after the substitution step, the right hand side of Inequality (5.20) is (1?arowi ;i). Because operation i is scheduled in rowi , we know that arowi;i = 1, and thus the right hand side of Inequality (5.20) is also 0. Thus, Inequalities (5.19) and (5.20) are identical when r = rowi . Second, we must show that Inequality (5.20) is trivially satis ed when r 6= rowi. Since r 6= rowi , we know that ar;i = 0, and thus we may rewrite the r 6= rowi cases 178
of Inequality (5.20) as follows:
$
% (r+li;j ? X1)modII r + l i;j ? 1 ki + ax;j 1 ? k ? ! + j i;j II x=0
(5.21)
8r 2 [0; II ); r 6= rowi
We must now show that Inequality (5.21) is always less constraining than Inequality (5.19). Comparing these two inequalities, the condition under which Inequality (5.21) holds without introducing additional constraints is: $ % $ % r + li;j ? 1 ? rowi + li;j ? 1 + II II (r+li;j ? X1)modII
x=0
ax;j ?
(rowi +li;jX ?1)modII
x=0
ax;j 1
(5.22)
8r 2 [0; II ); r 6= rowi
Since r 6= rowi, we must consider two cases: 0 r < rowi < II and 0 rowi < r < II . In the rst case, we know that the di erence of the two oor functions in Inequality (5.22) is either 0 or ?1. However, by the assignment constraints, each of the summations in Inequality (5.22) is either 0 or 1. Thus the left hand side of Inequality (5.22) is in the range [?2; 1]. Inequality (5.22) is thus trivially satis ed and introduces no new constraints if r < rowi . In the second case, r > rowi , and the di erence of the two oor functions in Inequality (5.22) is either 0 or 1. (a) If this di erence is 0, we note that since the di erence of the two summations is either ?1, 0, or 1, Inequality (5.22) is trivially satis ed and introduces no additional constraints. (b) If the di erence of the two oor functions is 1, i.e. the integer part of (r + li;j ? 1)=II is greater than the integer part of (rowi + li;j ? 1)=II , it follows that, since r ?rowi < II , the fractional part of (r +li;j ?1)=II is less than the fractional part 179
of (rowi + li;j ? 1)=II , i.e. (r + li;j ? 1) mod II < (rowi + li;j ? 1) mod II . The set of the terms in the rst summation is thus a subset of the terms in the second summation. The di erence between the two summations in Inequality (5.22) is thus either 0 or ?1. Therefore, once again, Inequality (5.22) is trivially satis ed and introduces no new constraints. As a result, Inequality (5.22) is always trivially satis ed, as is Inequality (5.21), and the only constraining case of Inequality (5.20) is r = rowi which is equivalent to Inequality (5.19).
Final Form of the Structured Scheduling Constraints Since we have just shown that Inequality (5.20) is equivalent to Inequality (5.19) when r = rowi and is trivially satis ed otherwise, we can simply state that the scheduling constraints of all the scheduling edges are satis ed when: % $ (r+li;j ? X1)modII r + l i;j ? 1 ax;j 1 ? ar;i (5.23) ? kj ? !i;j + ki + II x=0
8r 2 [0; II ); 8(i; j ) 2 Esched Chaudhuri et al [23] make following observation for two dependent operations in straight line (nonloop, nonbranching) code: consider operation i, with latency l, that produces a value used by operation j . When operation i is assigned to cycle t, or any subsequent cycles, operation j must be assigned in a cycle t0 t + l. Using a similar observation here, we may replace ar;i in the right hand side of Inequality (5.23) by the sum of the ax;i variables over x 2 [r; II ). Thus, we obtain the following nal form of the structured scheduling constraints:
180
(r+li;j ? X1)modII ax;j ax;i + x=r x=0
IIX ?1
+ ki ? kj
$
% r + l i;j ? 1 !i;j ? +1 II
(5.24)
8r 2 [0; II ); 8(i; j ) 2 Esched At rst, it may appear counterintuitive to replace Inequality (5.6) with Inequality (5.24) in order to obtain a more ecient formulation of the problem, since it corresponds to replacing each constraint with II new constraints. The crucial point, however, is that the new constraints are structured, since each variable, namely each k and each element of A appears at most once, and is multiplied by only +1, 0, or ?1 coecients.
5.3.5 Average Lifetime Requirements We have now completed the constraints that de ne the modulo scheduling space and continue with the presentation of three distinct objective functions. We have seen in Section 3.2 that the average lifetime requirements correspond to the sum of all the lifetimes divided by II . We represent the lifetime here by f , an integer vector of dimension N , where fi is the number of cycles in which the value produced by operation i is live. In our machine model, a lifetime begins in the cycle in which its def-operation is scheduled, and lasts through the cycle in which its last-use operation is scheduled. Thus, we may de ne the lifetime fi as follows: (timej + !i;j II ) ? timei + 1 fi
8(i; j ) 2 Ereg
(5.25)
where timei is the cycle in which the def-operation i is scheduled and timej + !i;j II is the cycle in which the use-operation j , !i;j iterations later, uses that value. By substituting Equations (5.3) and (5.4) into Inequality (5.25) we obtain the following
181
inequality: IIX ?1 r=1
r (ar;j ? ar;i) + (kj ? ki ) II + !i;j II + 1 fi
(5.26)
8(i; j ) 2 Ereg Note that since, for example, the variables ki and kj are multiplied by II , and the variables ar;i and ar;j are multiplied by r 2 [1; II ), Inequality (5.26) is not structured. The average lifetime is simply the sum of the fi for each operation in the loop, divided by II : NX ?1 AvgLive = II1 fi i=0
(5.27)
Since multiplying (or dividing) an objective function by a strictly positive number is guaranteed to result in an equivalent solution (i.e. a solution with the same objective function value, once multiplied by that number), we may simply ignore the 1=II factor in Equation (5.27) when searching for a schedule with minimum AvgLive. After the 1 II factor is removed from Equation (5.27), the formulation is identical to the one proposed by Dupont de Dinechin [29], for example, except for the presence of the +1 term (which is particular to our virtual register model) in Inequality (5.26).
5.3.6 Bu er Requirements As indicated in Section 3.2, bu ers must be reserved for only intervals that are integer multiples of II cycles. We represent here the bu er requirements by b, an integer vector of dimension N , where bi corresponds to the number of bu ers that are associated with operation i. The bu er requirements resulting from register edge (i; j ) is de ned as: (timej + !i;j II ) ? timei + 1 bi II 182
(5.28)
where, unlike fi in Inequalities (5.25) and (5.26), the variable bi is multiplied by II since it must cover an interval that is a multiple of the initiation interval. By substituting Equations (5.3) and (5.4) into Inequality (5.28) we obtain in the following inequality: IIX ?1 r=1
r (ar;j ? ar;i) + (kj ? ki ) II + !i;j II + 1 bi II
(5.29)
8(i; j ) 2 Ereg where the variables ki, kj , and bi are all multiplied by II , and the variables ar;i and ar;j are multiplied by r 2 [1; II ). The total bu er requirements is then simply equal to the sum of the bu er requirements associated with each operation, i.e.
Buff =
NX ?1 i=0
bi
(5.30)
The formulation of Bu is identical to the one proposed by Govindarajan et al [44], except for the presence of the +1 term (which is particular to our virtual register model) in Inequality (5.29). Note that, as formulated here, the formulation of Bu is not structured because of Inequality (5.29).
5.3.7 Register Requirements We now de ne the register requirements associated with a modulo schedule. Since we are minimizing MaxLive, we must compute the precise number of live virtual registers in each of II consecutive cycles. We de ne a new II N integer matrix, called V , where vr;i corresponds to the number of live virtual registers generated by operation i in row r. Matrix V is de ned as follows: r X z=0
az;i ?
rX ?1 z=0
az;j + (kj + !i;j ) ? ki vr;i
8(i; j ) 2 Ereg ; 8r 2 [0; II ) 183
(5.31)
Inequality (5.31) quanti es the register requirements in row r generated by the register dependence (i; j ), from the def operation i to the use operation j . To demonstrate the validity of Inequality (5.31), we compute the fractional and integral parts of the lifetime associated with a register edge (i; j ), namely:
fracr;i;j
fractional part of the lifetime in row r associated with a register edge (i; j ). integral part of the lifetime associated with a register edge (i; j ).
inti;j
We will demonstrate that their sum is exactly equal to vr;i if operation j is the lastuse of the operation i result, and never exceeds vr;i if j is not the last-use operation. We introduce three de nitions speci c to def operation i and use operation j :
dr =
r X z=0
az;i ;
ur =
rX ?1 z=0
az;j ;
8 > < 1 if rowi > rowj x=> : 0 otherwise
(5.32)
Recall that among all rows, z, for a given operation, i, only one az;i is nonzero and that az;i is 1. As a result, both dr and ur are nondecreasing functions of r, with minimum value 0 and maximum value 1. As de ned in Equation (5.32), dr becomes 1 exactly in the row z where def operation i is scheduled (az;i = 1). Similarly, ur becomes 1 exactly in the row z following the row where use operation j is scheduled (az?1;j = 1). Once dr or ur become 1, they remain 1 through row II ? 1. When computing the fractional part of the lifetime associated with def operation i and use operation j , two cases must be considered. In the rst case, rowi rowj . Therefore, the initial value of the term dr ? ur is initially 0, becomes 1 in rowi , and returns to 0 in the row after rowj . As a result, the term dr ? ur is 1 exactly where the fractional part of the lifetime associated with (i; j ) would contribute to the register requirements if operation j is the last use; elsewhere, dr ? ur is 0. In Figure 5.3a for example, the def operation i is scheduled in row 1 (a1;i = 1) and the use operation j is scheduled in row 3 (a3;j = 1). The plots of dr , ur , and dr ? ur are also shown in 184
a) Value live from row 1 to 3 row
MRT (II=5)
dr
ur
b) Value live from row 4 to 1 (with wraparound) d r−u r
MRT (II=5)
dr
ur
0 1
0 1
d r−u r+1
0 1
use: op j
def: op i
2 3
use: op j def: op i
4 0 1
0 1
0 1
0 1
Figure 5.3: Determining the fractional and integral part of a lifetime. Figure 5.3a. Notice that the fractional part of this lifetime is correctly determined to be live in rows 1, 2 and 3. In the second case, rowi > rowj . The initial value of the term dr ? ur + 1 is 1, becomes 0 in the row following rowj , and returns to 1 in rowi . As a result, the term dr ? ur + 1 is 1 exactly where the fractional part of the lifetime associated with (i; j ) would contribute to the register requirements if operation j is the last use; elsewhere, dr ? ur + 1 is 0. In Figure 5.3b for example, the def operation i is scheduled in row 4 (a4;i = 1) and the use operation j is scheduled in row 1 (a1;j = 1). The plots of dr , ur , and dr ? ur + 1 are shown in Figure 5.3b. We see that the fractional part of this lifetime is correctly determined to be live in rows 4, 0, and 1. Using the de nitions of Equation (5.32), we can express the fractional part of the lifetime associated with operations i and j in row r as:
fracr;i;j = dr ? ur + x
(5.33)
where fracr;i;j is 1 if and only if row r contributes to the fractional part of the lifetime associated with operations i and j . To compute the integral part of the lifetime associated with operations i and j , we must also consider two cases. In the rst case, rowi rowj , and the integer part corresponds directly to the di erence between the stage numbers of the use and the def operations, namely (kj + wi;j ) ? ki . In Figure 5.3a, the integral part is 0 when 185
kj = ki and !i;j = 0. If the use operation was delayed by II cycles, kj would increase by 1 and so would the integral part. In the second case, rowi > rowj , and the integer part corresponds directly to (kj + wi;j ) ? ki ? 1, where the -1 term is introduced to counterbalance the wraparound e ect. In Figure 5.3b, the integral part is 0 when kj = ki + 1 and !i;j = 0. Again, delaying the use operation by II cycles would increase the integral part by 1. As a result, we can express the integral part of the lifetime associated with operations i and j as: inti;j = kj + wi;j ? ki ? x
(5.34)
By summing the fractional and integral parts of the lifetime, as de ned in Equations (5.33) and (5.34), the x terms cancel each other, resulting in the left hand side of Inequality (5.31). Finally, the register requirement contribution of an operation i in row r corresponds to the maximum register requirements among all uses of operation i. Using the previous result, MaxLive is de ned as the sum of the fractional and integral part of the lifetime associated with each virtual register in the row with the largest fractional plus integral part: NX ?1 i=0
vr;i MaxLive
8r 2 [0; II )
(5.35)
Since the objective is to minimize MaxLive, it is sucient to use Inequality (5.31) rather than setting vr;i equal to the maximum left-hand side over all operations j for the operation i under consideration. Note also that the formulations of both Inequalities (5.31) and (5.35) are structured. For a machine with multiple register les, we would introduce one MaxLive variable per register le and we would transform Inequality (5.35) by summing the vr;i variables only over each operation that writes its results in the register le being considered. The same modi cation also apply to Equations (5.27) and (5.30) to min186
imize, respectively, the average lifetime or bu er requirements for individual register les.
5.4 Optimum Modulo Schedules We present in this section an algorithm that nds a schedule with the highest steady state throughput over all modulo schedules, and the minimum register requirements among such schedules. The formulation is based on the scheduling space de ned by Inequalities (5.1), (5.2), and (5.24) and on the objective function for minimum MaxLive de ned by Inequalities (5.31) and (5.35). This model and its variables are summarized, respectively, in Figure 5.4 and Table 5.1. Note that the formulation nds a schedule with minimum MaxLive for a given initiation interval. Therefore, a schedule with minimum register requirements for the smallest feasible initiation interval is obtained by solving a series of problems until the smallest initiation interval with a feasible solution is found.
Algorithm 5.1 (MinReg Modulo Scheduler) Consider a machine description Mq , Resi;q , and li;j and consider a data dependence graph G = fV; Esched ; Ereg g and !i;j . The following algorithm nds a schedule with the highest steady state throughput over all modulo schedules, and the minimum register requirements among such schedules: 1. Compute the Minimum Initiation Interval (MII ) [78] and set the tentative II to MII . 2. Build the integer linear programming system as indicated in Figure 5.4 and search for a solution. 3. If the system built in Step 2 fails to nd a feasible solution, increase II by one and return to Step 2. Otherwise, the solution found is optimal.
For comparison purposes, we also investigated a model minimizing AvgLive, as formulated by the Inequalities (5.26) and (5.27) in place of the Lifetimes and MaxLive inequalities in Figure 5.4. Similarly, a model minimizing Buff as formulated by 187
Minimize:
MaxLive
Subject to: Assignment: Resources: Dependences:
IIX ?1 r=0 NX ?1
8i 2 [0; N )
ar;i = 1
X
i=0 c2Resi;q
a(r?c)modII;i Mq
(r+li;j ? X1)modII ax;j ax;i + x=r x=0
IIX ?1
8q 2 Q ; 8r 2 [0; II ) $
% r + l i;j ? 1 + ki ? kj !i;j ? +1 II
8r 2 [0; II ); 8(i; j ) 2 Esched Lifetimes: MaxLive:
De nitions:
r X z=0
az;i ?
NX ?1 i=0
rX ?1 z=0
az;j + kj ? ki + !i;j vr;i 8r 2 [0; II ) ; 8(i; j ) 2 Ereg
vr;i MaxLive
8r 2 [0; II )
MaxLive; ar;i; vr;i; ki : nonnegative integers
Figure 5.4: Modulo scheduling for minimum MaxLive.
188
the Inequalities (5.29) and (5.30) in place of the Lifetimes and MaxLive inequalities in Figure 5.4, and a model with no objective function formulated by deleting the objective function and the Lifetimes and MaxLive inequalities in Figure 5.4. Each of these formulations can use precisely formulation of the precisely the same scheduling space as used when minimizing MaxLive. Objective function none AvgLive Bu MaxLive
Variables N (II + 1) N (II + 2) N (II + 2) N (2II + 1)
Constraints N + II jQj + II jEsched j N + II jQj + II jEsched j + jEreg j N + II jQj + II jEsched j + jEreg j N + II jQj + II jEsched j + II (jEreg j + 1)
Table 5.2: Numbers of variables and constraints for 4 distinct formulations. The complexity of these models, in terms of the number of variables and constraints, is shown in Table 5.2. Note that the numbers of binary and integer variables needed for the formulation of the modulo scheduling space are N II and N , respectively. And N + II jQj + II jEsched j constraints are used to de ne the space of valid modulo schedules. When either AvgLive or Bu is minimized, N additional integer variables are introduced as well as jEreg j additional constraints. If MaxLive is minimized, N II additional integer variables are introduced, along with II (jEreg j + 1) new constraints. Note also that if, instead of the structured scheduling constraints, the traditional scheduling constraints are used, as de ned by Inequality (5.6), the number of constraints decreases by (II ? 1) jEsched j for all 4 models in Table 5.2. Experimental evidence gathered from the 1327-loop benchmark in Section 5.6.1 shows that an ecient formulation is not only dependent on the numbers of variables and constraints in the formulation, but also on whether the constraints are structured or not.
189
5.5 Bounding the Schedule Space As presented in Figure 5.4, the integer linear programming search space is unbounded because there is no limit on the ki values and hence on the maximal schedule length. To bound the schedule length of an iteration and thereby signi cantly improve the eciency of the integer programming solver, we introduce two pseudo operations in the dependence graph: start is a predecessor of every operation in the graph, and stop is a successor to every operation in the graph. The latency and dependence distance of the outgoing edges from start and of the incoming edges to stop are 0. Using the methodology presented by Hu , the MinDist relation can be used to determine a lower bound and an upper bound on the schedule time of each operation [49]. We know that operation i cannot be scheduled earlier than MinDist(start; i) cycles after the start pseudo operation. Similarly, we know that operation i cannot be scheduled later than MinDist(i; stop) cycles before the stop pseudo operation. To bound the search space of the integer linear programming solver, we must relate the schedule time of these two pseudo operations to one another. Without loss of generality, we de ne the maximal schedule length of a schedule to be MinDist(start; stop) + S where S is a nonnegative integer value. Using this result, we bound the schedule time of operation i as follows:
MinDist(start; i) timei (5.36) timei MinDist(start; stop) + S ? MinDist(i; stop) (5.37) where S represents the schedule slack or degree of freedom given to the scheduler.
Theorem 5.1 Let RS be the minimum MaxLive for a dependence graph G and initiation interval II among all schedules of schedule length no greater than MinDist(start; stop) + S . The minimum MaxLive values are bounded as follows:
R0 R1 : : : Ry?1 Ry = Ry+1 = : : : = R1; where y = (N ? 1)(lmax + II ? 1) ? MinDist(start; stop) 190
Note also that lmax corresponds to the longest latency among all operations present in the loop iteration.
Proof. The relation RS RS+1 (for any positive schedule slack S ) clearly holds
since any schedule with minimum MaxLive and schedule length no greater than MinDist(start; stop) + S is necessarily included in the set of schedules of length no greater than MinDist(start; stop) + S + 1. To prove Ry = R1 , we must show that Ry = minimum MaxLive and hence no Rx can be less than Ry if x > y. Suppose that Ry 6= minimum MaxLive, i.e. Ry > minimum MaxLive. Then the only minimum MaxLive schedules must have a schedule length greater than MinDist(start; stop)+ y. We demonstrate that this is impossible by showing that for any schedule of length greater than MinDist(start; stop)+y, there must be another valid schedule of length no greater than MinDist(start; stop) + y with no larger register requirements. Consider a schedule of length z > MinDist(start; stop)+y, i.e. z > (N ?1)(lmax + II ? 1). We now show that we can trivially transform this schedule to obtain a valid schedule of length no greater than MinDist(start; stop) + y with no larger register requirements. This demonstration is based on the pigeonhole principle. In the schedule of length z, consider each interval [ti; tj ] where no operation is scheduled in the interval (ti; tj ). Since there are N operations, there are at most N ? 1 such intervals. Furthermore, since the length of the schedule, z, is strictly larger than (N ? 1)(lmax + II ? 1), there must be at least one interval1 of length at least lmax + II . Consider [t0i; t0j ] to be such an interval. We may then trivially reschedule all operations scheduled after t0i, II cycles earlier, without violating any dependence constraints. Moreover, rescheduling operations II cycles earlier is guaranteed to satisfy the resource constraints as the MRT is unchanged by the new schedule. Consequently, this transformation results in a valid modulo schedule with a schedule length that is reduced by II cycles. By nite induction, we can reduce the schedule length z to z0, where z0 (N ? 1)(lmax + II ? 1). We assume here N > 1, i.e. we exclude here the trivial problem of scheduling a loop with a single operation. 1
191
Furthermore, reducing the schedule length by multiples of II is guaranteed to result in no larger register requirements because it reduces all the lifetimes, if any, that span the selected [t0i; t0j ] intervals by II cycles for each [t0i; t0j ] intervals spanned. Thus this new schedule of length z0 is precisely the schedule we set out to nd. Therefore, limiting the schedule length to (N ? 1)(lmax + II ? 1), i.e. limiting the scheduling slack to y = (N ? 1)(lmax + II ? 1) ? MinDist(start; stop), does not preclude nding a minimum MaxLive solution, and Ry = Ry+1 = : : : = R1 . 2
5.6 Measurements In this section, we compare the performance of the optimal modulo scheduling algorithms developed in this chapter, the stage-scheduling algorithms presented in Chapter 4, and a register-insensitive modulo scheduler. In addition to the scheduling algorithms presented in Section 4.6, we use here the following modulo scheduling algorithms.
MinReg Modulo-Scheduler. This scheduler nds a schedule with the maximum steady-state throughput over all modulo schedules, and with the minimum register requirements among such schedules. It uses the integer programming model developed in Section 5.3 with the minimum MaxLive objective function, as summarized in Figure 5.4.
MinBu Modulo-Scheduler. This scheduler nds a schedule with the maximum
steady-state throughput, and the minimum bu er requirements among all such schedules. It uses the same formulation of the scheduling space as the MinReg ModuloScheduler and di ers only by its objective function, which minimizes Bu , as computed using Inequalities (5.29) and (5.30) in place of the Lifetimes and MaxLive inequalities of Figure 5.4. Although it minimizes bu ers, in our comparisons we always present the actual register requirements associated with these schedules.
192
MinLife Modulo-Scheduler. This scheduler nds a schedule with the maximum steady-state throughput, and the minimum average lifetime among all such schedules. It di ers from MinReg Modulo-Scheduler only by its objective function, which minimizes AvgLive, as computed by using Inequalities (5.26) and (5.27) in place of the Lifetimes and MaxLive inequalities of Figure 5.4. In our comparisons, we also present the actual register requirements associated with these schedules.
NoObj Modulo-Scheduler. This scheduler simply nds a schedule with the maxi-
mum steady-state throughput, without minimizing any objective function. It uses the same formulation of the modulo scheduling space as the MinReg Modulo-Scheduler, i.e. the same constraints as in Figure 5.4 except that the Lifetimes and MaxLive constraints are deleted. It simply returns the rst schedule that it nds. In our experimental setup, we use the CPLEX solver running on an HP-9000/715 workstation. The solver searches for an optimal solution using no more than 15 minutes of execution time. If no solution is found within the time limit, the solver aborts and \no schedule" is reported. The benchmark used here is the 1327-loop benchmark as compiled for the Cydra 5 and presented in Section 2.7. In addition to the constraints presented in Section 5.3, we bound the scheduling space using Inequalities (5.36) and (5.37) with a schedule slack of 20 cycles. We use here a constant schedule slack in order to nd schedules with small schedule length, since the length of a schedule impacts the transient performance of a loop, as indicated by Equation (3.1). We also use a lower bound on the register requirements, i.e. MinAvg as computed by Hu [49], to bound the values of the objective functions. The resource constraints are formulated for the reduced machine representation of the Cydra 5 shown in Figure 2.5b, as produced by the algorithm described in Chapter 2 for the discrete representation. For operations with alternatives, we use here the alternative that was selected by the Iterative Modulo Scheduler [77][78]. Furthermore, for a given loop, we only model the resources that are used by at least two operations, since the other resources, if any, pose no resource con icts. 193
In this section, we evaluate the performance of the integer linear programming formulations in Section 5.6.1, present the increase in throughput achieved by the modulo schedulers in Section 5.6.2, and present the decrease in register requirements obtained by these schedulers in Section 5.6.3.
5.6.1 Evaluation of the Integer Linear Programming Formulation In this section, we evaluate the performance of the integer linear programming formulation presented in Section 5.3, using several metrics such as the number and size of problems that are successfully solved, the number of branch-and-bound nodes traversed by the solver, and the number of simplex iterations performed by the solver. In our rst experiment, we investigate the bene t of using the structured formulation of the scheduling dependence (derived in Section 5.3.4) instead of the traditional unstructured formulation (presented in Section 5.3.3). When the structured scheduling constraints are used, both the NoObj and the MinReg Modulo-Schedulers use formulations that are fully structured, unlike the formulation of the MinLife and the MinReg Modulo-Schedulers since both AvgLive and Bu , respectively, are computed using some unstructured constraints. We search for optimal schedules within the 15 minute execution time limit using the MinReg, MinBu , MinLife, and NoObj Modulo-Schedulers, once using the structured scheduling constraints and once using the traditional scheduling constraints. We evaluate the performance of the modulo schedulers for the 711 loops that were successfully scheduled by all algorithms. Our rst graph presents the average number of branch-and-bound nodes visited by the solver when using either the structured or the traditional scheduling constraints. The CPLEX solver used in our experiment explores branch-and-bound nodes when it must force variables to integral values. Note that if the initial solution found by the solver is integral, no branch-and-bound node is ever visited. This graph is shown in Figure 5.5 with a logarithmic Y-axis. 194
Numbers of Branch & Bound Nodes
1000 Traditional Structured
100
10
1
0.1 NoObj
MinLife
MinBuff
MinReg
Figure 5.5: Average number of branch-and-bound nodes per loop visited by the CPLEX solver in the 711-loop benchmark.
10000
Number of Iterations
Traditional Structured
1000
100
10
1 NoObj
MinLife
MinBuff
MinReg
Figure 5.6: Average number of simplex iterations per loop performed by the CPLEX solver in the 711-loop benchmark.
195
The rst observation is that the four schedulers bene ts signi cantly from using the structured constraints which, for example, decrease the average number of nodes visited by the MinReg Modulo-Scheduler by a factor of 128. The second observation is that the two algorithms that result in the lowest average number of branch-and-bound nodes are the MinReg and the NoObj Modulo-Scheduler, when both are based on formulations that are structured, i.e. consisting exclusively of structured constraints. Figure 5.6 plots the average number of simplex iterations when scheduling for the same algorithms and dataset. Simplex iterations are used by the CPLEX program to solve the linear programming model associated with each branch-and-bound node. This graph is shown in Figure 5.6, also with a logarithmic Y-axis. Similar observations apply here: using the structured scheduling constraints signi cantly decreases the average number of simplex iterations, for example, by a factor of 34 for the MinReg Modulo-Scheduler. Note also that the smallest average number is obtained by the NoObj Modulo-Scheduler, with an average of 6.3 iterations per loop with the structured formulation and 9.7 with the unstructured formulation. Interestingly, the third lowest average number is achieved by the MinLife ModuloScheduler, with an average of 24.6 iterations per loop with the structured formulation, even though the structured MinLife Modulo-Scheduler visits on average more branchand-bound nodes than the structured MinReg Modulo-Scheduler. This e ect may be due to the fact that the MinLife Modulo-Scheduler has signi cantly fewer variables and constraints than the MinReg Modulo-Scheduler, and thus can apparently process each node with fewer simplex iterations. Experimental evidence thus indicates that searching for optimal schedules requires traversing fewer branch-and-bound nodes and performing fewer simplex iterations, on average, when using the structured rather than the traditional formulation of the dependence constraints. As a result, schedulers based on the structured scheduling constraints run signi cantly faster; for example, the structured MinReg ModuloScheduler completes in 11.6% of the time (i.e. 101.0 instead of 870.2 seconds) when scheduling all the loops successfully scheduled by the MinReg Modulo-Scheduler with unstructured scheduling constraints. Consequently, the solver can schedule a larger 196
fraction of the loops in the benchmark suite when using the structured scheduling constraints; e.g. 1181 versus 1089 loops for the NoObj Modulo-Scheduler, 937 versus 861 loops for the MinLife Modulo-Scheduler, 801 versus 765 loops for the MinBu Modulo-Scheduler, and 920 versus 781 loops for the MinReg Modulo-Scheduler. We thus exclusively employ the structured scheduling constraints in the remainder of this chapter. In our second experiment, we investigate the performance of each modulo scheduling algorithm with the structured scheduling constraints in a benchmark containing all successfully scheduled loops. We present the performance characteristics of the four modulo scheduling algorithms in Table 5.3. In addition to previous performance numbers (i.e. the number of branch-and-bound nodes and simplex iterations), the table also lists data on the numbers of variables and constraints, prior to any simpli cations that might be performed by the CPLEX solver, as well as the number of operations, N , and the initiation interval, II . First, observe the distribution of the data in Table 5.3. As indicated by the low median values, relative to the average numbers, there is a large number of simple loops in the benchmark suites. However, the solvers clearly succeed in solving some rather large problems, as indicated by the large max values. The second observation is that the NoObj Modulo-Scheduler processes loops with signi cantly larger average numbers of operations, II , variables, and constraints than the three other algorithms. It also handles the loops with the largest maximum number of operations and II , i.e. 80 operations and 118 cycles, respectively. Note also that since it simply returns the rst valid integral solution, the number of visited branch-and-bound nodes directly corresponds to the nodes that are needed by the solver to force all variables to integer values. As indicated in the table, very few branch-and-bound nodes are required (0 node in 73.9% of the loops, 7.93 nodes on average, and and never more than 337 nodes). This data con rms again the bene t of structured formulations. The third observation is that the MinReg Modulo-Scheduler processes loops that are nearly as large as those the MinLife Modulo-Scheduler can process, and larger 197
Measurements: NoObj Modulo-Sched: (1181 loops) Variables Constraints Branch-and-bound nodes Iterations II N MinLife Modulo-Sched: (937 loops) Variables Constraints Branch-and-bound nodes Iterations II N MinBu Modulo-Sched: (801 loops) Variables Constraints Branch-and-bound nodes Iterations II N MinReg Modulo-Sched: (920 loops) Variables Constraints Branch-and-bound nodes Iterations II N
min freq median average 182:50 270:86 7:93 331:98 7:51 13:91
max
4:00 0:3% 4:00 0:3% 0:00 73:9% 0:00 37:0% 1:00 31:9% 2:00 0:4%
33:00 45:00 0:00 11:00 2:00 9:00
3880:00 5320:00 337:00 20645:00 118:00 80:00
6:00 0:4% 5:00 0:4% 0:00 62:8% 0:00 24:8% 1:00 39:7% 2:00 0:5%
28:00 78:22 1232:00 34:00 109:34 1783:00 0:00 320:08 28712:00 14:00 4380:51 353654:00 2:00 5:09 59:00 7:00 8:68 41:00
6:00 0:5% 5:00 0:5% 0:00 58:8% 0:00 29:3% 1:00 47:2% 2:00 0:6%
20:00 55:80 2520:00 19:00 76:62 3464:00 0:00 441:18 25969:00 3:00 2013:64 159121:00 2:00 4:46 118:00 6:00 6:76 21:00
6:00 0:4% 6:00 0:4% 0:00 63:7% 0:00 24:3% 1:00 41:1% 2:00 0:5%
35:00 120:43 5809:00 43:00 155:23 6901:00 0:00 124:42 10711:00 23:00 2533:77 167504:00 2:00 4:72 118:00 7:00 8:37 41:00
Table 5.3: Characteristics of the modulo scheduling algorithms (using the structured scheduling constraints).
198
than those of the MinBu Modulo-Scheduler, even though it handles problems with 54.0% and 115.8% more variables, on average, than the MinLife and the MinBu Modulo-Scheduler, respectively. Thus, the structured formulation of the MinReg Modulo-Scheduler appears to successfully counterbalance the impact of its increased number of variables due to the precise modeling of the register requirements. Incidentally, the loop with the largest number of variables is solved by the MinReg Modulo-Scheduler; this loop has 37 operations, 36 register edges, 45 scheduling edges, 6 con icting resources, an II of 78, and resulted in a problem with 5809 variables and 6901 constraints. To summarize the ndings of this section, the NoObj Modulo-Scheduler clearly schedules the largest fraction of the loops in the benchmark suite. This result is not unexpected since this formulation does not minimize any objective function. The second and surprising result is that using the MinReg scheduler is nearly as ecient as using the MinLife scheduler, and is much more ecient than using the MinBu scheduler, in spite of the fact that precisely minimizing MaxLive requires signi cantly more variables than minimizing some approximations of the register requirements.
5.6.2 Maximum Throughput Measurements In our third experiment, we use the NoObj Modulo-Scheduler to search for schedules with maximum throughput. Since the Iterative Modulo Scheduler proposed by Rau [78] results in schedules with optimal throughput for 1274 (or 96.0%) of the 1327 loops in the benchmark suite, we only need to investigate here the remaining 53 loops. Among these 53 loops, the NoObj Modulo-Scheduler nds 2 loops where II can be decreased by 2 (but not 3) cycles, i.e. the scheduler nds a schedule with an II decreased by 2 cycles and shows that decreasing II by 3 cycles is infeasible. The scheduler also nds 6 loops where II can be decreased by 1 (but not 2) cycles. It furthermore shows that the II of 22 of these loops cannot be decreased even by 1 cycle. Finally, the scheduler does not complete the remaining 23 loops within the 15 minute execution time limit. Thus, the NoObj Modulo-Scheduler succeeds in nding 199
schedules with better throughput for 8 (or 15.1%) of the 53 interesting loops. Using the NoObj Modulo-Scheduler, we have thus shown that the Iterative Modulo Scheduler actually nds a schedule with maximum throughput for 22 more loops than previously known, i.e. it results in schedules with maximum throughput for 1296 (or 97.7%) of the 1327 loops in the benchmark suite. In addition to these 22 loops, the NoObj Modulo-Scheduler nds 8 more loops with maximum throughput (with a higher throughput than the schedule found by the Iterative Modulo Scheduler); thus we have now schedules with maximum throughput for 1304 (or 98.3%) of the loops in the benchmark suite. At this point, it is unknown whether the remaining 23 loops have suboptimal throughput.
5.6.3 Register Requirements Measurements Our fourth experiment investigates the register requirements of the modulo scheduling algorithms investigated in this chapter as well as some of the stage scheduling algorithms presented in Chapter 4. We use for this measurement a subset of 920 loops from the 1327-loop benchmark, corresponding to all the loops successfully scheduled by the MinReg Modulo Scheduler by using no more than 15 minutes of execution time on a HP-9000/715 workstation. The principal characteristics of the 920-loop benchmark are shown in Table 5.4. Measurements: Operations II II=MII nonredundant register edges nonredundant scheduling edges modeled machine resources
min 2:00 1:00 1:00 1:00 1:00 1:00
freq median average max 0:5% 7:00 8:37 41:00 41:1% 2:00 4:72 118:00 98:9% 1:00 1:00 1:50 0:5% 7:00 8:80 50:00 0:5% 7:00 9:20 57:00 54:8% 1:00 2:51 8:00
Table 5.4: Characteristics of the 920-loop benchmark suite. The Iterative Modulo Scheduler was used to generate the required MRT input for all stage scheduling algorithms. All scheduling algorithms used the initiation interval found by the Iterative Modulo Scheduler to compare the register requirements of the 200
1 0.9 Fraction of loops scheduled
0.8 0.7 0.6 0.5 0.4 0.3
Schedule-Independent Lower Bound
0.2
MinReg Modulo-Scheduler
0.1
Iterative Modulo Scheduler
0 0
16
32
48 64 MaxLive
80
96
Figure 5.7: Register requirements in the 920-loop benchmark. various scheduling algorithms at identical levels of instruction level parallelism. As indicated in Table 5.4, only 10 loops in the 920-loop benchmark failed to achieve MII . Of these 10 loops, we know that II can be decreased by 2 cycles for 1 loop, by 1 cycle for 4 loops, and cannot be decreased for 2 loops; the situation is unknown for the 3 remaining loops because no solution was found by the solver within the time limit. We rst investigate the register requirements of the MinReg Scheduler solutions which have the lowest register requirements over all valid modulo schedules with the achieved II . Figure 5.7 presents the fraction of the 920 loops that can be scheduled for a machine with any given number of registers without spilling and without increasing II . In this graph, the X-axis represents MaxLive and the Y-axis represents the fraction of loops scheduled. This graph presents three curves. The rst curve, labeled \Schedule Independent Lower Bound," corresponds to [49] and is representative of the register requirements of a machine without any resource con icts. The second curve presents the register requirements of the schedules generated by the MinReg Modulo201
1 0.9 Fraction of loops scheduled
0.8 0.7 0.6 0.5 0.4
MinReg Stage-Scheduler
0.3
Heuristic AC+3*UP+RSS
0.2
Heuristic 3*UP+RSS (Bounded)
0.1
Iterative Modulo Scheduler
0 0
4
8
12
16
20 24 28 32 Additional MaxLive
36
40
44
48
Figure 5.8: Additional register requirements relative to the MinReg Modulo-Scheduler. Scheduler. The fact that these rst two curves are nearly identical indicates that the complex resource requirements of the Cydra 5 machine do not signi cantly impact the register requirements of the optimal schedules in the benchmark suite considered. The third curve presents the register requirements of the Iterative Modulo Scheduler, which presently does not attempt to minimize the register requirements. The gap between the last two curves is indicative of the degree of improvement that is possible with register-sensitive modulo scheduling algorithms. The next graph illustrates the number of additional registers needed relative to the MinReg Modulo-Scheduler when heuristic and MinReg stage schedulers are used on the MRT found by the Iterative Modulo Scheduler. The X-axis of Figure 5.8 represents the number of additional registers and the Y-axis represents the fraction of loops that can be scheduled with this number of additional registers. The curves represent the additional register requirements for the MinReg Stage-Scheduler, the best stage scheduling heuristic with unlimited schedule length (AC+3*UP+RSS), the 202
best stage scheduling heuristic with bounded schedule length (3*UP+RSS), and the Iterative Modulo Scheduler itself. We observe that the MinReg Stage-Scheduler nds a schedule with no additional register requirements in 65.9% of the loops in the 920-loop benchmark. It requires no more than 3 additional registers to schedule 95.0% of the loops, and may require up to 10 additional registers for the remaining loops. We may also notice that the curves for both stage scheduling heuristics, as in Chapter 4, nearly overlap with the MinReg Stage-Scheduler curve. In particular, the AC+3*UP+RSS and the bounded 3*UP+RSS heuristics nd a schedule with no additional register requirements in, respectively, 65.9% and 60.4% of the loops, require no more than 3 and 4 additional registers to schedule 95% of the loops, and never need more than 10 additional registers. In particular, the 3*UP+RSS stage scheduling heuristic with bounded schedule length combined with Rau's Iterative Modulo Scheduler [78] schedules 92.8% of the 920 loops with no more than 3 additional registers, and 98.3% with no more than 6 additional registers. Furthermore, these schedules with near optimal register requirements are obtained without impact on the performance, since the Iterative Modulo Scheduler solely focus on nding a schedule with low initiation interval and schedule length, and the bounded stage scheduling heuristic does not increase either of these while reducing the register requirements. As shown in Figure 5.8, the Iterative Modulo Scheduler results in a schedule with minimum register requirements in 44.9% of the loops, a surprisingly high percentage for a heuristic that does not attempt to minimize the register requirements. However, the overall register requirements of the Iterative Modulo Scheduler is severely degraded by a few loops that require up to 49 additional registers. Our next experiment investigates to what extent the stage scheduling approach degrades in loops with large II , since, unlike a modulo scheduler, a stage scheduler can only move operations by multiples of II cycles. For this experiment, we divided the 920 loops successfully scheduled by the MinReg Modulo-Scheduler into 7 distinct subsets, where each subset is associated with a range of II . Figures 5.9 and 5.10 plot, 203
for each of these subsets, the additional MaxLive that is required if, instead of using the MinReg Modulo-Scheduler, we use the MinReg Stage-Scheduler with the MRTs produced by the Iterative Modulo Scheduler. The rst observation is that, by de nition, no loop with II = 1 require any additional registers, as the scheduling space of the stage scheduling and the modulo scheduling approaches are identical for II = 1. We notice, however, that a signi cant fraction of the loops with larger II also have no additional register requirements The second observation is that II and the additional register requirements appear to be correlated by the following rule of thumb, which is particularly apparent in Figure 5.10: \A loop with an initiation interval of II > 1 cycle is likely to require no more than II additional registers." In our data set, for example, no loop with II = 2 requires more than 2 additional registers, only 2 loops with II = 3 need more than 3 additional registers, 1 loop with II = 4 requires more than 4 additional registers, 3 loops with II = 5 requires more than 5 additional registers, and so on. Recall, however, that these observations are derived from a subset of the 1327loop benchmark, and that conclusions drawn for this subset may not apply for the larger loops that are omitted here because they required more than 15 minutes of computation time for the MinReg Modulo-Scheduler. In our last experiment, we illustrate the bene t of precisely modeling the register requirements by investigating the register requirements of 752 loops successfully scheduled by each of the modulo schedulers, using the structured scheduling constraints and up to 15 minutes of CPU time per loop. Figure 5.11 illustrates the additional register requirements incurred when another scheduler is used instead of the MinReg Modulo-Scheduler. One interesting observation is that approximating the register requirements by AvgLive, as used by the MinLife Modulo-Scheduler, results in schedules with fewer additional register requirements than when approximating the register requirements by Buff , as used by the MinBu Modulo-Scheduler. For example, the MinLife Modulo-Scheduler nds schedules with no additional register requirements for 85.5% of the 752 loops, and never requires more than 2 additional registers. The the MinBu 204
0.7 II=50-200 (6 loops)
Fraction of the scheduled loops
0.6
II=21-50 (57 loops) 0.5 II=6-20 (66 loops) 0.4 II=4-5 (96 loops) 0.3
II=3 (87 loops)
0.2
II=2 (230 loops)
0.1
II=1 (378 loops)
0 0
1
2
3
4 5 6 Additional MaxLive
7
8
9
10
Figure 5.9: Additional register requirements of the MinReg StageScheduler relative to the MinReg Modulo-Scheduler in the 920-loop benchmark, broken down by loops in each II range.
Fraction of the scheduled loops
0.18 0.16
II=50-200 (6 loops)
0.14
II=21-50 (57 loops)
0.12
II=6-20 (66 loops)
0.1
II=4-5 (96 loops)
0.08 II=3 (87 loops) 0.06 II=2 (230 loops) 0.04 II=1 (378 loops)
0.02 0 0
1
2
3
4 5 6 Additional MaxLive
7
8
9
10
Figure 5.10: Detail of the additional register requirements of the MinReg Stage-Scheduler (blown up region of Figure 5.9). 205
1 0.9 Fraction of loops scheduled
0.8 0.7 0.6
MinLife Modulo-Scheduler
0.5
MinReg Stage-Scheduler
0.4
Heuristic AC+3*UP+RSS Heuristic 3*UP+RSS (Bounded)
0.3
MinBuff Modulo-Scheduler
0.2
Iterative Modulo Scheduler 0.1
NoObj Modulo-Scheduler
0 0
1
2
3
4
5 6 7 8 Additional MaxLive
9
10
11
12
Figure 5.11: Additional register requirements relative to the MinReg Modulo-Scheduler in the 752-loop benchmark. Modulo-Scheduler, however, nds schedules with no additional register requirements in only 65.0% of the 752 loops, and needs up to 10 additional registers. It is also interesting to note that using the MinReg Stage-Scheduler or one of the two stage scheduling heuristics to minimize MaxLive for the MRT selected by the Iterative Modulo Scheduler results in lower register requirements than minimizing bu ers (integral MaxLive) among all MRTs.
5.7 Summary Modulo scheduling is an ecient technique for exploiting instruction level parallelism in a variety of loops. It results in high performance code, but increases the register requirements. As the trend toward higher concurrency continues, whether due to using and exploiting machines with faster clocks and deeper pipelines, wider issue, or a combination of both, the register requirements will increase even more. As 206
a result, scheduling algorithms that reduce register pressure while scheduling for high throughput are increasingly important. This chapter presents an approach that schedules loop operations to achieve the minimum register requirements for the smallest possible initiation interval on machines with nite resources and arbitrary reservation tables. It is based on a well structured integer linear programming formulation that schedules medium sized loops (i.e. up to 41 operations, II of up to 118 cycles, and up to 5809 variables in the integer linear program) in a reasonable time (i.e. no more than 15 minutes on a HP-9000/715 workstation). Moreover, it precisely models the register requirements of a modulo schedule in each cycle of the schedule. Our contributions are threefold. First, we have increased the applicability of this approach by handling machines with nite resources and arbitrary reservation tables. The proposed formulation is guarantee to nd a feasible schedule for machine that satis es the mapping-free property. We demonstrated this increased modeling capability by precisely handling the resource requirements of the Cydra 5, a machine with complex reservation tables. Our second contribution is a more structured formulation of the modulo scheduling solution space. This more ecient formulation addresses a major concern with modulo schedulers that are based on integer linear programming solvers [83], which is their traditionally high execution time. Experimental evidence (gathered for a benchmark suite of 1327 loops from the Perfect Club, SPEC-89, and the Livermore Fortran Kernels, compiled for the Cydra 5 machine) indicates that the number of branch-andbound nodes is decreased on average by a factor of 103, when using the structured instead of the traditional formulation of the MinReg Modulo-Scheduler on the 781 loops successfully scheduled by both formulations. It results, in turn, in a decrease in the total execution time by a factor of 8.6, from 870.2 to 101.0 seconds. Also, using the more ecient representation enabled us to successfully schedule a larger fraction of the 1327 loops; for example the coverage increases from 58.9 to 69.3% for the MinReg Modulo Scheduler and from 82.1 to 89.0% for the NoObj ModuloScheduler. Using this ecient formulation, the NoObj Modulo-Scheduler succeeded 207
in scheduling loops with up to 80 operations and with an II of up to 118 cycles, using no more than 15 minutes per loop on an HP-9000/715 workstation. Our third contribution is a precise formulation of the register requirements. Compared to approximating the register requirements by the average lifetime, minimizing MaxLive results in schedules with lower register requirements for 13.0% of the 1327 loops, and decreases the register requirements by up to 2 registers. Similarly, if the register requirements are approximated by bu ers (i.e. if only the integral part of MaxLive is considered), minimizing MaxLive results in schedules with lower register requirements for 21.4% of the 1327 loops, and decreases the register requirements by up to 10 registers. Furthermore, we used the MinReg Modulo Scheduler to assess the e ectiveness of the stage scheduling algorithms presented in Chapter 4 when used in conjunction with the Iterative Modulo Scheduler. First, our experimental evidence con rms the e ectiveness of the Iterative Modulo Scheduler (as proposed and implemented by Rau). We already knew that it achieves MII for 96.0% of the 1327 loops of a benchmark suite from the Perfect Club, SPEC-89, and the Livermore Fortran Kernels, compiled for the Cydra 5 machine. Using the NoObj Modulo-Scheduler, we were able to show that the Iterative Modulo Scheduler nds a schedule with the smallest feasible II for 22 more loops, for a total of 97.7% of the 1327 loops. Among the remaining 2.3% of the loops, the NoObj Modulo-Scheduler succeeded in nding schedules with the lower feasible II for 25.8% of these loops, decreasing their II by up to 2 cycles. Second, our experimental ndings indicate that the stage scheduling approach presented in Chapter 4 achieves most of its decrease in register requirements, when using the MRTs produced by the Iterative Modulo Scheduler in a subset of the 1327loop benchmark that contains all the 920 loops successfully scheduled by the MinReg Modulo-Scheduler using no more than 15 minutes per loop on an HP-9000/715 workstation. Our results indicate that the average register requirements decrease by 22.2% for an optimal modulo scheduler with respect to a register-insensitive modulo scheduler. When using an optimal stage scheduler in conjunction with the Iterative Modulo Scheduler, the average register requirements decrease by 19.9% over the 208
920-loop benchmark suite. Also, the average register requirements decrease by 19.8 and 19.2%, respectively, when the stage scheduling heuristics AC+3*UP+RSS with unlimited schedule length and 3*UP+RSS with bounded schedule length are used in conjunction with the Iterative Modulo Scheduler. Thus, stage scheduling appears to provide almost all of the register reduction, with another few percent possible by coupling it with an algorithm that searches for a better MRT than the one provided by the Iterative Modulo Scheduler. However, the search for a better MRT is very computationally intensive process, with an average time of 13.2 seconds (and up to 15 minutes) for the MinReg Modulo-Scheduler in the 920-loop benchmark compared, for example, to an average time of 0.4 milliseconds (and up to 12.7 milliseconds) for the AC+3*UP+RSS stage scheduling heuristics. Thus, the use of a stage scheduling is highly recommended for production compilers.
209
CHAPTER 6 Register Allocation for Predicated Code Current research compilers for VLIW and superscalar machines focus on exposing more of the inherent parallelism in an application to obtain higher performance by better utilizing wider machines and reducing the schedule length of a code. There is generally insucient parallelism within individual basic blocks, higher levels of parallelism can be obtained by also exploiting the instruction level parallelism among successive basic blocks. A well established approach uses predication to merge several basic blocks into a single enlarged predicated block. This approach relies on predicated operations [48], a class of operations that complete normally when a logical expression, referred to as the predicate of an operation, evaluates to true. Otherwise, the predicated operation is transformed into a no-op and has no side e ect. For scalar code, the hyperblock approach [60][62] used in the IMPACT compiler combines frequently executed basic blocks from multiple execution paths into an enlarged predicated block. This technique enables a range of compiler optimizations and facilitates the task of the scheduler, resulting in greatly improved schedules. For loop code, conditional statements within the innermost loop are merged to enable ecient software pipelining techniques [49][77] which exploit the instruction level parallelism present among the iterations of a loop by overlapping the execution of consecutive loop iterations. IF-conversion [4][72] is the enabling technique used in both scalar and loop code to translate the selected basic blocks into an enlarged predicated block. 210
However, predicated blocks with high levels of parallelism result in higher register requirements, for two main reasons. First, the register requirements increase as more values are needed to support more concurrent operations. This e ect is inherent to parallelism and is exacerbated by wider machines and higher latency pipelines [64]. Second, we show in this chapter that the register requirements increase for predicated code as current compilers do not allocate registers as well in predicated code as in unpredicated code. Developing compiler techniques that exploit instruction level parallelism while containing the register requirements is crucial to the performance of future machines. In previous work, a virtual register in a single predicated block is considered live from the cycle in which it is rst de ned to the cycle in which it is last used, regardless of the predicates under which its def and use operations are guarded. In this chapter, we demonstrate that the register requirements can be decreased if this assumption is relaxed, i.e. if the predicate expressions are taken into account. The rst contribution of this chapter is a framework to analyze predicated codes. Information about predicated values are extracted from the code, live ranges and their predicate expressions are discovered, and live range interferences are computed. This framework can process arbitrary predication schemes and arbitrary relations among predicates, including correlations between IF-converted branches. This framework is applicable either before or after scheduling. The current implementation relies on a symbolic package to generate accurate results, using an approach similar to the one taken in the PARADIGM compiler [88]. For register allocators based on the classical graph coloring method, originally proposed by Chaitin [15][18], register allocation for predicted codes can be simply achieved by using a re ned interference graph instead of the conventional one. However, several register allocators depart from the graph coloring method: e.g. Hendren et al [46] investigate a framework based on cyclic interval graphs, introducing the notion of time in the register allocator paradigm. This additional notion of time is particularly useful for the live ranges of a loop, where live ranges may cross the boundary of an iteration. Another approach, investigated by Rau et al [82], proposes 211
a general framework for the allocation of registers in software pipelined loops for various code generation and hardware support schemes. The second contribution of this chapter is a set of heuristics that reduces the register requirements by allowing non-interfering virtual registers that overlap in time to share a common virtual register. We refer to this process as the bundling of compatible virtual registers. Bundling is similar to Chaitin-style coalescing [18] in that two or more virtual registers are combined; however, bundling di ers in that it combines virtual registers with distinct values. Bundling is performed after constructing the re ned interference graph, and before using a conventional register allocation based on either graph coloring, interval graphs, or Rau's software pipelining register allocator. Bundling is also applicable either before or after scheduling. We investigate the performance of several bundling heuristics in a benchmark suite of modulo scheduled loops extracted from the Perfect Club, SPEC-89, and the Livermore Kernels. To minimize the interaction between bundling and scheduling heuristics, we used the scheduling technique described in Chapter 4 which minimizes the register requirements for a modulo schedule. Preliminary results indicate that our best bundling heuristic decreases the average register requirements from 39.3 to 36.5 registers and increases the percentage of loops with no more than 64 registers from 85% to 95%. The average register requirements of our best heuristic is 1% above a predicate-sensitive lower bound. Preliminary results indicate that bundling compatible virtual registers is more successful when applied after rather than before scheduling. To our knowledge, the general issue of resource allocation in the context of predicated code has not been addressed in the literature. However, information about predicate values has recently been introduced in research compilers. In the IMPACT compiler, for example, information about predicates is stored in a Predicate Hierarchy Graph (PHG) [60][62] used to determine useful relations among predicate values. This information is used to re ne data ow analysis, optimization, scheduling, and allocation in presence of hyperblocks. Additionally, this information is used to conditionally reserve functional units when modulo scheduling under the Reverse-IF212
Conversion scheme [95]. A complementary approach is taken with the Gated Single Assignment (GSA) form [10], where precise predicate information is embedded in the data ow graph. This approach is used in the Polaris parallelizing compiler to re ne data and memory dependence analysis and to aid loop parallelization [90]. In this chapter, we illustrate the impact of predicated code on the register requirements and outline our general framework in Section 6.1. The bundling heuristics after scheduling and before scheduling are respectively introduced in Sections 6.2 and 6.3. We present experimental results in Section 6.4 and conclude in Section 6.5.
6.1 Predication and Live Ranges In this section, we illustrate why current compilers succeed in accurately determining the live ranges of virtual registers in regular basic blocks, but fail when these basic blocks are converted to a single predicated block. We illustrate this negative impact by investigating the register requirements of a simple conditional statement before and after IF-conversion. We propose a solution that eliminates this negative impact by developing a precise representation of the live range in single-entry single-exit predicated blocks.
Example 6.1 Our example is based on the simple conditional statement presented
in Figure 6.1a. In this example, the value of y is de ned by one of two expressions of x depending on the value of x. Figure 6.1b shows the ow graph which has four basic blocks. Note that either bb1 or bb2 will be executed, but not both, and they both store their result in virtual register vr3. We assume that there are no further references to any of these virtual registers before the rst or after the last basic block.
During the register allocation phase, an interference graph [15][18] is typically constructed to determine the virtual registers that must be assigned to distinct physical registers. An interference graph corresponds to an undirected graph in which the vertices represent the virtual registers and an edge connects two vertices if one is live 213
a) Conditional statement if (x>10) y = 3*(x+2);
else y = 4*x+5;
c) Interference graph
b) Flow graph and live ranges for each basic block bb0
vr0
vr0 = load x vr0 vr0 vr1 vr3
bb1
t
vr1 = vr0+2 vr3 = 3*vr1
vr0>10
f
bb2
vr0 vr2 vr3
vr2 = 4*vr0 vr3 = vr2+5
vr1
bb3 vr3 Live ranges
vr2
vr3
store vr3,y
Figure 6.1: Register requirements of a conditional statement (Example 6.1). at the point where the other is de ned. To determine the live ranges, global data ow analysis is performed on the ow graph to determine the set of virtual registers that are live-in at the entry point of each basic block, and those that are live-out at the exit point of each basic block. Then, local analysis for each basic block determines the cycle-by-cycle live range of each virtual register that is live-in, live-out, and/or referenced in the basic block [2, pp 534{525]. This second step is highly machine dependent; we assume in our examples and experiments that a virtual register is reserved in the cycle where its earliest-de ne operation is scheduled and becomes free in the cycle following its last-use operation. However, our formulation treats the cycles in which a register is reserved and freed as parameters. Figure 6.1c presents the resulting interference graph. In bb1 for example, vr0 is live when vr1 is de ned; therefore, there is an edge between vertices vr0 and vr1 in the interference graph. Notice that there is no edge between vertices vr1 and vr2 since the thread of execution moves exclusively to one of the two basic blocks bb1 and bb2. The resulting interference graph can be colored by no less than 2 colors; consequently, the (minimum) register requirements of this conditional statement is 2. 214
a) Predicated block and live ranges bb0−1−2−3
Interference graph vr0 vr1 vr2 vr3
b) Traditional
c) Refined
vr0=load x vr0
p1(u),p2(u)=(vr0>10) vr1=vr0+2 ?p1;
vr2=4*vr0 ?p2
vr3=3*vr1 ?p1;
vr3=vr2+5 ?p2
vr1
vr0
vr2
vr1
vr2
store vr3,y Unconditional live ranges
Live range under p1
Live range under p2
vr3
vr3
Figure 6.2: Register requirements of an IF-converted conditional statement (Example 6.1). We now investigate the impact of predication on the register requirements of Example 6.1. Using IF-conversion, the four basic blocks of Figure 6.1b are merged into the single predicated block of Figure 6.2a. The operations previously executed in bb1 and bb2 are guarded, respectively, by the predicates p1 and p2. The two predicates are de ned in the second line by a special operation which sets p1 to true if the comparison succeeds and false otherwise. Similarly, p2 is de ned by the same operation to true if the comparison fails and false otherwise. Note that the values of p1 and p2 are disjoint, i.e. p1 and p2 cannot both evaluate to true for any instance of vr0. An interference graph using traditional live ranges is shown in Figure 6.2b. This interference graph contains an edge between vr1 and vr2 because vr1 and vr2 are considered live simultaneously, as traditional analysis does not examine the predicate expressions. As a result, the register requirements of this predicated block increase from 2 to 3. This increase in register requirements is mainly due to two reasons. First, without knowledge about predicates, the data ow analysis must make conservative assumptions about the side e ects of the predicated operations [2]. Second, the solutions of the data ow analysis rely heavily on the connection topology among basic blocks in the ow graph, which is altered by the IF-conversion process. 215
To eliminate the negative impact of predicated operations on the register requirements of a predicated block, one solution is to maintain the ow graph as predicated blocks are formed. This approach presents the advantage of using existing compiler techniques, but maintaining a consistent data ow graph through the several optimization phases may be complex and time consuming. For example, extracting instruction level parallelism relies heavily on moving operations among predicated blocks and on speculating operations, e.g. by promoting the predicate guarding operations. Therefore, all optimizations modifying predicates would have to maintain the consistency of the data ow graph at each step. Another approach is to reconstruct a data ow graph from the predicated blocks each time data ow analysis is needed [60]. This approach eliminates the complexity of maintaining a consistent data ow graph and requires no changes to the classical data ow analysis, traditional optimizations and register allocation. Additionally, the complexity of this approach is low, resulting in low compile time and good maintainability. However, this approach may produce in conservative results. Consider the code of Figure 6.3a where operation vr4=vr4+1 can be scheduled in any cycle after the load operation. If this operation is scheduled in cycle 2 (or in any cycles before or after all the predicated operations), reconstructing a ow graph results in the topology of Figure 6.3b. When scheduled in cycle 4, however, the graph is as in Figure 6.3d. Determining precise interference would then require the analysis phase to realize that both conditionals are correlated. In this chapter, we investigate an approach that directly takes into account the predicate expressions under which virtual registers are live. This approach presents none of the cited drawbacks of the previous approaches; moreover, precise information about predicate expressions of live ranges may also be extremely useful to other optimizations, such as memory disambiguation [90]. The drawback of this approach is its reliance on a more expensive analysis, partly due to the use of a symbolic package to detect predicate expression disjointness and partly due to the computation of a more complicated interference relation. However, ongoing research on ecient algorithms for precise or approximate analysis of predicated code [51] will signi cantly improve 216
a) Increment operation in cycle 2
b) Equivalent flow graph vr0 = load x vr4=vr4+1;
vr0=load vr4; vr4=vr4+1;
t
p1(u),p2(u)=(vr0>10)
f
vr0>10
vr1 = vr0+2 vr3 = 3*vr1
vr2 = 4*vr0 vr3 = vr2+5
vr1=vr0+2 ?p1; vr2=4*vr0 ?p2 vr3=3*vr1 ?p1; vr3=vr2+5 ?p2 store vr3,y
store vr3,y
c) Increment operation in cycle 4
d) Equivalent flow graph vr0 = load x
vr0=load vr4;
t
vr0>10
vr1 = vr0+2
p1(u),p2(u)=(vr0>10) vr1=vr0+2 ?p1; vr2=4*vr0 ?p2 vr4=vr4+1; vr3=3*vr1 ?p1; vr3=vr2+5 ?p2
f
vr2 = 4*vr0
vr4=vr4+1; t vr3 = 3*vr1
vr0>10
f
vr3 = vr2+5
store vr3,y store vr3,y
Figure 6.3: Reconstructing a data ow graph. the execution time of this approach. We now present the mechanisms required by our framework in more detail.
6.1.1 Predicate Extraction The predicate extraction mechanism allows the compiler to nd how predicates are de ned and used. This knowledge is expressed as logically invariant expressions, or P-facts, that are guaranteed to hold, regardless of the execution trace, the results of comparisons, and the values of other predicates. These P-facts will be used in later phases of the analysis to determine feasible execution paths, i.e. execution paths that do not violate any of the P-facts gathered during this initial extraction phase. Note that failing to extract a P-fact does not compromise the integrity of the analysis, it merely causes more conservative results. This extraction mechanism is very sensitive to the instruction set architecture, since it relies on the precise semantics of the operations that de ne the predicate val217
ues. In this work, we adopt the predication scheme of the HPL PlayDoh architecture [54]. Support for predicated execution is provided by a le of 1-bit predicate registers which holds the predicate values, an additional source operand for most operations to specify a predicate register, and a rich set of operations to de ne values for these predicate registers. Predicate registers are allocated like any other registers in the machine. Each predicated operation is associated with an additional predicate register operand which speci es the predicate value under which the operation may a ect the processor state. The semantics of a predicated operation is de ned as follows:
op>
Dest =
), Pout1() = (src0 src1) ? Pin
Pout0(
15) S1; else S2;
p1(u),p2(u) = (i>15) ?p0 S1 ?p1; S2 ?p2;
if(i>10) S4; if(i>5) S5;
p4(u) = (i>10) ?p3; p5(u) = (i>5) ?p3; S4 ?p4; S5 ?p5;
P-facts p1 , (x1 ^ p0) p2 , (x1 ^ p0) p4 , (x2 ^ p3) p5 , (x3 ^ p3) x 2 ) x3
Table 6.1: Extracting P-facts. types are required to evaluate several comparisons in parallel, as advocated in [87]. For example, the or type may be used to implement a \wired-or" function of several comparisons executed in parallel. Based on the precise semantics of the predicate type used by a compare-to-predicate operation, we may construct a corresponding logically invariant expression that is guaranteed to hold regardless of the run time program values. Table 6.1 presents the P-facts extracted from two simple code fragments. The rst example illustrates a simple predicated if-then-else construct similar to Example 6.1. Following the unconditional type semantics (u type), the rst P-fact asserts that p1 is true when both the comparison and the input predicate p0 are true. Following the negated unconditional type semantics (u type), the second P-fact asserts that p2 is true when the comparison is false and the input predicate p0 is true. P-facts do not make any assumptions about the result of a comparison: P-facts merely assume that a comparison will evaluate either to true or false, resulting in the described e ect on the predicate values. Therefore, comparisons are simply abstracted as logical variables; in our rst example, the comparison (i>15) is abstracted as the variable x1. The second example in Table 6.1 illustrates two correlated predicates. We notice that the two conditions (i>10) and (i>5) are correlated, since i larger than 10 implies i larger than 5. The third P-fact expresses this correlation.
219
6.1.2 Live Range Analysis In this section, we compute the predicate expression under which a virtual register is live. We consider live ranges de ned and used by one or more operations within a single predicated block. Consider a virtual register vr and operation i, producing a value of vr during the cycle interval tpei to tpli, namely from the earliest production time to the latest in which operation i may write back its results in vr. For example, the earliest production time of a divide operation corresponds to the earliest time where the divide operation may write its result back (e.g. divide by 1), and the latest production time corresponds to the latest possible time. Consider also operation j , consuming a value of vr during the cycle interval tsej to tslj , namely from the earliest sampling time to the latest sampling time in which operation j may read the value of vr. In a single unpredicated basic block, a value ows from operation i to operation j only if no intervening operation produces a value for the same virtual register between operations i and j . In a predicated block however, the predicate under which operations execute must be taken into consideration. A value e ectively ows from operation i to j if the predicates of operations i and j both evaluate to true and the predicates of all producer operations that could overwrite that value evaluate to false. The exact condition under which a value ows from operation i to operation j is referred to as the ow condition and is determined by the following conjunction of predicates:
pi ^ pj
^
pk
(6.2)
tpek > tpli tplk < tsej
where the pk are the predicates of all the def-operations of vr that may potentially overwrite vr. Figure 6.4a illustrates a data ow graph with two producer and two consumer operations of virtual register x. The ow condition associated with each edge is also 220
a) Data flow graph tpe tpl
tpe tpl
b) Flow− c) Def−components (p0)
x= ?p0 (p0 & ~p1 & p2)
(p0 & ~p1 & p2)
(p0 & ~p1 & p3)
(p0 & ~p1 & p3) (p1)
x= ?p1
(p1 & p2)
(p1 & p2) tse tsl
(p1 & p3)
=x ?p2 (p1 & p3)
tse tsl
time
=x; ?p3 (flow conditions) producer
consumer
(def conditions) reaching definition
Figure 6.4: Flow and Def component of a live range. presented in this gure. For example, a value e ectively ows from the rst producer to the last consumer when the condition p0 ^ p1 ^ p3 evaluates to true, i.e. when both the rst producer and the last consumer operations execute, and the intervening producer operation does not. We investigate now the contribution to the live range of vr generated by operations i and j . One contribution to the live range of vr is associated with the data ow between operations i and j . This contribution is associated with the cycle range [tpei; tslj ]. This contribution is referred to as the ow component and is live under the ow condition associated with the edge between operations i and j . Figure 6.4b illustrates the ow component for each of producer-consumer pair of x. Another contribution is associated with the cycle range [tpei; tpli]. This contribution is referred to as the def component and is live under the predicate guarding operation i, referred to as the def condition. Note that the def components exist regardless of whether the produced values are used. Figure 6.4c illustrates the def components for the producers of x.
221
b) Predicated live range y
a) Predicated live range x (p0)
x= ?p0
(p0 & ~p1 & p2) (p0 & ~p1 & p3) 1 x= ?p1
(p4)
y= ?p4
(p1) 2
(p4 & p5)
(p1 & p2) (p1 & p3) =x ?p2
c) Interference if and only if 1
(p0 & ~p1 & p2 & p4)| (p0 & ~p1 & p3 & p4)
2
(p1 & p4 & p5)
time
=x; ?p3 flow component
=y ?p5
def component
Figure 6.5: Determining interferences.
6.1.3 Interference In this section, we compute the interference between two predicated live ranges based on Chaitin's de nition: two live ranges interfere if one of them is live at a de nition point of the other [18]. This de nition implies that overlapping def components do not interfere, provided no operations use these values. In the context of predicated blocks, this de nition implies that simultaneous writes to a unique physical register should be tolerated by the hardware of the machine. In this chapter, we assume that simultaneous writes are allowed, resulting in an unspeci ed value. Figures 6.5a and 6.5b illustrate the predicated live ranges of virtual registers x and y, respectively. We notice that two def components, marked 1 and 2, each overlap the ow components of the other virtual register. We use the ow and def conditions associated with each component to determine the predicate expression under which x and y may interfere, as shown in Figure 6.5c. For example, the second def component of x interferes with the ow component of y (interval labeled 2) if there is a legal execution path in which expression p1 ^ p4 ^ p5 evaluates to true. To see if this is possible, the compiler queries whether this expression is consistent with all of the P-facts, which are known to be satis ed by 222
all legal paths. If the conjunction of the query expression with all the P-facts does not evaluate to false, the compiler must assume that there is some legal path that may satisfy the query expression, i.e. that there may be interference between x and y. Otherwise, the conjunction evaluates to false, indicating that there is no legal execution path that satis es consistency of the query expression with all the P-facts, i.e. these particular components of x and y do not interfere. In this chapter, we consider query expressions that are of the form pi1 ^ : : : ^ pin ^ pin +1 ^ : : : ^ pin , i.e. a conjunction of predicate terms and negated predicate terms. Consistency between a query expression expr and the P-facts is checked by function consistent(expr). This function investigates whether the following equation can be proven: ! m ^ (6.3) P-factk ^ (expr) 6= false 0
0
k=1
returning true when Inequality (6.3) is proven true, and false otherwise. This function can be implemented in several ways; in this work, we use a symbolic package that simpli es logic equations. An interference algorithm based on Chaitin's de nition of interference is presented in Figure 6.6. Note that this algorithm relies on the speci c scheduling time of each operation. When interferences are computed before scheduling, we must make a conservative assumption on the scheduling time of each producer and consumer operation. Function Live-at-def investigates if any of the def components of x may interfere with any of the ow components of y. These tests are performed in function Def-at- ow. Consider prod-x, prod-y, and cons-y to be, respectively, a producer of x, a producer of y, and a consumer of prod-y. First, Def-at- ow determines if the def component of prod-x overlaps in time with the ow component associated with the producer-consumer pair (prod-y, cons-y). If it does, the ow condition expr is computed, as de ned in Section 6.1.2, and the conjunction of the def condition and the ow condition is checked for consistency with the P-facts. When it is, the compiler must assume that prod-x may interfere with the value owing from prod-y 223
procedure Def-at- ow (Op prod-x, Op prod-y, Op cons-y) f
1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12:
g
/* Determine if prod-x overwrite the value owing from prod-y to cons-y. */ if def-component(prod-x) overlaps with
ow-component(prod-y, cons-y) then expr = predicate(prod-y) and predicate(cons-y); for each other producer of y strictly in interval(prod-y, cons-y): prod-y' do expr = expr and not predicate(prod-y');
endfor return consistent(expr and predicate(prod-x)); endif return false;
procedure Live-at-def (Live Range x, Live Range y) f 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: 24:
g
/* Determine if the producers of x interfere with live range y */ for each producer of x: prod-x do for each producer of y: prod-y do for each consumer of prod-y: cons-y do if Def-at- ow(prod-x, prod-y, cons-y) then return true;
endif endfor endfor endfor return false;
procedure Interference (Live Range x, Live Range y) f 25: 26:
g
/* Determine if live ranges x and y interfere */ return Live-at-def(x, y) or Live-at-def(y, x);
Figure 6.6: Interference algorithm. 224
to cons-y and thus consider the live ranges x and y to interfere. If no def component of x or y is detected to interfere with a ow component of the other, then vrx and vry do not interfere and the edge between them is removed from the interference graph. As presented in Figure 6.6, the algorithm has a computational complexity of O(n4), where n is the number of producers and consumers. However, the loop in line 6 can be computed in advance for each live range, thus reducing the complexity of the algorithm to O(n3). To reduce the cost of detecting live range interferences, a regular interference algorithm can be used rst, followed by a predicated analysis phase used to remove spurious interference edges.
6.2 Post-scheduling Bundling For register allocators based on Chaitin's graph coloring framework [15][18], register allocation for predicated code can be achieved simply by using the re ned interference graph instead of the conventional one. However, several register allocators depart from the graph coloring method [46][82] as graph coloring methods do not provide a notion of time that is particularly useful for the live ranges of a loop, which may cross the boundary of an iteration. Also, non-traditional constraints such as the one presented in [82] to support various code generation and hardware support schemes are dicult to express within the graph coloring framework. To address the allocation of registers in loops with predicated code, we propose a technique that reduces the register requirements by allowing non-interfering virtual registers that overlap in time to share a common virtual register. Once virtual registers have been suitably bundled, a traditional predicate-insensitive register allocator can be used. In this section, we investigate this technique, referred to as bundling, after completion of the scheduling process. The key issue is to determine which virtual registers should be bundled together. We propose a parameterized heuristic that determines advantageous bundling. 225
First, the interferences between virtual registers of a predicated block are computed using the algorithm presented in Section 6.1.3. Second, the virtual registers are placed on the list of unprocessed virtual registers, ordered according to a heuristic. Then, the bundling algorithm selects the rst virtual register on that list and assigns it to a suitable bundle. Several selection heuristics can be speci ed to identify a suitable bundle. A simple heuristic chooses the rst bundle in which the current virtual register does not interfere with any of the virtual registers already in the bundle. A more elaborate heuristic would select the bundle that minimizes some cost function. Once a bundle is selected, the current virtual register is added to the bundle and the bundling heuristic selects the next unprocessed virtual register. This algorithm thus maps virtual registers to bundles. The virtual registers in a bundle are guaranteed to be compatible (no pairwise interference), and can be reassigned to a unique virtual register. The remainder of this section discusses the ordering and the bundle selection heuristics. In this discussion, we de ne the live range of a virtual register as the interval [tpei ; tslj ] where operations i and j are, respectively, the earliest producer and latest consumer of that virtual register. By extension, we de ne the live range of a bundle as the union of the live ranges of all virtual registers of that bundle. Furthermore, a virtual register is compatible with a bundle if it does not interfere with any of the virtual registers of that bundle.
6.2.1 Ordering Heuristics Arranging the virtual registers in a suitable order is critical for heuristics that consider the bundling of registers only one virtual register at a time. We present here some of the ordering heuristics we investigated. Ordering heuristics can be combined to break ties.
Start-Time. Arrange the virtual registers by increasing start-time of their live ranges. This heuristic tries to capture the time locality in the dependence graph.
226
Decreasing-Length. Arrange the virtual registers by decreasing length of their live ranges. Bundling virtual registers with long live ranges rst is expected to result in a larger decrease in register requirements.
Decreasing-Interference. Arrange the virtual registers by decreasing number of
interferences. Bundling virtual registers that interfere with the largest number of virtual registers rst is expected to result in a larger decrease in register requirements.
6.2.2 Selection Heuristics The selection heuristics place the current virtual register in a suitable bundle. When a selection heuristic tries every bundle unsuccessfully, a new bundle is created at the end of the bundle list and the current virtual register is added to it.
First-Compatible Bundle. Choose the rst bundle for which the current virtual
register is compatible and their live ranges intersect.
Last-Compatible Bundle. Choose the last bundle for which the current virtual register is compatible and their live ranges intersect. This heuristic presumes that more recently created bundles are more likely to accept new virtual registers than older bundles.
Best-Compatible Bundle. Choose the best bundle in which the current virtual
register is compatible and their live ranges intersect. The best bundle is the one that maximizes the intersection of the current virtual register live range and the bundle live range (empty intersection not considered).
Best Bundle. Choose the best bundle in which the current virtual register is par-
tially compatible and their live ranges intersect. This heuristic rst computes for each bundle the set of virtual registers that interfere (inter) and the set the set of virtual registers that do not interfere (comp) with the current virtual register (vr). Then, it computes the number of cycles (vr in) in which the live ranges of comp and the live range of vr intersect. It also computes the number of cycles (vr out) in which one 227
or more live ranges of comp and the live ranges of inter intersect. The best bundle is the one that maximizes vr in - vr out (values 0 not considered). If a bundle is selected, the virtual registers in inter are removed from the bundle and restacked on the list of unprocessed virtual registers. This selection heuristic is guaranteed to complete because virtual registers are restacked only if the overall bene t strictly increases (note that the maximum possible overall bene t is nite).
6.3 Pre-scheduling Bundling We investigate in this section the process of bundling compatible live ranges before scheduling. The motivation for bundling before scheduling is that a register-sensitive scheduler may better schedule the operations of a bundle if the register-sensitive scheduler is aware of which virtual registers belong to a bundle. With pre-scheduling bundling heuristics, interferences between virtual registers of a predicated block cannot be computed as precisely as in Section 6.1.3 since the scheduling time of the operations is unknown. Instead, we must rely on the earliest and latest feasible scheduling time of an operation [43] [49]. In our current implementation, the pre-scheduling bundling heuristics rely on a weaker form of interference where two virtual registers are considered to interfere if any two of their producer operations do not execute under mutually disjoint predicates. A more elaborate approach might alleviate this restriction by adding scheduling edges in the dependence graph to enable advantageous bundles. In this scheme, additional scheduling edges would supply some further temporal constraints on the subsequent scheduling. Some edges are guaranteed not to actually constrain the scheduler, as shown by Pinter [75]. Others, however, may constrain the scheduler. In the context of modulo scheduled loops, the danger of this approach resides in the fact that adding scheduling edges may, if not carefully applied, overly constrain the schedule of predicated blocks and may thus result in schedules with increased schedule length or decreased throughput. For this reason, we did not investigate this approach. 228
6.4 Measurements We investigated the register requirements of the integer and oating point register le for a subset of the 1327-loop benchmark suite as compiled for the Cydra 5 and presented in Section 2.7. From the 1327-loop benchmark suite, we selected all the loops with at least one pair of mutually disjoint predicate values, resulting in a benchmark of 21 loops. In the current implementation, we did not attempt to correlate any of the IF-converted branches. Table 6.2 summarizes the principal characteristics of the benchmark suite. The loops in the benchmark suite are signi cantly more complex than the average loop in the input set, with 51.43 operations and 61.62 register edges on average, versus 17.54 and 22.30 for the input set, respectively, as reported in Table 4.2. As a result of this bias, the loops of the benchmark suite require an average of 39.3 registers versus 27.9 in the input set when using the scheduling technique described in Chapter 4 which minimizes the register requirements for a modulo schedule. As it contains loops with larger register requirements, decreasing the register requirements of the loops in this benchmark suite may be crucial. Measurements: min freq median average max number of operations 11:00 4:8% 50:00 51:43 107:00 register edges 9:00 4:8% 62:00 61:62 132:00 scheduling edges 12:00 4:8% 66:00 69:48 155:00 II 4:00 4:8% 16:00 26:86 109:00 number of predicate values 2:00 50:0% 2:00 3:05 9:00 disjoint pairs of predicates 1:00 72:7% 1:00 1:95 13:00
Table 6.2: Characteristics of the 21-loop benchmark suite. In this chapter, we used the MinReg Stage-Scheduler and Iterative Modulo Scheduler to produce modulo schedules with high performance and low register requirements.
229
Fraction of loops scheduled
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3
Schedule Independent Lower Bound
0.2
Best Post-Scheduling Bundling Heuristic
0.1
MinReg Stage Scheduler Iterative Modulo Scheduler
0 8
16
24
32
40
48
56
64
72
80
88
Registers
Figure 6.7: Register requirements for the 21-loop benchmark suite. We rst investigate the register requirements of these two schedulers and the register requirements of our best bundling heuristic1. Figure 6.7 presents the fraction of the loops in the benchmark suite that can be scheduled for a machine with any given number of registers without spilling and without increasing II . The \Schedule Independent Lower Bound" curve corresponds to [49] and is representative of the register requirements of a machine without any resource con icts, but does not support bundling, i.e. it ignores the possible decrease in register requirements due to the bundling of compatible virtual registers. Notice that none of the schedulers, including our best bundling heuristic, approaches the register requirements of the Schedule Independent Lower Bound. This large gap is partly explained by the strong simpli cation (no resource con ict) of the lower bound, a gap that is particularly apparent for large loops and negligible for small loops [36]. The second and third curves illustrate the decrease in register requirements due to The best bundling heuristic corresponds to a post-scheduling bundling heuristic using the Decreasing-Length ordering (with Decreasing-Interference for tie-breaking) and the Best-Bundle selection. 1
230
the bundling of compatible virtual registers when applied to the schedules produced by the MinReg Stage Scheduler. Using our best bundling heuristic, the average register requirements decrease from 39.3 to 36.5 registers and the number of loops with no more than 64 registers increases from 18 to 20 of the 21 loops. Note that this improvement is obtained only for schedules for the given MRT. The lowest curve illustrates the register requirements associated with the Iterative Modulo Scheduler and is indicative of the register requirements of a good modulo scheduler that currently does not attempt to minimize the register requirements. To investigate the performance of the bundling heuristics, we introduce a lower bound on the register requirements:
Predicate-Sensitive Lower Bound. A lower bound on the register requirements
for a given loop schedule and set of P-facts is obtained by bundling the virtual registers on a cycle-by-cycle basis. This clearly represents a lower bound, as normally virtual registers are allowed to be bundled on a register-by-register basis only.
We focus here on the decrease in register requirements achieved by a heuristic, normalized to the decrease in register requirements from no-bundling to the PredicateSensitive Lower Bound. The schedule used to investigate the post-scheduling bundling heuristics was obtained by the MinReg Stage Scheduler. In our benchmark suite, the average register requirements without bundling was 39.3 and the Predicate-Sensitive Lower Bound was 36.2. In the following paragraphs, a decrease in register requirements of 100% corresponds to a bundling heuristic that achieves the lower bound and 0% corresponds to a bundling heuristic that does not decrease the register requirements at all. All numbers are averaged over the 21 loops of the benchmark suite. We rst investigate the performance of the bundle selection heuristics presented in Section 6.2.2. Table 6.3 presents the normalized decrease in register requirements associated with each of four selection heuristics, averaged over all loops and ordering heuristics. Note that the last two selection heuristics require a cost function to evaluate the intersections of the virtual register live range and the bundle live range. We 231
Cost Function none intersection Bundle Selection Heuristic First-Compatible 72% | Last-Compatible 75% | Best-Compatible | 72% Best | 90%
Table 6.3: Normalized decrease in register requirements for the selection heuristic. may conclude from Table 6.3 that the Last-Compatible Bundle Heuristic is the better selection heuristic without cost function, and that Best Bundle Heuristic, which is the least sensitive to the ordering of the virtual registers, is the best selection heuristic overall. Cost Function Ordering Heuristic Reference Start-Time Length Interference Length+Interference
none intersection 61% 80%
85% 76% 85%
76% 85% 86% 84%
87%
Table 6.4: Normalized decrease in register requirements for the ordering heuristics. The performance of the ordering heuristics described in Section 6.2.1 is summarized in Table 6.4. Each table entry corresponds to the normalized decrease in register requirements achieved by an ordering heuristic, averaged over all loops and selection heuristics that share the same cost function. The rst row, labeled \Reference" corresponds to the decrease in register requirements averaged over the initial and reverse orderings. Comparing the numbers of the other ordering heuristics to the reference numbers provides insight on the achieved decrease in register require232
Fraction of loops scheduled
1 0.9 0.8 0.7 0.6 0.5
Post-Scheduling bundling heuristic Best/Length+Interf.
0.4 0.3
Post-Scheduling bundling heuristic Last Compatible/Length
0.2
Pre-Scheduling Bundling Heuristic Best/Length MinReg Stage Scheduler
0.1
Iterative Modulo Scheduler 0 0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
Additional registers
Figure 6.8: Additional register requirements of the pre- and postscheduling bundling heuristics relative to the PredicateSensitive Lower Bound. ments due to the ordering heuristics. We conclude from Table 6.4 that the ordering heuristics do a ect the register requirements, and that the largest decrease is generated by the Length+Interference ordering heuristic which uses Decreasing-Length with Decreasing-Interference for tie-breaking. The individual results con rm the above ndings, with the best results for the Best/Length+Interference, Best/Length, and Best/Start-Time heuristics, decreasing the average register requirements from 39.3 to 36.52 registers, followed by Best/Interference and Last-Compatible/Length, decreasing the average register requirements form 39.3 to 36.57 registers. All of these ve post-scheduling bundling heuristics are less than 1% above the Predicate-Sensitive Lower Bound. Figure 6.8 illustrates the number of additional registers needed by the bundling heuristics and other scheduling algorithms when compared to the Predicate-Sensitive Lower Bound. From top to bottom, the curves represent the additional registers needed by two post-scheduling bundling heuristics, a pre-scheduling heuristic, the 233
MinReg Stage Scheduler, and the Iterative Modulo Scheduler. Notice that the two post-scheduling bundling heuristics reduce the register requirements signi cantly, with all loops scheduled using no more than 2 additional registers above the possibly unattainable Predicate-Sensitive Lower Bound. The decrease in register requirements of the pre-scheduling bundling heuristic is signi cantly less, with all loops scheduled using no more than 8 additional registers. This disappointing performance is due, we believe, to the fact that our pre-scheduling bundling approach uses a more conservative de nition of non-interfering live range. A more elaborate approach could alleviate this situation by adding scheduling edges in the dependence graph to enable advantageous bundling, at the risk of increasing the schedule length or decreasing the throughput of the schedule. In its current form however, experimental evidence suggests that the bundling of compatible virtual registers, without adding scheduling edges in the dependence graph, is more successful as a post-scheduling optimization phase than as a pre-scheduling phase.
6.5 Summary In this chapter, we discussed the need for better analysis of predicated code by showing an example of code where the register requirements increase when several basic blocks are merged into a single predicated block. To remedy this increase in register requirements, we proposed a framework that analyzes predicated codes by extracting P-facts from the predicated block, analyzing the predicate expressions associated with live ranges, and re ning Chaitin's interference de nition to account for predicate expressions. This framework generalizes to arbitrary predication schemes and may capture correlations between IF-converted branches. Register allocators based on the graph coloring framework can simply use the re ned interference graph to perform register allocation on predicated code. For register allocators based on interval graphs, or others, we propose a new technique that bundles non-interfering virtual registers to lower the register requirements of a predicated block. Bundling may be performed after the scheduling process to capitalize on 234
the availability of the actual scheduling time of each operation to determine precisely the interferences among virtual registers. Alternatively, bundling before scheduling capitalizes on the fact that a register-sensitive scheduler may better schedule the operations associated with a pre-bundled virtual register. Results using our best post-scheduling bundling heuristic decreased the average register requirements from 39.3 to 36.5 registers, just 1% above the Predicate-Sensitive Lower Bound, and increased the number of loops with no more than 64 registers from 18 to 20, in a benchmark suite of 21 modulo scheduled loops extracted from the Perfect Club, SPEC-89, and the Livermore Kernels. These results also suggest that bundling compatible virtual registers, without adding scheduling edges in the dependence graph, is more successful when applied after rather than before scheduling.
235
CHAPTER 7 Conclusions 7.1 Summary of Contributions High performance for VLIW and superscalar processors is achieved by exposing the inherent parallelism in an application, capitalizing on the exposed instruction level parallelism to achieve an ecient instruction schedule that utilizes the machine well, avoids resource con icts, and minimizes unmasked stall cycles. To better utilize machine resources and reduce or prevent resource contentions, high performance compilers use precisely detailed microarchitecture representations. Ecient representations are important since high performance compilers spend a signi cant amount of compilation time scheduling operations, and thus testing for potential resource contentions. For example, our measurements indicate that approximately half the scheduling time is spent in the contention query module, when using a state-of-the-art modulo scheduler in a benchmark suite of 1327 loops from the Perfect Club, SPEC-89, and the Livermore Fortran Kernels compiled for the Cydra 5. We addressed this issue in the context of high-performance compilers by proposing an ecient contention query module that supports the elaborate scheduling techniques used by today's high performance compilers. Our module is based on a reduced machine description that results in signi cantly faster detection of resource contentions while exactly preserving the scheduling constraints present in the original machine description. This approach achieves two major goals. First, it handles queries 236
signi cantly faster: for example, dynamic measurements obtained when scheduling the 1327-loop benchmark for the Cydra 5 using the Iterative Modulo Scheduler [80] indicates an average decrease in work units (i.e. the essential work performed by the contention queries) by a factor of 2.76, resulting in queries that complete on average in 58.2% of the time when a reduced machine description is used instead of the original machine representation. Second, the reduced machine description approach does not limit the functionality of the contention query module. Additional functionality that is fully supported here, and used in our experiments, allows scheduling an operation earlier than others that are already scheduled, unscheduling operations due to resource contentions, and ecient handling of periodic resource requirements found in software pipelined schedules. Thus, by using our reduced machine description, the computational requirements of these scheduling algorithms, and other similar algorithms, should decrease as they would no longer be penalized by less ecient machine resource models. While most machine resources can be eciently modeled using contention query modules, other machine resources, such as registers, are typically not handled by such modules. In the case of the registers, this problem is further aggravated by the fact that schedules exhibiting high concurrency generally result in higher register requirements as well. We addressed this issue by investigating register-sensitive schedulers in the context of software pipelined loops. Our approach combines optimal algorithms to gain understanding and fast heuristics to improve compiler technology. We rst investigated an optimal algorithm for modulo scheduling (MinReg ModuloScheduler) that schedules loop operations to achieve the minimum register requirements for the smallest possible initiation interval. Our approach has extended the applicability of optimal modulo scheduling algorithms to machines with nite resources and arbitrary reservation tables. We demonstrated this increased modeling capability by precisely handling the resource requirements of the Cydra 5, a machine with complex reservation tables. Our approach also uses a well structured formulation of the modulo scheduling solution space, which signi cantly reduces the traditionally high execution time of 237
modulo schedulers based on integer linear programming formulations. Experimental evidence with the 1327-loop benchmark indicates an average decrease in the number of branch-and-bound nodes (visited by the solver) by a factor of 103, resulting in an average decrease in execution time by a factor of 8.6, when using the structured formulation instead of the traditional formulation of the MinReg Modulo-Scheduler on the 781 loops successfully scheduled by both formulations using no more than 15 minutes per loop. Also, using the more ecient representation enabled us to successfully nd a minimum MaxLive schedule in no more than 15 minutes for more loops (increasing from 781 to 920 loops) and larger loops (increasing from a maximum of 25 to a maximum of 41 operations). Furthermore, our approach fully capitalizes on the strength of optimal solvers by minimizing a precise formulation of the actual register requirements. Compared to approximating the register requirements by the average lifetime, the most e ective approximation that we found in the literature, minimizing MaxLive results in schedules with lower register requirements for 13.0% of the 1327 loops. Similarly, compared to approximating the register requirements by bu ers, minimizing MaxLive results in schedules with lower register requirements for 21.4% of the 1327 loops. Consequently, modulo schedulers based on integer linear programming formulations should be more applicable (because of their reduced computational complexity and accurate characterization of their register requirements for machines with arbitrary reservation tables) to situations where a achieving schedule with the minimum register requirements for the smallest possible initiation interval is highly desirable. For situations where the computational complexity of this approach exceeds practical considerations (e.g. for the 11.0% of the 1327 loops for which the NoObj ModuloScheduler does not complete in less than 15 minutes), or for situations where a schedule must be found quickly, we investigated an alternative approach to register-sensitive modulo scheduling with far reduced computational complexity. Our second contribution to register-sensitive modulo schedulers, is a technique, referred to as stage scheduling, that reduces the register requirements of a given MRT by shifting operations by integer multiples of II cycles. The major advantages of the 238
stage scheduling approach are its reduced complexity relative to the full scheduling problem above and the fact that lower register requirements are achieved without impacting the steady-state performance of the original modulo schedule. Furthermore, by bounding the maximum schedule length of an iteration to be no more than that of the original schedule, a signi cant decrease in register requirements can be achieved with no impact on the transient performance of the schedules as well. We contribute an optimal stage scheduling algorithm (MinReg Stage-Scheduler) that nds a schedule with minimum register requirements among all stage schedules that share a given MRT. We also contribute a set of stage scheduling heuristics that closely approximate the minimum solution at linear, or at most quadratic1 computational complexity. Our results indicate that for all the 920 loops successfully scheduled by the MinReg Modulo Scheduler in no more than 15 minutes per loop, the average register requirements are decreased by 22.2% by using the optimal modulo scheduler rather than a register-insensitive modulo scheduler. When using the optimal stage scheduler in conjunction with the register-insensitive Iterative Modulo Scheduler [80], the average register requirements decrease by 19.9%. Moreover, the average register requirements decrease by 19.8 and 19.2% when our best stage scheduling heuristics with unlimited schedule length and bounded schedule length, respectively, are employed. Interestingly, even the best bounded stage scheduling heuristic (using the MRTs produced by the register-insensitive Iterative Modulo Scheduler) results in lower average register requirements than when approximating the actual register requirements by bu ers and searching for a minimum bu er schedule among all MRTs. Thus, for situations where the computational complexity of the MinReg ModuloScheduler exceeds practical considerations, we recommend the following approach to achieve a high-performance, low register requirements modulo-schedule. First, search for a high-performance schedule using a state-of-the-art modulo scheduler, such as Rau's Iterative Modulo Scheduler. The level of performance achieved by this scheduler is remarkable: it was known to result in the minimum initiation interval for 96.0% of the 1327 loops; furthermore, we have shown that it actually achieves the minimum 1
Provided that the MinDist relation is given as input.
239
feasible initiation interval for 22 more loops, thus resulting in optimal throughput for 1296 (or 97.7%) of the 1327 loops. Note also that this result is achieved for the Cydra 5, a machine with very complex resource requirements. Second, to decrease the register requirements, if desired, using a hierarchy of stage scheduling heuristics. As shown in Chapter 4, linear time stage scheduling heuristics achieve a respectable decrease in register requirements, and should suce in most cases. If a larger decrease in register requirements is desirable, our best stage scheduling heuristics can be employed. If further reductions in register requirements are needed, we can compute a lower bound on the register requirements, as proposed by Hu [49], to estimate the likelihood of nding schedule with lower register requirements, by searching for either a new MRT-schedule, a minimum MaxLive stage schedule, or a combination of both. Increasing II or introducing spill code may also be required for loops with extreme register requirements or machines with small register les. The resulting schedule should have a signi cantly reduced register requirements, while preserving the steady-state performance of the initial register-insensitive schedule, since II is preserved during the stage scheduling process. Furthermore, the transient performance can also be preserved if the schedule length is bounded to be no more than that of the initial schedule. As a result the approach outlined here should result in a high throughput, low register requirements schedule that is close to the optimal solution since the Iterative Modulo Scheduler achieves the minimum feasible II for 97.7% of the 1327 loops, and the unbounded and bounded best stagescheduling heuristics achieve minimum MaxLive among all stage schedules that share the given MRT for, respectively, 95.9% and 91.6% of the 1327 loops. Finally, the register requirements can be further decreased in predicated code by computing the interferences among virtual registers in the presence of predicated operations and allowing noninterfering virtual registers to share a physical register even when their lifetimes overlap.
240
7.2 Future Research The work presented in this dissertation can be extended in numerous ways. First, the work on reduced machine representation indicates that generating a synthesized machine with fewer resources and resource usages signi cantly decreases the work units performed by the queries. Extending and broadening this result for machines with large numbers of alternative operations would be extremely interesting. One approach would be to factor subsets of resource usages among alternatives such that the query module can more quickly determine if none of the alternatives may be scheduled in a given cycle. Another approach would be to use counters to keep track of resources with multiple instances that are indistinguishable (i.e. each operation can use any resource instances). Counters may be integrated nicely in an extension of the bitvector scheme, if desired. Also, when alternative resources are not completely indistinguishable, there may be some mixture of counters and synthesized resources that can eciently represent such resources. Another direction of research for ecient representation of machine resources is to extend the nite state automaton approach to handle some of the functionality that is now only provided by the reservation table approach. For example, there may be an elegant way to implement the functionality of the assign&free function within the nite state automaton approach. A third direction of research is to investigate if, in presence of multiple alternatives, some knowledge can be integrated into the nite state automaton approach. For example, if in a given state of the automaton, one alternative is frequently selected, but also frequently unscheduled due to resource contention, there may be a way to incorporate this knowledge into the automata, by reducing the priority of that alternative relative to others. The work on optimal modulo schedulers can also be extended to cover a wider range of machines, such as machines with clusters of functional units. It can also be extended to consider optimal generation of spill code, as proposed for straight line code by Meleis and Davidson [66]. Beyond extending the functionality of the formulation, nding more ecient formulations would be very useful. 241
There are also numerous directions of research for ecient modulo scheduling heuristics, such as integrating scheduling and spilling heuristics, generating modulo schedules that include only some of the basic blocks found within a loop iteration, or integrating unrolling before scheduling with the unrolling required by the modulo variable extension. Also interesting would be to extend modulo scheduling heuristics to eciently handle machines with clusters of functional units, perhaps along the lines of the scheduling technique proposed by Ellis [41] for straight line code.
242
APPENDIX A Completion of Proof of Theorem 4.1 We demonstrate here that the stage scheduling problem for minimum integral MaxLive can be solved with a linear programming (LP) solver. In particular, we show that if an LP-solver nds an optimal solution, the optimal solution is integral. In the rst step of the demonstration, we reformulate the stage scheduling problem as de ned in Algorithm 4.1 and Figure 4.7. We express here the stage scheduling problem using the traditional integer linear programming formulation:
zILP = minfcx : Ax b; x 2 Z+n g
(A.1)
namely, we formulate the constraint matrix A and the vector b that de ne the search space and we express the vector c that de nes the objective function for the minimum integral MaxLive stage scheduling problem. Note that Equation (A.1) de nes an integer linear programming problem since the solution vector x explicitly belongs to Z+n , i.e. x is an n component vector of nonnegative integers. This rst step is developed in Section A.1. In the second step of the proof, we demonstrate that the matrix A and vectors b and c have properties that ensure that the solution of the LP-relaxation of Equation (A.1):
zLP = minfcx : Ax b; x 2 Rn+ g 243
(A.2)
, which permits nonnegative reals, does in fact result in an integer value for zLP , i.e. the zLP and zILP solutions are both integer and have the same cost, when an LP-solver is successfully used for zLP . Note that since the solution vector x in Equation (A.2) belongs to Rn+ instead of Z+n , an LP-solver can be used to solve Equation (A.2), as presented in Section A.2.
A.1 Formalization of Stage-Scheduling Problem As de ned in Section 4.3, the minimumintegral MaxLive stage-scheduling problem has the following objective function: minimize:
Bu =
X i2V
bi
(A.3)
subject to the following constraints:
X (i;j )2u
signi;j (u) pi;j = ? u
bi ? pi;j i;j pi;j 0 bi 0
8u 2 UG
(A.4)
8(i; j ) 2 Ereg 8(i; j ) 2 Esched [ Ereg 8i 2 V
(A.5) (A.6) (A.7)
for the dependence graph G = fV; Esched ; Ereg g. In this formulation, bi and pi;j are integer variables and u and i;j are integer constants that correspond to the constant terms of Equation (4.6) and Inequality (4.12), respectively. Recall that UG is the set of all elementary cycles in the underlying graph of graph fV; Esched [Ereg g. Remember also that signi;j (u) is de ned as +1 if edge (i; j ) is traversed in the forward direction in the cycle u, ?1 if edge (i; j ) is traversed in the backward direction, and 0 otherwise. As formulated, the above problem formulation is not exclusively of the type \Ax b" because of Equation (A.4), which corresponds to the scheduling constraints generated by the underlying cycles. However, we may successively eliminate each of these constraints by isolating one of its pi;j variables per equation and substituting 244
for that variable in the remaining relations of the system. To illustrate the e ect of this substitution, let us consider an elementary cycle, u, in the underlying graph of graph G1 = fV; Esched [ Ereg g. When isolating the variable pi;j associated with edge (i; j ) 2 u in the instance of Equation (A.4) that corresponds to u, we obtain the following equation: 0 1 B X C pi;j = signi;j (u) B B@? CA (A.8) signx;y (u) px;y ? uC (x;y)2u (x;y)6=(i;j )
Since the edges of the underlying cycle u form a close path, the edges (x; y) that satisfy (x; y) 2 u and (x; y) 6= (i; j ) precisely correspond to the edges of the alternate path from vertex i to vertex j along underlying cycle u. Furthermore, the ?signx;y (u) factors correspond to the direction in which the edges (x; y) are traversed in the alternate path from vertices i to j , when signi;j (u) is positive; and when signi;j (u) is negative, we we may simply rede ne the close path u to traverse the same underlying cycle in the reverse direction, thus resulting in signi;j (u) = 1. When the right hand side of Equation (A.8) is used in place of pi;j , we have eliminated all occurrences of pi;j in the system, and replaced them with a linear combination of the px;y that are associated with the edges of the alternate path from vertices i to j along the underlying cycle u. Since there is one instance of Equation (A.4) per underlying cycle, we may successively perform one substitution per underlying cycle. Instead of arbitrarily isolating and removing one of the pi;j variables from each cycle, we compute ahead of time an arbitrary spanning tree, T1, of the graph G1 and only substitute for those pi;j variables that correspond to edges not in T1. This substitution policy is feasible because there is a one to one mapping between the underlying cycles of G1 and the edges of G1 that are not in the spanning tree T1, as explained below by the second property of spanning trees. The rst property of a spanning tree T of graph G is that T de nes a unique path between any two connected vertices. We refer here to the path in the spanning tree T between vertices i and j as spathi;j (T ). The second property of a 245
spanning tree is that each edge (i; j ) that is not in T de nes a unique cycle in G, which consists of edge (i; j ) and the edges in spathi;j (T ). By eliminating each equality in the system in turn, we eventually obtain a system of inequalities of the type \Ax b." By using the spanning tree T1 to decide which pi;j variables to eliminate, the nal system contains only those pi;j variables that correspond to edges in the spanning tree. Moreover, we can show by induction that each of the eliminated pi;j is substituted for in the nal system by a function of the px;y variables associated with the edges (x; y) 2 spathi;j (T1), i.e. the edges in the unique path in the spanning tree from vertices i to j . Similarly, we may also show that the coecient in front of each of these px;y variables is precisely signx;y (spathi;j (T1)). Thus, the nal system of inequalities can be formulated as follows: minimize:
Bu =
X i2V
bi
(A.9)
subject to:
bi ?
X
signx;y (spathi;j (T1)) px;y 0i;j
8(i; j ) 2 Ereg
signx;y (spathi;j (T1)) px;y i;j0
8(i; j ) 2 Esched [ Ereg (A.11)
(x;y)2spathi;j (T1 )
X
(x;y)2spathi;j (T1 )
bi 0
8i 2 V
(A.10)
(A.12)
where bi and pi;j are integer variables, T1 is a spanning tree of the dependence graph G1, and 0i;j and i;j0 are integer constants that are linear combinations of the u and i;j integer constants in Equation (A.4) and Inequality (A.5). Note that Inequalities (A.10), (A.11), and (A.12) corresponds to Inequalities (A.5), (A.6), and (A.7), respectively. Note also that the only pi;j variables that appear in Inequalities (A.10) and (A.11) are the ones associated with edges of the spanning tree T1. Consider the dependence graph in Figure A.1a, which corresponds to Example 4.4 with its dependence graph, MRT, and formulation as shown in Figure 4.6a, Figure 4.6b, and Problem (4.13). The integer linear programming formulation of Ex246
a) Dependence graph
b) Underlying graph and spanning tree
ld0
v0
p02
e7
p05
m2
a5
v2
a1
p23
e1 e1’
v3 p56
p34
e3 e3’
a4 p46
add: mult: div: load: store:
a1,a4,a5 m2, m3 d6 ld0 st7
p67 edges in spanning tree
st7
e5 e5’
v4 e4 e4’
d6
v5
v1
e2 e2’
p14
m3
e8 e0 e0’
p01
v6 e6 e6’ v7
Figure A.1: Dependence graph and its spanning tree (Example 4.4).
Minimize: Subject to:
from (A.4) from (A.4) from (A.5) from (A.5) from (A.5) from (A.6) from (A.6) from (A.7)
b 0 + b1 + b2 + b3 + b4 + b5 + b6 + b7 p0;2 + p2;3 + p3;4 ? p0;1 ? p1;4 = ?2 (A.13) p0;1 + p1;4 + p4;6 ? p0;5 ? p5;6 = 0 (A.14) b0 ? p0;1 1; b0 ? p0;2 0; b0 ? p0;5 0 b1 ? p1;4 0; b2 ? p2;3 0; b3 ? p3;4 0 b4 ? p4;6 0; b5 ? p5;6 0; b6 ? p6;7 0 p0;1 0; p0;2 0; p0;5 0; p1;4 0; p2;3 0 p3;4 0; p4;6 0; p5;6 0; p6;7 0 bi 0 8i 2 V
Figure A.2: Formulation with underlying cycle constraints (Example 4.4).
247
Minimize: Subject to:
from (A.5) and (A.13) from (A.5) and (A.14) from (A.5) from (A.5) from (A.6) and (A.13) from (A.6) and (A.14) from (A.6) from (A.7)
b 0 + b1 + b2 + b3 + b4 + b5 + b6 + b7 b0 ? p0;1 1; b0 ? (p0;1 + p1;4 ? p3;4 ? p2;3) ?2 b0 ? (p0;1 + p1;4 + p4;6 ? p5;6) 0; b1 ? p1;4 0 b2 ? p2;3 0; b3 ? p3;4 0; b4 ? p4;6 0 b5 ? p5;6 0; b6 ? p6;7 0 p0;1 0; (p0;1 + p1;4 ? p3;4 ? p2;3) 2 (p0;1 + p1;4 + p4;6 ? p5;6) 0; p1;4 0; p2;3 0 p3;4 0; p4;6 0; p5;6 0; p6;7 0 bi 0 8i 2 V
Figure A.3: Formulation with variables p0;2 and p0;5 eliminated; remaining pi;j variables correspond to the edges of the spanning tree of Figure A.1b. ample 4.4 is shown here in Figure A.2. It contains two equations, Equations (A.13) and (A.14) that correspond, respectively, to the left and right underlying cycles of Figure A.1. The spanning tree that is used to guide the pi;j variable substitution is shown in Figure A.1b with the bold edges. This spanning tree includes all but the edges (0; 2) and (0; 5), which are associated, respectively, with the left and right underlying cycles of Figure A.1b. When p0;1 + p1;4 ? p3;4 ? p2;3 + 2 is substituted for p0;2, and p0;1 + p1;4 + p4;6 ? p5;6 is substituted for p0;5, we obtain the system of inequalities shown in Figure A.3. Comparing the initial and the nal system of inequalities, we can see that p0;2 has been removed in the nal system by substituting p0;1 + p1;4 ? p3;4 ? p2;3 +2, which precisely corresponds to the path in the spanning tree from vertex 0 to vertex 2, plus a constant term; we can also verify that each px;y variable in this path is multiplied by the correct +1 or ?1 factor, accordingly to the direction shown in Figure A.1a. A similar observation applies to the removal of p0;5. We now introduce two matrix de nitions that are used to characterize a directed graph: 248
De nition A.1 The incidence matrix of a graph G = fV; E g with m vertices and n edges is the m n matrix A where each row corresponds to a vertex and each column corresponds to an edge of G. Matrix A is de ned as:
ai;j
8 > > 1 if ej = (i; x) for some x 2 V nfig > > < = >?1 if ej = (x; i) for some x 2 V nfig > > > : 0 otherwise
where V nfig denote the set of vertices V minus the vertex i, which enforce the elimination of the self-loop edges from the incidence matrix. The out-incidence matrix, A, is constructed similarly, with ai;j = 1 if ai;j = 1, and ai;j = 0 otherwise.
Note that each column of the incidence matrix has precisely two nonzero entries; e.g. the column associated with edge (i; j ) has a +1 value in the row associated with the tail vertex i, a ?1 value in the row associated with the head vertex j , and 0 values elsewhere. Similarly, a column of the out-incidence has precisely one nonzero entry; e.g. the column associated with edge (i; j ) has a +1 value in the row associated with the tail vertex i and 0 values elsewhere.
De nition A.2 The network matrix of a graph G = fV; E g with m vertices and n edges is the (m?1)n matrix A, where each row corresponds to the edges of a spanning tree of graph G and each column corresponds to the edges of G. Let e00 : : :e0m?2 be the edges of the spanning tree T of graph G and let e0 : : : en?1 be the edges of G. In the de nition below, we say that \ej = (x; y) passes e0i " if the path in the spanning tree T between vertices x and y contains the spanning tree edge e0i . Matrix A is de ned as:
ai;j
8 > > 1 if ej passes e0i in the forward direction > > < = >?1 if ej passes e0i in the backward direction > > > : 0 otherwise
In general, the rows and the columns of A are arranged such that the rst m ? 1
249
columns of A corresponds to the edges of the spanning tree, i.e. e00 = e0 : : :e0m?2 = em?2, and thus A = [A1 A2] where A1 = I (the identity matrix).
Note that the column associated with edge (i; j ) of a network matrix for graph G and spanning tree T precisely de nes the signx;y function for spathi;j (T ), the path in the spanning tree T from vertex i to vertex j . Consider the dependence graph in Figure A.1a. The incidence matrix, D, associated with this graph is shown in Figure A.4. The legend for the rows and columns of D corresponds to, respectively, the vertex and edge labels in Figure A.1b. The network matrix, H , associated with the graph in Figure A.1a and the spanning tree shown in Figure A.1b. Note, for example, that column e7 = (0; 2) in H indicates that the path from vertex 0 to vertex 2 contains spanning tree edges e0 and e1 traversed in the forward direction and the spanning tree edges e2 and e3 traversed in the backward direction. We may verify in Figures A.1a and A.1b that this is indeed the case. Using the above de nitions, we may formulate the system of inequalities for the minimum integral MaxLive stage scheduling problem as a function of the outincidence matrices and network matrices. We consider here the dependence graph G = fV; Esched ; Ereg g with m vertices and n scheduling and register edges (n = jEsched [ Ereg j). Let H1 be the (m ? 1) n network matrix of the graph G1 = fV; Esched [ Ereg g with spanning tree T1. Let p be the vector of size (m ? 1) containing the pi;j variables that correspond to spanning tree edges associated with each row of the H1 matrix. Similarly, let 0 be the vector of size n containing the i;j0 integer constants that correspond to edges associated with each column of the H1 matrix. We may formulate Inequality (A.11) as:
p H1 0
(A.15)
since a column of H1 precisely de nes the signx;y function for spathi;j (T1), i.e. the path in the spanning tree T1 from vertex i to vertex j as used in Inequality (A.11). Let D2 be the mn out-incidence matrix of the graph G2 = fV; Ereg g. Let b be the vector of size m containing the bi variables that correspond to the vertices associated 250
2 1 0 0 0 0 0 66 ?1 1 0 0 0 0 66 0 0 1 0 0 0 66 0 0 ?1 1 0 0 D = 66 0 ?1 0 ?1 1 0 66 0 0 0 0 0 1 4 0 0 0 0 ?1 ?1
0 1 1 0 0 0 0 ?1 0 0 0 0 0 0 0 0 0 ?1 1 0 0 0 0 0 0 0 0 ?1 0 0 e0 e1 e2 e3 e4 e5 e6 e7 e8
3 77 77 77 77 77 5
v0 v1 v2 v3 v4 v5 v6 v7
Figure A.4: Incidence matrix for dependence graph of Figure A.1 (Example 4.4). 2 66 66 6 H = 66 64
1 0 0 0 0 0 0 e0
0 1 0 0 0 0 0 e1
0 0 1 0 0 0 0 e2
0 0 0 1 0 0 0 e3
0 0 0 0 1 0 0 e4
0 0 0 0 0 1 0 e5
0 1 1 0 1 1 0 ?1 0 0 ?1 0 0 0 1 0 0 ?1 1 0 0 e6 e7 e8
3 77 77 77 77 5
e0 e1 e2 e3 e4 e5 e6 0 0 0 0 0 0 0
Figure A.5: Network matrix for spanning tree of Figure A.1. e0 e1 e2 e3 e4 e5 e6 e7 e8 e0 e1 e2 e3 e4 e5 e6 e7 e8
2 66 66 66 66 66 66 66 66 66 66 66 66 66 66 66 66 4
1 0 0 0 0 0 0 1 1
0 1 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0
0 0 0 1 0 0 0 0 0
0 0 0 0 1 0 0 0 0
0 0 0 0 0 1 0 0 0
0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 v0
0 0 0 0 0 0 0 0 0 v1
0 0 0 0 0 0 0 0 0 v2
0 0 0 0 0 0 0 0 0 v3
0 0 0 0 0 0 0 0 0 v4
0 0 0 0 0 0 0 0 0 v5
0 0 0 0 0 0 0 0 0 v6
0 0 0 0 0 0 0 0 0 v7
?1 0 0 0 0 ?1 0 0 0 0 ?1 0 0 0 0 ?1 0 0 0 0 0 0 ?1 ?1 ?1 ?1 1 0 0 0 0 0 0 1 1 e0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 ?1 0 0 0 0 ?1 0 0 0 0 ?1 1 0 0 0 0 ?1 1 0
0 0 0 1 0
0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 1 ?1 ?1 1 0 0 e1 e2 e3 0
0
0
0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 1 ?1 e4 e5 0
0
0 0 0 0 0 0 1 0 0 e6
3 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 5
2 b 66 b01 66 b2 66 b3 66 b4 66 b5 66 b6 66 b7 66 66 p0 1 66 p1 4 66 p2 3 66 p3 4 66 p4 6 4p ;
;
;
;
;
p
5;6 6;7
2 1 3 66 0 77 666 00 77 66 0 77 66 0 77 66 0 77 66 ?2 77 66 0 77 66 77 66 0 77 66 0 77 66 0 77 66 0 77 66 0 77 66 0 5 66 0 64 2
0
Figure A.6: System of inequalities corresponding to Figure A.3. 251
0
3 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 5
with each row of the D2 matrix. Similarly, let 0 be the vector of size n containing the 0i;j integer constants that correspond to edges associated with each column of the D2 matrix. Note that each column in D2 associated with edge (i; j ) 62 Ereg consists of 0 entries only; similarly, each element in 0 associated with edge (i; j ) 62 Ereg is set to ?1. Using the above de nitions, we may formulate Inequality (A.10) as:
b D2 ? p H1 0
(A.16)
For each edge in Ereg , Inequality (A.16) results in one of the inequalities generated by Inequality (A.10); otherwise Inequality (A.16) results in an inequality that trivially holds since the 0 value associated with an edge not in Ereg is equal to ?1. There is only a minor problem with Inequality (A.16), because the self-loop register edges that appear in Inequality (A.10) do not appear in Inequality (A.16) since self-loop edges are not present in incidence and network matrices. However, since a self-loop edge (i; i) 2 Ereg only introduce inequality bi 0i;i, we may simply substitute bi ? 0i;i for bi in the system, for edge (i; i) 2 Ereg . We can then safely ignore these selfloop register edges. Note also that this problem does not occur for Inequality (A.15) because self-loops are not considered as an underlying cycle. Inequalities (A.15) and (A.16) can be combined and transposed to result in an integer linear programming formulation of the type \minfcx : Ax b; x 2 Z+n g," namely: 9 8 2 32 3 2 3 > > T ?H T b 0 = < D 2 176 7 6 7 6 2 m ? 1 ; [ b p ] 2 Z min >[1 0] [b p] : 4 5 4 5 4 5 + > ; : 0 0 H1T p (A.17) where the constraint matrix is a 2n (2m ? 1) matrix, the [1 0] and [b p] vectors are of dimension (2m ? 1), and the [ 0 0] vector is of dimension 2n. Note that we may omit from Equation (A.17) the inequalities generated by Inequality (A.12) since the solution vector [m p] is by de nition nonnegative in the integer linear programming formulation. The constraint matrix for Example 4.4 as formulated in FigureA.3 is 252
shown in Figure A.6.
A.2 LP-Relaxation of Stage-Scheduling Problem In this section, we demonstrate that the integer linear programming problem developed in Section A.1 can be solved using an LP-solver. In this demonstration, we use the following matrix property.
De nition A.3 An m n integral matrix A is totally unimodular (TU) if the deter-
minant of each square submatrix of A is equal to 0, 1, or -1.
The TU property is used in the following theorem [69, pp. 536 and 541] to guarantee that the LP-relaxation of an integer linear programming problem results in an integer solutions, if any solution is found:
Theorem A.1 The value zLP = minfcx : Ax b; x 2 Rn+ g produced by an LPsolver integral if m n matrix A is TU, vector b 2 Z m, and vector c 2 Z n, provided that the problem is feasible, i.e. a solution is found.
Since the vectors [ 0 0] and [1 0] of Problem (A.17) have integer values, it is sucient to show that the constraint matrix of Problem (A.17) is TU. Showing that a matrix is TU is NP in general [69, p. 540]; because of the structure of Problem (A.17). However, we show here that its constraint matrix is always TU. We introduce the following de nition and theorems that are used in the demonstration.
De nition A.4 Let A be a matrix with entries equal to 0, 1, and -1. A submatrix
of A, comprised of the set of rows I and the set of columns J , is Eulerian if the sum of its elements in each row, and in each column, is even, i.e.
X j 2J
X i2I
ai;j 0 mod 2
8i 2 I
ai;j 0 mod 2
8j 2 J
253
This de nition is used in the following theorem by Camion [17] to show the TUproperty of a matrix.
Theorem A.2 Let A be a matrix with entries equal to 0, 1, and -1. Matrix A is TU
if and only if for every square Eulerian submatrix, comprised of the set of rows I and the set of columns J , the sum of the elements can be divided by 4, i.e.
X i2I; j 2J
Ai;j 0 mod 4
We demonstrate the TU property of the constraint matrix of Problem (A.17) in two steps. We rst show in Theorem A.3 that a matrix related to the constraint matrix of Problem (A.17) is TU. Then we demonstrate in Theorem A.6 that we may transform the related matrix into the constraint matrix of Problem (A.17) while preserving the TU-property.
Theorem A.3 Consider the dependence graph G = fV; Esched ; Ereg g. The matrix: 3 2 64D2 0 75 D1 ?D1
(A.18)
is TU, where D1 is the incidence matrix of graph G1 = fV; Esched [ Ereg g and D2 is the out-incidence matrix of graph G2 = fV; Ereg g. The incidence matrices are matrices of size m n, where m is the number of vertices and n is the number of edges, i.e. n = jEsched [ Ereg j).
Ning and Gao [70] have shown that the above theorem applies in the case where Esched = Ereg . Our proof for the general case where Esched 6= Ereg is directly based on their proof with the necessary adaptation for giving it the broader applicability that we require here.
Proof. Let A be the matrix de ned by Equation (A.18), consisting of the upper-left, upper-right, lower-left, and lower-right submatrices D2 , 0, D1 , and ?D1, respectively. 254
We demonstrate Theorem A.3 by showing that the sum of elements of an arbitrary Eulerian submatrix of A can be divided by 4. If this result holds, A is TU by Theorem A.2. Consider an arbitrary square Eulerian submatrix of A, referred to as A0, which includes the set of rows I and the set of columns J . The demonstration will account for each of the nonzero elements in A0, and show that their total sum can be divided by 4. First, we assume by hypothesis H1 that the Eulerian submatrix A0 includes at least one +1 element in the upper-left submatrix of A. Consider ai;j = 1, i 2 I , and j 2 J in the upper left matrix where 0 i < m and 0 j < n. Recall that the upper-left submatrix of A corresponds to the out-incidence matrix of graph G2 = fV; Ereg g and thus contains only 0 and +1 elements. Remember also that the upper-right matrix of A is a zero-matrix. Thus, all the elements in row i in the upper matrices of A are either 0 or +1. Since hypothesis H1 states some that A0 includes some row i and column j with a +1 entry in element ai;j , A0 must also include another column with a +1 element in row i in order to be Eulerian. This +1 element can only be found in the upper left matrix; thus there must be a column j 0 such that ai;j = 1, j 0 2 J , and 0 j 0 < n. Furthermore, we know that columns j and j 0 are associated with two register edges, referred to as e and e0, respectively. We also know that edges e and e0 are both outgoing edges from the same vertex, i.e. the vertex associated with row i which we refer to as v. To be Eulerian, the sum of the elements in columns j and j 0 of A0 must also be a multiple of 2. Note that there is only one nonzero entry per column in the upper-left submatrix since it corresponds to the out-incidence matrix of graph G2. Thus, at least one nonzero element in columns j and j 0 must be selected from the lower-left submatrix of A. Furthermore, the lower-left submatrix contains precisely one +1 and one ?1 entry per column since it corresponds to the incidence matrix of the graph G1 = fV; Esched [ Ereg g. Let us further assume by hypothesis H2 that A0 selects the +1 entry in column j from the lower-left submatrix of A, i.e. there is a row i0 such that ai ;j = 1, i0 2 I , and 0
0
255
m i0 < 2m. Since we know by hypothesis H1 that ai;j = 1 (i.e. edge e associated with column j is an outgoing edge from vertex v associated with row i in graph G2) and by hypothesis H2 that ai ;j = 1 (i.e. the same edge e is also an outgoing edge from the vertex associated with row i0 in graph G1), rows i and i0 must be associated with the same vertex v. In turn, this implies that the element at the intersection of the selected row i0 and column j 0 is positive (i.e. ai ;j = 1) because we know that the edge e0 associated with the selected column j 0 is also an outgoing edge from vertex v. Thus, hypotheses H1 and H2 imply four +1 entries, namely ai;j , ai ;j , ai;j , and ai ;j . Since these four entries add 4 to the total sum of the elements in A0, the contribution of these element is clearly a multiple of 4. If, instead of hypothesis H2, we assume by hypothesis H3 that A0 selects the ?1 entry in column j from the lower-left submatrix, i.e. there is a row i00 such that i00 6= i0, ai ;j = ?1, i00 2 I , and m i00 < 2m. Note that since hypotheses H1, H2, and H3 cannot all be true1, assuming H1 and H3 implies that H2 is false. Because a +1 and a ?1 entry are the only selected elements in column j , the total contribution of the entries ai;j and ai ;j is 0, which clearly adds a contribution to the total sum of the elements in A0 that is a multiple 4. Similarly, we know by hypothesis that H2 is false, i.e. i0 62 I . As a result, we know that A0 must also select its ?1 entry in column j 0 from the lower-left submatrix, say in row i000. Thus, the total contribution of the entries ai;j and ai ;j also adds up to 0, again a contribution that is a multiple of 4. We have now proved that any nonzero entries that can be selected from the upperleft matrix result in a total contribution that is a multiple of 4, since we have analyzed the only two possible (and disjoint) cases with hypothesis H1, i.e. \H1 and H2 and not H3" and \H1 and not H2 and H3." Let us now discard hypotheses H1, H2, and H3 and assume by hypothesis H4 that the Eulerian submatrix A0 selects an arbitrary nonzero element that is not yet accounted for from the lower submatrices of A, i.e. a new element in row i and column j such that ai;j 6= 0, i 2 I , j 2 J , m i < 2m, 0
0
0
0
0
0
0
00
00
0
000
0
1 If we assume H1, H2, and H3 to be true, they contribute, respectively, the +1, +1, and ?1 entries in column j , which does not sum up to a multiple of 2. Since these 3 entries are the only nonzero entries in column j , A cannot be Eulerian in column j . Thus assuming H1, H2, and H3 results in a contradiction. 0
256
and 0 j < 2n. Since we cannot select another nonzero entry from the upper-right submatrix (case already covered in hypotheses H1, H2, and H3) or from the upperleft matrix (which is a zero submatrix), another nonzero element in column j must be selected from the lower submatrices of A if A0 is to be Eulerian. Because the lower matrices of A are incidence matrices, there are only 2 nonzero entries per column, one with the value +1 and one with the value ?1. Thus, we must select the other nonzero entry, i.e. there is a row i0 such that ai ;j = ?ai;j , i0 2 I , and m i0 < 2m. As a result, the contribution of the two entries ai;j and ai ;j also sums up to 0. To conclude, hypotheses H1 to H4 account for each nonzero entry in some arbitrary Eulerian submatrix A0. We have shown that the contribution of these nonzero entries can always be grouped such that the sum for each of these disjoint groups of nonzero entries adds up to either 0 or 4. Thus, the total sum of the entries in A0 can be divided by 4, and A is TU by Theorem A.2. 2 0
0
In the second step, we show that we may transform Matrix (A.18) into the constraint matrix of Problem (A.17) by using transformation steps that preserve the TU property of Matrix (A.18). The TU-preserving transformations that we used are found in [69, pp 540] and described here.
Theorem A.4 The following statements are equivalent: (1) A is TU; (2) the trans-
pose of A is TU; (3) a matrix obtained by deleting a row or a column, or interchanging a row or a column, of A is TU; (4) a matrix obtained by multiplying a row or a column, of A by -1 is TU; (5) a matrix obtained by a pivot operation on A is TU.
We also use a theorem that indicates how network matrices relate to incidence matrices [69, pp. 546-547].
Theorem A.5 A network matrix is precisely a matrix whose columns represent the
incidence matrix of a graph after one row has been deleted and a number of pivots have been executed until the resulting matrix is of the form [I A].
Using Theorems A.4 and A.5, we may prove the following theorem. 257
Theorem A.6 The constraint matrix of Problem (A.17) is TU. Proof. We already know from Theorem A.3 that Matrix (A.18) is TU. In this proof,
we simply show how this matrix can be transformed in the constraint matrix of Problem (A.17) using exclusively transforms that preserve the TU property of the matrices. The rst step is to transform the incidence matrix of graph G1 into the network matrix of graph G1, i.e. D1 into H1. Using Theorem A.5, we know that D1 can be transformed into H1 by only using TU-preserving transformation steps, as de ned in Theorem A.4. This implies, in turn, that the matrix [D1 ? D1] can also be transformed into the matrix [H1 ? H1] by using exactly the same TU-preserving transformation steps. As a result, the TU Matrix (A.18) can be transformed in the following matrix: 3 2 64D2 0 75 (A.19) H1 ?H1 using only TU-preserving transforms. Multiplying the rows of the submatrix [H1 ? H1] from Matrix (A.19) by -1 and taking its transpose results in the following matrix: 2 3 T ?H T D 17 64 2 (A.20) 5 0 H1T which is TU since both operations preserves the TU-property. Matrix (A.20) corresponds to the constraint matrix of Problem (A.17), and thus Theorem A.6 is proved.
2
As a result, Theorem A.1 now applies, and the solution of Problem (A.17) is therefore integral whenever an LP-solver nds an optimal solution.
258
REFERENCES [1] A. V. Aho, J. E. Hopcroft, and J. D. Ullman. The Design and Analysis of Computer Algorithms. Addison-Wesley, Reading, MA, 1974. [2] A. V. Aho, R. Sethi, and J. D. Ullman. Compilers: Principles, Techniques and Tools. Addison-Wesley, Reading, MA, 1988. [3] A. Aiken, A. Nicolau, and S. Novak. Resource-constrained software pipelining. IEEE Transactions on Parallel and Distributed Systems, 6(12):1248{1269, December 1995. [4] J. R. Allen, K. Kennedy, C. Porter eld, and J. Warren. Conversion of control dependence to data dependence. In Conference Record of the 10th Annual ACM Symposium on Principles of Programming Languages, pages 177{189, January 1983. [5] V. H. Allen, U. R. Shah, and K. M. Reddy. Petri net versus modulo scheduling for software pipelining. In Proceedings of the 28th Annual International Symposium on Microarchitecture, pages 105{110, November 1995. [6] E. R. Altman. Optimal Software Pipelining with Function Unit and Register Constraints. PhD thesis, Department of Electrical Engineering, McGill University, Montreal, Canada, October 1995. [7] E. R. Altman, R. Govindarajan, and G. R. Gao. Scheduling and mapping: Software pipelining in the presence of structural hazards. In Proceedings of the ACM SIGPLAN'95 Conference on Programming Language Design and Implementation, pages 139{150, 1995. [8] V. Bala. Personal communication. February 1996. [9] V. Bala and N. Rubin. Ecient instruction scheduling using nite state automata. Proceedings of the 28th Annual International Symposium on Microarchitecture, pages 46{56, November 1995. [10] R. A. Ballance, A. B. Maccabe, and K. J. Ottenstein. The program dependence web: A representation supporting control-, data-, and demand-driven interpretation of imperative languages. Proceedings of the ACM SIGPLAN'90 Conference on Programming Language Design and Implementation, pages 257{271, 1990. 259
[11] G. R. Beck, D. W. L. Yen, and T. L. Anderson. The Cydra 5 mini-supercomputer: Architecture and implementation. In The Journal of Supercomputing, volume 7, pages 143{180, 1993. [12] D. Bernstein and M. Rodeh. Global instruction scheduling for superscalar machines. In Proceedings of the ACM SIGPLAN'91 Conference on Programming Language Design and Implementation, pages 241{255, June 1991. [13] M. Berry et al. The Perfect Club Benchmarks: E ective performance evaluation of supercomputers. The International Journal of Supercomputer Applications, 3(3):5{40, Fall 1989. [14] D. A. Berson, R. Gupta, and M. L. So a. URSA: A uni ed resource allocator for registers and functional units in VLIW architectures. In IFIP Working Conference on Architecture and Compilation Techniques for Fine and Medium Grain Parallelism, pages 241{255, January 1993. [15] Preston Briggs, Keith D. Cooper, Ken Kennedy, and L. Torczon. Coloring heuristics for register allocation. Proceedings of the ACM SIGPLAN'89 Conference on Programming Language Design and Implementation, 24(7):275{284, June 1989. [16] R. A. Brualdi. Introductory combinatorics. New York: North-Holland, 1992. [17] P. Camion. Characterization of totally unimodular matrices. In Proc. Amer. Math. Soc., volume 16, pages 1068{1073, 1965. [18] G. J. Chaitin. Register allocation and spilling via graph coloring. Proceedings of the ACM SIGPLAN'82 Symposium on Compiler Construction, pages 98{105, June 1982. [19] P. P. Chang, S. A. Mahlke, W. Y. Chen, N. J. Warter, and W. W. Hwu. IMPACT: An architectural framework for multiple-instruction-issue processors. In Proceedings of the Eighteenth Annual International Symposium on Computer Architecture, pages 266{275, May 1991. [20] P. P. Chang, N. J. Warter, S. A. Mahlke, W. Y. Chen, and W. W. Hwu. Three architectural models for compiler-controlled speculative execution. IEEE Transactions on Computers, 44(4):481{494, April 1995. [21] L. Chao, A. LaPaugh, and E. H. Sha. Rotation scheduling: A loop pipelining algorithm. 30th Design Automaton Conference, pages 556{572, 1993. [22] A. E. Charlesworth. An approach to scienti c array processing: The architectural design of the AP-120B/FPS-164 family. Computer, 14(9):18{27, September 1981. [23] S. Chaudhuri, R. A. Walker, and J. E. Mitchell. Analyzing and exploiting the structure of the constraints in the ILP approach to the scheduling problem. IEEE Transactions on Very Large Scale Integration Systems, 2(4):456{471, December 1994. 260
[24] R. G. Cytron. Compile-Time Scheduling and Optimization for Asynchronous Machines. PhD thesis, Department of Electrical and Computer Engineering, University of Illinois, Urbana, IL, 1984. [25] E. S. Davidson, L. E. Shar, A. T. Thomas, and J. H. Patel. E ective control for pipelined computers. Spring COMPCON-75 digest of papers, pages 181{184, February 1975. [26] J. C. Dehnert, P. Y.-T. Hsu, and J. P. Bratt. Overlapped loop support in the Cydra 5. 3rd International Conference on Architectural Support for Programming Languages and Operating Systems, pages 181{227, May 1989. [27] J. C. Dehnert and R. A. Towle. Compiling for the Cydra 5. In The Journal of Supercomputing, volume 7, pages 181{227, 1993. [28] Digital Equipment Corp., Maynard, MA. DecChip 21064 Microprocessor Hardware Reference Manual EC-N0079-72. [29] B. Dupont de Dinechin. Simplex scheduling: More than lifetime-sensitive instruction scheduling. Proceedings of the International Conference on Parallel Architecture and Compiler Techniques, pages 327{330, 1994. [30] K. Ebcioglu, R. D. Groves, K.-C. Kim, G. M. Silberman, and I. Ziv. VLIW compilation techniques in a superscalar environment. In Proceedings of the ACM SIGPLAN'94 Conference on Programming Language Design and Implementation, pages 36{48. 1994. [31] A. E. Eichenberger and E. S. Davidson. A reduced multipipeline machine description that preserves scheduling constraints. Technical Report CSE-TR-266-95, University of Michigan, Ann Arbor, MI, 1995. [32] A. E. Eichenberger and E. S. Davidson. Register allocation for predicated code. Proceedings of the 28th Annual International Symposium on Microarchitecture, pages 338{349, November 1995. [33] A. E. Eichenberger and E. S. Davidson. Stage scheduling: A technique to reduce the register requirements of a modulo schedule. Proceedings of the 28th Annual International Symposium on Microarchitecture, pages 180{191, November 1995. [34] A. E. Eichenberger and E. S. Davidson. Ecient formulation for optimal modulo schedulers. Proceedings of the ACM SIGPLAN'97 Conference on Programming Language Design and Implementation, May 1996. [35] A. E. Eichenberger, E. S. Davidson, and S. G. Abraham. Minimum register requirements for a modulo schedule. Proceedings of the 27th Annual International Symposium on Microarchitecture, pages 75{84, November 1994.
261
[36] A. E. Eichenberger, E. S. Davidson, and S. G. Abraham. Optimum modulo schedules for minimum register requirements. Proceedings of the International Conference on Supercomputing, pages 31{40, July 1995. [37] A. E. Eichenberger, E. S. Davidson, and S. G. Abraham. Minimizing register requirements of a modulo schedule via optimum stage scheduling. In International Journal of Parallel Programming, volume 24, pages 103{132, 1996. [38] C. Eisenbeis, W. Jalby, and A. Lichnewsky. Squeezing more performance out of a Cray-2 by vector block scheduling. Proceedings of Supercomputing '88, pages 237{246, November 1988. [39] C. Eisenbeis, S. Lelait, and B. Marmol. The meeting graph: A new model for loop cyclic register allocation. Proceedings of the International Conference on Parallel Architecture and Compiler Techniques, June 1995. [40] C. Eisenbeis and D. Windheiser. Optimal software pipelining in presence of resource constraints. Proceedings of the International Conference on Parallel Architecture and Compiler Techniques, August 1993. [41] J. R. Ellis. Bulldog: a Compiler for VLIW Architectures. The MIT Press, 1985. [42] J. A. Fisher. Trace scheduling: a technique for global microcode compaction. IEEE Transactions on Computers, 30(7):478{490, July 1981. [43] J. R. Goodman and W.-C. Hsu. Code scheduling and register allocation in large basic blocks. Proceedings of the International Conference on Supercomputing, pages 442{452, 1988. [44] R. Govindarajan, E. R. Altman, and G. R. Gao. Minimizing register requirements under resource-constrained rate-optimal software pipelining. Proceedings of the 27th Annual International Symposium on Microarchitecture, pages 85{94, November 1994. [45] J. C. Gyllenhaal. A machine description language for compilation. Master's thesis, Department of Electrical and Computer Engineering, University of Illinois, Urbana, IL, 1994. [46] L. J. Hendren, G. R. Gao, E. R. Altman, and C. Mukerji. A register allocation framework based on hierarchical cyclic interval graphs. Proceedings of the 4th International Conference on Compiler Construction. Lecture Notes in Computer Science 641, Springer, pages 176{191, 1992. [47] F. S. Hillier and G. J. Lieberman. Introduction to Mathematical Programming. McGraw-Hill, 1990. [48] P. Y. Hsu. Highly Concurrent Scalar Processing. PhD thesis, Department of Electrical and Computer Engineering, University of Illinois, Urbana, IL, 1986. 262
[49] R. A. Hu . Lifetime-sensitive modulo scheduling. Proceedings of the ACM SIGPLAN'93 Conference on Programming Language Design and Implementation, pages 258{267, June 1993. [50] W. W. Hwu, S. A. Mahlke, W. Y. Chen, P. P. Chang, J. Warter N, R. A. Bringmann, R. G. Ouellette, R. E. Hank, T. Kiyohara, G. E. Haab, J. G. Holm, and D. M. Lavery. The superblock: An e ective technique for VLIW and superscalar compilation. In The Journal of Supercomputing, volume 7, pages 229{248, 1993. [51] R. C. Johnson. Personal communication. June 1995. [52] K. Ebcioglu and T. Nakatani. A New Compilation Technique for Parallelizing Loops with Unpredictable Branches on a VLIW Architecture. In D. Gelernter, A. Nicolau, and D. Padua, editors, Languages and Compilers for Parallel Computing, pages 213{229. Pitman/The MIT Press, 1990. [53] G. Kane and J. Heinrich. MIPS RISC Architecture. Prentice Hall, 1992. [54] V. Kathail, M. S. Schlansker, and B. R. Rau. HPL PlayDoh architecture speci cation: Version 1.0. Technical Report HPL-93-80, HP Laboratories, February 1994. [55] M. Lam. Software Pipelining: An e ective scheduling technique for VLIW machines. Proceedings of the ACM SIGPLAN'88 Conference on Programming Language Design and Implementation, pages 318{328, June 1988. [56] D. M. Lavery and W. W. Hwu. Unrolling-based optimizations for modulo scheduling. Proceedings of the 28th Annual International Symposium on Microarchitecture, pages 327{337, November 1995. [57] J. Llosa, A. Gonzalez, E. Ayguade, and M. Valero. Swing modulo scheduling: A lifetime-sensitive approach. Proceedings of the International Conference on Parallel Architecture and Compiler Techniques, October 1996. [58] J. Llosa, M. Valero, E. Ayguade, and A. Gonzalez. Hypernode reduction modulo scheduling. Proceedings of the 28th Annual International Symposium on Microarchitecture, pages 350{360, November 1995. [59] G. P. Lowney, S. M. Freudenberger, T. J. Karzes, W. D. Lichtenstein, R. P. Nix, J. S. O'Donnell, and J. C. Ruttenberg. The Multi ow trace scheduling compiler. In The Journal of Supercomputing, volume 7, pages 51{142, 1993. [60] S. A. Mahlke. Exploiting Instruction Level Parallelism in the Presence of Conditional Branches. PhD thesis, Department of Electrical and Computer Engineering, University of Illinois, Urbana, IL, 1996. [61] S. A. Mahlke, W. Y. Chen, W.-M. W. Hwu, B. R. Rau, and M. S. Schlansker. Sentinel scheduling for VLIW and superscalar processors. In 5th International Conference on Architectural Support for Programming Languages and Operating Systems, pages 238{247, October 1992. 263
[62] S. A. Mahlke, D. C. Lin, W. Y. Chen, R. E. Hank, and R. A. Bringmann. E ective compiler support for predicated execution using hyperblocks. Proceedings of the 25th Annual International Symposium on Microarchitecture, pages 45{54, December 1992. [63] W. Mangione-Smith. Performance Bounds and Bu er Space Requirements. PhD thesis, University of Michigan, Department of Electrical Engineering and Computer Science, Ann Arbor, MI, 1992. [64] W. Mangione-Smith, S. G. Abraham, and E. S. Davidson. Register requirements of pipelined processors. Proceedings of the International Conference on Supercomputing, pages 260{271, July 1992. [65] F. H. McMahon. The Livermore Fortran Kernels: A computer test of the numerical performance range. Technical Report UCRL-53745, Lawrence Livermore National Laboratory, Livermore, California, 1986. [66] W. M. Meleis and E. S. Davidson. Optimal local allocation for a multiple-issue machine. Proceedings of the International Conference on Supercomputing, pages 107{116, 1994. [67] S.-M. Moon and K. Ebcioglu. An ecient resource-constrained global scheduling technique for superscalar and VLIW processors. Proceedings of the 25th Annual International Symposium on Microarchitecture, pages 55{71, September 1992. [68] T. Muller. Employing nite automata for resource scheduling. Proceedings of the 26th Annual International Symposium on Microarchitecture, pages 12{20, 1993. [69] G. L. Nemhauser and L. A. Wolsey. Integer and Combinatorial Optimization. Wiley, New York, 1988. [70] Q. Ning and G. R. Gao. A novel framework of register allocation for software pipelining. Twentieth Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pages 29{42, 1993. [71] S. Olariu. Optimal greedy heuristic to color interval graphs. Information Processing letters, 37(1):21{25, 1991. [72] J. C. H. Park and M. S. Schlansker. On predicated execution. Technical Report HPL-91-58, HP Laboratories, May 1991. [73] J. H. Patel and E. S. Davidson. Improving the throughput of a pipeline by insertion of delays. Proceedings of the Third Annual International Symposium on Computer Architecture, pages 159{164, 1976. [74] K. Paton. An algorithm for nding a fundamental set of cycles of a graph. Communications of the ACM, 12(9):514{518, September 1969. 264
[75] S. S. Pinter. Register allocation with instruction scheduling: a new approach. Proceedings of the ACM SIGPLAN'93 Conference on Programming Language Design and Implementation, pages 248{257, June 1993. [76] T. A. Proebsting and C. W. Fraser. Detecting pipeline structural hazards quickly. Twenty-First Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pages 280{286, January 1994. [77] B. R. Rau. Iterative Modulo Scheduling: An algorithm for software pipelining loops. Proceedings of the 27th Annual International Symposium on Microarchitecture, pages 63{74, November 1994. [78] B. R. Rau. Iterative Modulo Scheduling. International Journal of Parallel Programming, 24(1):2{64, 1996. [79] B. R. Rau and J. A. Fisher. Instruction-level parallel processing: History, overview, and perspective. In The Journal of Supercomputing, volume 7, pages 9{50, 1993. [80] B. R. Rau and C. D. Glaeser. Some scheduling techniques and an easily schedulable horizontal architecture for high performance scienti c computing. Fourteenth Annual Workshop on Microprogramming, pages 183{198, October 1981. [81] B. R. Rau, C. D. Glaeser, and R. L. Picard. Ecient code generation for horizontal architectures: Compiler techniques and architecture support. Proceedings of the Ninth Annual International Symposium on Computer Architecture, pages 131{139, 1982. [82] B. R. Rau, M. Lee, P. P. Tirumalai, and M. S. Schlansker. Register allocation for software pipelined loops. Proceedings of the ACM SIGPLAN'92 Conference on Programming Language Design and Implementation, pages 283{299, June 1992. [83] J. R. Ruttenberg, G. R. Gao, and A. Stoutchinin. Software pipelining showdown: Optimal vs. heuristic methods in a production compiler. Proceedings of the ACM SIGPLAN'96 Conference on Programming Language Design and Implementation, pages 1{11, May 1996. [84] M. S. Schlansker. Personal communication. June 1995. [85] M. S. Schlansker and V. Kathail. Acceleration of rst and higher order recurrences on processors with instruction level parallelism. Proceedings of the Languages and Compilers for Parallel Computing, 6th International Workshop, August 1993. [86] M. S. Schlansker and V. Kathail. Critical path reduction for scalar code. Proceedings of the 28th Annual International Symposium on Microarchitecture, pages 57{69, November 1995. 265
[87] M. S. Schlansker, V. Kathail, and S. Anik. Height reduction of control recurrences for ILP processors. Proceedings of the 27th Annual International Symposium on Microarchitecture, pages 40{51, November 1994. [88] E. Su, A. Lain, S. Ramaswamy, D. J. Palermo, E. W. Hodges IV, and P. Banerjee. Advanced compilation techniques in the PARADIGM compiler for distributedmemory multicomputers. Proceedings of the International Conference on Supercomputing, pages 424{433, July 1995. [89] P. P. Tirumalai, M. Lee, and M. S. Schlansker. Parallelization of loops with exits on pipelined architectures. Proceedings of Supercomputing '90, pages 200{212, November 1990. [90] P. Tu and D Padua. Gated SSA-based demand-driven symbolic analysis for parallelizing compilers. Proceedings of the International Conference on Supercomputing, pages 414{423, July 1995. [91] J. Uniejewski. SPEC Benchmark Suite: Designed for today's advanced system. SPEC Newsletter, Fall 1989. [92] J. Wang, C. Eisenbeis, M. Jourdan, and B. Su. Decomposed software pipelining: A new perspective and a new approach. In International Journal of Parallel Programming, volume 22, pages 357{379, 1994. [93] J. Wang, A. Krall, and M.A. Ertl. Decomposed software pipelining with reduced register requirement. In Proceedings of the International Conference on Parallel Architecture and Compiler Techniques, June 1995. [94] N. J. Warter. Modulo Scheduling with Isomorphic Control Transformations. PhD thesis, Department of Electrical and Computer Engineering, University of Illinois, Urbana, IL, 1994. [95] N. J. Warter, G. E. Haab, K. Subramanian, and J. W. Bockhaus. Enhanced Modulo Scheduling for loops with conditional branches. Proceedings of the 25th Annual International Symposium on Microarchitecture, pages 170{179, December 1992. [96] N. J. Warter-Perez and N Partamian. Modulo scheduling with multiple initiation intervals. Proceedings of the 28th Annual International Symposium on Microarchitecture, pages 111{118, November 1995.
266
