Apr 27, 1998 - In Section 2, the connection is made between the pragmatic goal of increasing ..... optimization, and not under explicit programmer control.
Interval Algebra, Composite Functions and Dependence in Compilers G. William Walster Eldon R. Hansen April 27, 1998
Abstract
Algebra of interval variables, together with composites of interval functions and relations are used to justify transformations that a compiler may automatically apply to interval expressions. A generalization of the fundamental theorem of interval arithmetic, as well as the value-assignment and expression-assignment operators are used to rationalize interval expression folding.
1 Overview In Section 2, the connection is made between the pragmatic goal of increasing interval sharpness and the conception of interval algebra and mathematics. Section 3 contains a short review of interval arithmetic. Section 4 introduces the distinction between independent and dependent intervals. Section 4.1 de nes interval arithmetic on dependent intervals. Section 4.2 introduces enclosures of independent and dependent functions. Section 4.3 introduces enclosures of relations, which enables interval expressions containing interval constants and variables to be algebraically manipulated. Section 5 is a discussion of the need for composite functions to justify expression folding of point expressions. In Section 6 a generalization of the fundamental theorem of interval arithmetic is proved to justify interval expression folding. Section 6.1 shows how expression folding interacts with dependent interval variables and independent interval constants. Section 6.2 documents how the valueassignment and expression-assignment operators are used with Fortran PARAMETER statements to de ne read-only variables and named constants, respectively. 1
2 Introduction The objective of performing mathematical transformations on interval expressions at compile-time is to increase sharpness and/or execution speed. Justifying these transformations leads to the conclusion that intervals are fundamental mathematical objects, with their own unique properties, and the same (or greater) stature as integer, real, and complex variables. Transformations of interval variable (as contrasted with constant) expressions are shown to follow all the rules of algebra, including the distributive law. Interval expression folding1 at compile-time expands the set of expressions to which algebraic transformations can be automatically applied. Interval expression folding depends on the existence of valid interval enclosures of composite functions and relations. Composites of interval enclosures are proved to contain interval enclosures of corresponding composite functions or relations.
3 Interval Arithmetic Let [a; b] and [c; d] denote interval constants2 and let X := [a; b] and Y := [c; d] denote the assignment of the value of interval constants to the interval variables X and Y . Beginning in Section 4, the di erent properties of interval constants and variables are made explicit. Let \op" denote a member of the set of arithmetic operations f+; ?; ; g. The settheoretic de nition of interval arithmetic operations (e.g., see [3]) requires that arithmetic operations, X op Y , contain the sets:
fx op y j x 2 X , y 2 Y g . The equivalent requirement for functions of two variables, is that f (X; Y ) contain the set ff (x, y) j x 2 X , y 2 Y g. In particular, f (X; Y ) can be X op Y and f (x; y ) can be x op y . Interval arithmetic operations and function evaluations must be interval enclosures. Expression folding is the algebraic concatenation of two or more expressions. For example, the Fortran expressions: X=A followed by X=X+B can be folded into the mathematically equivalent expression, X=A+B. 2 For simplicity of exposition, all interval constants are assumed to be non-degenerate unless explicitly denoted, for example, by [a; a]. 1
2
De nition 1 An interval function, f (X ), is an interval enclosure of the function, f (x), if
f (X ) ff (x) j x 2 X g .
The de nition of an interval enclosure carries over to functions of n variables.
4 Independent and Dependent Intervals Two mathematically equivalent conceptions of an interval variable exist: in the rst, a single unknown point is contained in an interval; and in the second, a set of single points is contained in an interval. In either conception, any particular single point is identical to the same point in every occurrence of the same interval variable. In every occurrence of the same interval variable, individual points contained therein are dependent. In [4], to make explicit the fact that interval variables are dependent, an interval variable is de ned to be a symbolic scalar variable and its range, which is operationally de ned by interval endpoints. The symbolic name of an interval variable is required to establish dependence between multiple occurrences of the same interval variable. In contrast, an interval constant is only the set of all possible numerical values in an interval. An interval constant can be conceptually and operationally de ned by its endpoints. To use the fact that the interval variables, X and X are dependent, but X and Y are not, the symbolic name of the interval variable is required. By de nition, there is no dependence between multiple occurrences of the same interval constant, because no symbolic name or underlying point variable even exists with which to conceptualize dependence. Point and interval constants and variables obey the commutative and associative laws of algebra for addition and multiplication. Point constants, point variables, and interval variables, also obey the distributive law of algebra and have additive and multiplicative inverses. However, each occurrence of an interval constant is equivalent to a new interval variable, independent of all other occurrences of the same interval constant. In the following two examples, let [a; b] be a constant interval and let X := Y := [a; b]. Example 1 Computing [a; b] ? [a; b] is the same as computing X ? Y fx ? y j x 2 X; y 2 Y g = [a ? b; b ? a]. Example 2 Computing [a; b]=[a; b] is the same as computing X=Y fx=y j x 2 X; y 2 Y g = [min(a=b; b=a); max(a=b; b=a)] given a > 0 or b < 0. 3
Two non-degenerate interval constants cannot be completely dependent or identically equal.
4.1 Interval Arithmetic on Dependent Intervals
The set-theoretic de nition of interval arithmetic operations on the same variable require that arithmetic operations, X op X , contain the sets:
fx op x j x 2 X g . When evaluated using only one variable, the equivalent requirement for the evaluation of functions of two variables is: f (X; X ) must contain the set
ff (x, x) j x 2 X g, where in the case of arithmetic, f (X; X ) X op X and f (x; x) x op x.
As a consequence of the fact that an interval variable is completely dependent on itself and identically equal to itself, interval variables have additive and multiplicative inverses: X ? X fx ? x j x 2 X g = [0; 0] and X=X fx=x j x 2 X g = [1; 1]. X op X is di erent from X op Y or [a; b] op [a; b] for op 2 f+; ?; ; =g, because X depends on itself. In this case, the following rules apply for X := [a; b]: Expression X+X
X?X X X = X2 X=X
Result
[2a; 2b] [0 ; 0] ( min a2; b2 ; max a2; b2 if a > 0 or b < 0 0; max a2 ; b2 ; otherwise [1; 1]
)
As noted above, analogous simpli cations do not exist for [a; b] op [a; b], where [a; b] is a constant interval. These results must be obtained by substituting a for c and b for d in the normal interval arithmetic de nitions for two independent intervals, [a; b] and [c; d]. Interval constants are independent, even though their corresponding endpoints are equal. For example, while 2 ? 2 = 0; [2; 3] ? [2; 3] = [?1; 1]. These results expose the fundamental difference between interval arithmetic on constants and interval mathematics on variables. The analogous distinction is made for points, at least implicitly, but only in the case of division. The rules of arithmetic and algebra of points 4
are typically quali ed with a statement such as: \except for division by zero". While 0=0 is not de ned for points, when both f and g approach zero at x0 , say, f (x0)=g (x0) may be resolvable using additional analysis, such as L'Hopital's rule, if the numerator and denominator are functions of the same variable and, therefore, dependent. Canceling x=x or f (x)=f (x) and substituting 1 is done automatically, either in complete expressions or when the cancellation is part of a larger expression. It may be that point variable cancellation is performed without question because the di erence between point constants and point variables is implicitly recognized. For intervals, the distinction between interval variables and constants is critical to distinguishing between interval arithmetic and interval algebra.
4.2 Enclosures of Dependent Functions
The distinction between independent and dependent intervals is made by considering their de ning point functions or relations. Two point functions (or relations) f and g are dependent if they share at least one common variable argument. For example, f (x; y ) and g (u; v ) are independent, while f (x; y ) and g (u; x) are dependent. Two interval enclosures f and g are dependent if they share at least one common variable argument. For example, f (X; Y ) and g (U; V ) are independent, while f (X; Y ) and g (U; X ) are dependent. Neither point nor interval constants are relevant, because f (2; y) and g(2; v) are independent. Similarly, f ([a; b]; Y ) and g([a; b]; V ) are independent.
4.3 Enclosures of Relations
The di erence between interval variables and constants can be elaborated by distinguishing between interval enclosures of mathematical functions and relations. Let f (X ) and g (X ) denote the interval enclosures of the real function and relation f (x) and g (x); respectively. Examples of simple functions are: f (x) = a or f (x) = x. Examples of simple relations are: g (x) = [a; b] or g (x) = [a; b]x. Functions are single-valued mappings from points in the function's domain to points in its range. Relations include mappings from points to sets. Both functions and relations can have interval enclosures. For example, f (X ) ff (x) j x 2 X g = a and f (X ) = X are interval enclosures of the functions f (x) = a and f (x) = x, respectively. g(X ) fg(x) j x 2 X g = [a; b], and g(X ) = [a; b]X are interval enclosures of the relations g (x) = [a; b] and g (x) = [a; b]x, respectively. Only 5
interval enclosures of functions can be completely dependent or identically equal. Interval enclosures of relations can be partially, but never completely dependent. Pure interval constants, as well as functions or relations having only interval constant arguments, are always completely independent.
5 Point Composite Functions and Expressionfolding In point mathematics, composite functions are de ned as follows:
De nition 2 Given the functions f (u) and g(x), for x in the domain of g
and the range of g in the domain of f , then the composite function h(x) = f (g(x)) is de ned.
De nition 3 Given h(u; v) in addition to f (x; y) and g (x; y), with x and y in the domain of f and g, and the ranges of f and g in the domain of h, then the composite function q (x; y ) = h(f (x; y ); g (x; y )) is de ned. The de nition of composite functions extends to m functions of n variables.
The Fortran standard [5] permits a compiler to substitute \mathematically equivalent" expressions within a given statement. The standard also de nes the \=" operator to be only value assignment. The \=" operator is neither mathematical equality, nor the de nition of an in-line function, nor expression assignment. Value assignment precludes expression folding. Nevertheless, in the interest of run-time performance, optimizing compilers perform expression folding followed by simpli cation such as: Original Code: Folded Code:
X=A+B Y=X-A
Y=B
Composite functions can be used to prove the above codes are mathematically equivalent: y = f (g(a; b); a); where g(a; b) = a + b and f (x; a) = x ? a, from which it follows that y = a + b ? a = b. Mathematical equivalence notwithstanding, when evaluated using
oating-point arithmetic, rounding errors and catastrophic cancellation can 6
cause the original and folded codes to produce completely di erent results. As the magnitude of A becomes larger than that of B, the results obtained from the folded and unfolded codes will diverge. One way to permit a compiler to perform expression-folding is to let the \=" operator be interpreted either as \value assignment" or \expression assignment". Expression assignment requires mathematical identity. In the above example, the folded code is more accurate and faster. The decision to fold may depend on data that are not known at compile time. Accuracy and speed may sometimes con ict, or in some situations both value and expression assignment may be required. Floating-point compilers perform expression folding only for speed, usually under high levels of optimization, and not under explicit programmer control. To illustrate the requirement both for expression folding and for value assignment to preclude expression folding, consider the case in which y = P P = x for various values =1 x . Later in the algorithm, computing y = of j may be required. If the magnitude of x is much greater than that of y , the value of d = y ? x will be inaccurate if computed using oating-point arithmetic. Provided the magnitude of x and y are di erent, computing d = y ? x will be just as accurate and much faster than computing y . Therefore, the requirement exists both to permit and to prohibit expressionfolding. To control when a compiler may perform expression-folding, two operators are required: \" meaning mathematical identity, as in the de nition of an in-line function, or \expression assignment"; and, \:=" meaning \value assignment". Given current practice and the relative frequency with which \" and \:=" will be used, the existing \=" operator is interpreted to permit \expression assignment" and the new \:=" operator is introduced to force \value assignment". Backward compatibility with the existing standard is achieved by prohibiting expression folding with a pragma and/or a command line option that force both \=" and \:=" to mean \value assignment". n i
i
j
i 6
j
i
j
j
j
j
j
j
j
j
6 Interval Composites and Expression-folding Expression folding is at least, if not more important for intervals than for points. Folding interval expressions creates opportunities for algebraic transformation of interval expressions, with the goal of increasing sharpness or speed, or both. To justify interval expression-folding, de ne the interval enclosure of a composite function, the composite of interval enclosures, and 7
their relation to each other: Theorem 1 Given real functions, f (x), g(x), the composite function h(x) = f (g(x)), and interval enclosures, f (X ) and g (X ), then f (g (X )) is an interval enclosure of h (x). Proof. From the de nition of interval enclosures: f (g (X )) ff (y) j y 2 g (X )g ff (y) j y 2 fg (x) j x 2 X gg . (1) However, the right hand side of (1) can be written as: ff (g (x)) j x 2 X gg = fh(x) j x 2 X g , which is the set of values that by de nition any interval enclosure of h (x) must contain. Composites of interval enclosures may not be sharp, but they are enclosures. The new theorem is signi cantly more general than the original fundamental theorem of interval arithmetic. No assumption of inclusion isotonicity3 is necessary. Interval enclosures need not be interval extensions4 , which can be problematic for transcendental functions. The theorem generalizes to composites of more than one function of more than one variable and, therefore, can be used to construct interval enclosures of rational or other functions and relations. If the de nition of an interval enclosure is extended to include limit sets at points of singularity, all restrictions on the domain of interval enclosures can be removed, see [4]. It follows at once that point and interval expression-folding are equally valid. With points, di erent results of varying accuracy are computed. With intervals, sharpness varies, but the new theorem guarantees interval containment cannot be violated. An optimizing interval compiler can safely use compile-time expression folding and transformations to produce code that executes fast, or produces sharp results, or both. Example 3 Hansen [2] used the following example from [1] to illustrate how an interval enclosure can fail to be inclusion isotonic. The function is: f (x) = x (1 ? x). The enclosure is: f (X ) = c (1 ? c) + (1 ? 2c) (X ? c) ? (X ? c)2 ; where
An interval function is inclusion isotonic if F (X ) F (Y ) 8X Y . An interval function, F , is an interval extension of a real function, f , if F ([x; x]) = f (x) 8x in the domain of f , see [3], page 21. 3
4
8
c = m (X ), the midpoint of X . f ([0; 1]) = [0; 0:25], while f ([0; 0:9]) = [0; 0:2925], thereby violating inclusion isotonicity. Suppose instead of an enclosure for f , an enclosure for g (x) = exp (x (1 ? x)) is required. Further, assume that for some positive value of ", it is desired to use the interval function g (X ) = h (f (X ) + " [?1; 1]) as an enclosure of g (x), where h (U ) is an interval enclosure of exp (u). From the original fundamental theorem, it is impossible to tell if g (X ) is an enclosure of g (x), because g (X ) is neither inclusion isotonic nor an interval extension of g (x). Theorem 1 holds in this example, because:
1. g (X ) is an enclosure of g (x), because g (x) is a composite of exp and f, 2. f (X ) + " [?1; 1] is an enclosure of f , 3. h (U ) is an enclosure of exp, and 4. g (X ) is the same composite of the enclosures as g is of exp and f .
Composites of dependent5 functions and relations sometimes result in signi cant simpli cations. For example, consider the function h(x; y ) = x op y; where op 2 f+; ?; ; =g : An example of a composite, h, of two dependent functions is: h(f (x; y ); f (u; x)); where f (x; y ) = x + y and h(x; y) = x ? y; from which:
h(f (x; y); f (u; x)) = y ? u.
(2)
A consequence of the extension of composite functions to intervals is that any dependent interval expression can be simpli ed in exactly the same ways as its point counterpart. For example, given interval enclosures of f and h:
h(f (X; Y ); f (U; X )) fy ? u j y 2 Y; u 2 U g = Y ? U .
(3)
6.1 Interval Expression folding and Independence
The presence of interval constants limits the circumstances in which cancelation or other identity transformations can be performed. Consider the following example: Dependent functions or relations share at least one common variable, see Sections 4.2 and 4.3. 5
9
Original Code:
Folded Code:
X=[0,1]*B Y=[0,1]*B Z=Y-X
Z=[-1,1]*B
While it is permissible to take into account the dependence between the two occurrences of the interval variable B, the two occurrences of the interval constant [0,1] are independent For a compiler to use dependence among interval variables, interval expression folding is not required. Subtraction P with cancellation, as de ned in [2], can be used. Suppose the interval Y := =1 X is computed. Later, it P is desired to compute Y := = X for various values of j . When X is not the dominant component of Y , D := Y ? X can be computed rather than Y , in which case the following subtraction with cancellation algorithm can be used: Subtraction with cancelation is de ned: X Y = [a ? c; b ? d]; where (4) X = [a; b]; Y = [c; d], and for this operation to be valid, Y must be an additive component of X . Note that while X X = 0, computing D = Y X is not always as sharp P as = X . n i
j
i 6
i
i
j
j
j
j
j
j
i 6
j
j
i
6.2 Fortran PARAMETER Statements
Sharper intervals can result from assigning the value of interval constants to interval variables. Consider the following two examples: Expression-assignment: \=" Original Code: Folded Code:
X=[0,1] Y=X-X
Y=[-1,1]
X:=[0,1] Y=X-X
Y=0
Using expression assignment, the constant [0,1] is substituted for X in the expression for Y. Value-assignment: \:=" Original Code: Folded Code:
10
Using value-assignment, the value [0,1] is assigned to the variable X, which in turn permits the cancelation to be made in the expression for Y. For this reason, the default de nition of the \=" operator in Fortran PARAMETER declarations must be that of a mathematical identity, or expression assignment, in which case, the de ned parameter is simply a named constant. For a parameter to be de ned as a read-only variable, the new value-assignment operator \:=" must be used.
7 Conclusion The development of interval arithmetic has not explicitly distinguished between interval constants and variables. All intervals have been treated as constants. Unlike interval constants, interval variables have all the algebraic properties of points, a fact that can be used to advantage at compile-time to perform algebraic transformations. In the process of exploring what optimizing transformations an intervalsupporting compiler can automatically perform, the distinction has been made between arithmetic on interval constants and algebra on interval variables. The practical consequence is that compilers can be permitted to perform mathematically equivalent transformations and interval expression folding. Value and expression assignment operators are introduced to control expression folding.
References [1] O. Caprani and K. Madsen. Mean value forms in interval analysis. Computing, 25:147{154, 1980. [2] E. Hansen. Global Optimization Using Interval Analysis. Marcel Dekker, Inc., New York, 1992. [3] R. E. Moore. Methods and Applications of Interval Analysis. SIAM Publ., 1979. [4] G. W. Walster. The extended real interval system. Preprint available at: http://www.mscs.mu.edu/~globsol/readings.html#Walster, March 1998. 11
[5] X3J3. International Standard Programming Language Fortran. Technical report, ISO/IEC 1539-1, 1996.
12
