Uniform rational approximation on large data sets - Wiley Online Library

1569

SHORT COMMUNICATIONS

UNIFORM RATIONAL APPROXIMATION ON LARGE DATA SETS E. H. KAUFMAN, JR.

Department of Mathematics, Central Michigan University, Mount Pleasant, Michigan, U.S.A.

s. F.

MCCORMICK-~AND G. D. TAYLOR?

Department of Mathematics, Colorado State University, Fort Collins, Colorado, U.S.A.

SUMMARY We report on a very effective program for uniform approximation of functions (or discrete data) by generalized rational functions of any number of variables over point sets so large that conventional methods are inapplicable. The points need not be arranged in a regular pattern, but if they are, the number of points is normally limited only by the computer time available. The program extends the differential correction algorithm discussed in an earlier article in this journal by combining it with a first Remes-type exchange procedure, while retaining the flexibility and robustness of differential correction. Furthermore, this algorithm is guaranteed to converge in theory. Testing has shown that, even when the point set is small enough for differential correction to be applied directly, this algorithm is usually faster. We discuss the algorithm and program along with several numerical examples, one of which involves more than three million data points. A FORTRAN listing with illustrative output can be obtained from the first author.

INTRODUCTION In this paper we present a program for computing uniform generalized rational approximations of the form P / Q = ( P ~ + ~ +. .. + ~ ~ + ~ ) /+.( .q. +q&,) ~ $ ~ on a finite point set G, where . . , +,, $?, . . . , are functions defined on G. We shall denote the set of all such quotients by 92.The coefficients of P and Q are chosen to minimize maxG I W . ( f -P/Q)I (or maxG 1 f - W . (P/Q)l if desired) subject to P / Q s u on a finite set U (possibly empty) and P / Q s 1 on a finite set L (possibly empty), where f , W, u and 1 are functions defined on G, G, U and L, respectively ( U and L need not be subsets of G), and Q 3 q on G v U v L, where q is a small positive constant. This last condition makes for more desirable output, lessens round-off problems in finding P/Q, and allows convergence theorems to be proved. Experimental evidence in References 7 and 11 indicates that the differential correction algorithm'-' is the most robust rational approximation algorithm presently available, although it can be inapplicable due to data storage and time limitations. The program described here alleviates these problems, while retaining the flexibility and robustness of differential correction by combining the latter with a variant of the First Algorithm of Remes. This hybrid algorithm is generally not as fast as the combination of differential correction and the Second Algorithm of Remes discussed in References 6, 7 and 9, but since it does not require an alternating theory, it has a wider range of applicability (including, for example, simultaneous approximation3and approximation of functions of several variables).

+',.

+,

t Supported in part by the National Science Foundation under Grant MCS-80-17056.

0029-5981/82/101569-07$01.00 @ 1982 by John Wiley & Sons, Ltd.

Received 24 February 1982

1570

SHORT COMMUNICATIONS

Functions of any number of variables can be approximated (subject to dimension declarations in the program), and if G u U u L is arranged in a rectangular pattern (i.e. as a cross-product of one-dimensional point sets with possibly different spacings in different directions), then usually the only limitation on the number of points is computer time, not storage. If G u U u L is not so arranged (i.e. the data is scattered), two subroutines in the program can be used to obtain a rectangular array by enclosing the points in boxes, yielding no more than one point in each box; the main algorithm will ignore empty boxes.

T H E ALGORITHM Given x k c G u U u L at stage k, the program uses differential correction to compute an approximation R k = & / Q k E #! satisfying

wherepk=inf{maxGnXkIW.(f-R)I: R E # ! ,R = P / Q , R a 1 o n L n X k , R G u on u n x k , and Q Z= q on (G u U u L ) The last inequality is often satisfied automatically, although it can be easily enforced in the differential correction program if desired. (See the comments in subroutines SETVl and SETV.) Note that these inequalities would be satisfied with S1 = 0 if there were no round-off error, and with 6 = 0 if the differential correction program’ actually produced a best approximation; in theory, we may assume A k is arbitrarily close to p k , 1 ’ 5 ’ 8 but 6 and S1 are present to take account of computational reality. The error function e k is defined as follows. For any x E G u u u L , let e k ( x ) = ( A k + 1) . lo6 if Q k ( x )< 77, and if Q k ( x )3 q, let e k ( X ) be e F ( x ) , e k ( x ) or e ; ( x ) accordingly as le?(x)l, ek(x) or - e ; ( x ) is largest (choosing the first in case of a tie), where eF(x) is W ( x ) ( f ( x ) - R k ( X ) ) if x E G and is 0 otherwise, e : ( x ) is l ( x ) + A k - R k ( x ) if x E L and is 0 otherwise and e?(x) is u(x)- Ak - R k ( x )if x E U and is 0 otherwise. Thus, an error arising from a constraint is greater than A k in absolute value iff Rk violates the constraint. Defining ‘extreme points’ in x k as those points in x k where l e k l > A k -TOL, where T OL is a small positive constant, we search for local extrema of t?k in G u U u L by ‘walking uphill’ from the extreme points (this is where the rectangular pattern of points is needed; see Reference 10 and comments in the program for further details), and if at least one point x with l e k (x)l> h k + p is found (where p is a small positive constant), we form x k + 1 from the extreme points in x k , the local extrema found in G u U u L, and a few other points where l e k l is large. If k a 1 and A k S max ( A k - l , h k - 2 ) + p (where AP1= O), we also include the non-extreme points of x k , and if A k < A k - 1 , we also include all of x k . . l (that is, if we d o not have a significant increase in Ak, we do not throw away any points of x k , while if we actually lost ground, then we pull back in points which were thrown away earlier). If, on the other hand, no x was found with j e k ( X ) / > h k +p, we check e k on G u U u L by a coarse scan, followed if necessary by a scan of the remaining points. If some x is found with l e k ( x ) l > Ak +p, we do the walking procedure again from this single point; otherwise, we terminate the algorithm and return R k as our computed approximation. It is showr: in Reference 10 that if W > 0 on G, p d lo6 and p > 6 + 81, then the algorithm

SHORT COMMUNICATIONS

1571

will terminate in a finite number of steps, and the final Rk will satisfy

where p=inf{maxGIW.(f-R)l: R E & ,R=P,JQ, R s l on L, R a u on U, and Q 3 q on G u U u L}. That is, if at each stage differential correction produces an approximation which is near best on the small subset on which it is applied, then the algorithm will eventually terminate at an approximation which is near best on all of G u U v L. Convergence of the algorithm is still guaranteed (under certain mild hypotheses) even if G u U u L is merely compact instead of finite, although the walking and scanning procedures would need to be changed. Rather than doing this, however, one can use the fact that (under certain mild hypotheses) a good approximation on the infinite set can be obtained by computing an approximation on a ‘sufficiently dense’ finite subset (see Reference 10 for details). Note that our program makes this approach more viable by greatly increasing the size of the finite subsets which can be used. In practice, we have found that we can cut down on the searching and save computer time by defining a nested sequence of grids, with the finest being the original grid and the coarsest being the first set on which differential correction is applied. Each grid is formed from its successor by removing all even-numbered points (except the last point) in each direction; the initial grid is taken to be the coarsest grid which contains at least m C n points. Once the algorithm as described earlier has finished with a grid, the final differential correction set and rational approximation are used as a starting point for the next finer grid. Our program does all this automatically. T H E PROGRAM Following is a brief description of the computer program, further details of which can be found in its comments. The program consists of subroutines MSHREF, INPUT, BOXIN and SORT, function subprogram ERRCOM, and the differential correction subroutines DFCOR, SETV1, INPQ, PQED, LNPRO, JELIM and SETV described in Reference 9. These last seven subroutines could be replaced by some other rational approximation package, if desired. BOXIN and SORT can be used to place scattered data points in boxes by initially separating the points by borders in each direction, then successively removing borders (in increasing order of the directional separation distance between points) whenever this can be done without putting two points in the same box. The idea is that, as much as possible, points which are close together in some direction should be on the same level in that direction. Entrance to BOXIN and SORT is achieved from the driver program by the statement CALL BOXIN (NUMDM, NUMGR, COORD, NPONT, L,OCAT, INCRS). The user must supply the number of dimensions in NUMDM, the number of data points in NUMGR and their co-ordinates in C O O R D ; BOXIN will return the dimensions of the box array in NPONT, the index of the box containing point I in LOCAT (I) and the index of the point in the Jth non-empty box in INCRS (J). The purpose of vector INCRS is to make it easier for the main program to determine which point (if any) is in a given box during search procedures. In order to save

1572

SHORT COMMUNICATIONS

central memory space, one could write a short driver program to read NUMDM, NUMGR and COORD, then call BOXIN, and then write NPONT, LOCAT and INCRS to a file where they can be read by the main driver program after BOXIN and SORT have been removed from central memory. Entrance to the rest of the program is achieved by the statement CALL MSHREF (NUMDM, NUMP, NUMQ, JOBNM, NPONT, ENDLO, ENDHI, INWTC, NUMGR, COORD, LOCAT, INCRS, F, X, T, ERROR, C, IPHSE, MESH, ICOUN, ANORM). The first 13 parameters are for input; NUMDM is the number of dimensions, NUMP and NUMQ are the numbers of basis functions in the numerator and denominator, respectively, JOBNM is the index of the present job, NPONT gives the number of points in each direction (not needed for scattered data), ENDLO and ENDHI give the co-ordinates of the points on the grid with smallest and largest indices, respectively (not needed for scattered data), INWTC is an option switch, and NUMGR, COORD, LOCAT and INCRS are as described above (needed only for scattered data). Function data can be entered by formulae in subroutine INPUT (as in the first two examples below). INPUT is called by MSHREF and ERRCOM, which will supply it with the co-ordinates of a point in C, .its indices in each direction relative to the finest grid in JORG, its index (actual in the scattered data case, and lexicographical otherwise) in IORG, the number of grid points (or actual points in the scattered data case) in NUMGR, and the parameters NUMDM, NUMP, NUMQ, JOBNM, INWTC, F, X described herein. The user must insert statements in the subroutine to put the value of f at the point in FVAL, the value of the Jth basis function in BFN (J), and other information (if required; see comments in subroutines INPUT and DFCOR) in INDFV, WVAL, INUPV, UPTBV, INLWV, ALTBV (default values are FVAL = 0.0, INDFV = 1, WVAL = 1.0, INUPV = 0, UPTBV = 0.0, INLWV = 0, ALTBV = 0.0). Alternatively, the values of the function f can be specified by reading them into the vector F in the driver program; in this case subroutine INPUT must contain the statement FVAL = F (IORG). With a small modification in the program, other function data could be entered the same way. MSHREF will return the coefficients of the computed rational function in X , the co-ordinates of the points in the last point set where differential correction was performed in T, the (unmodified for constraints) errors at these points in ERROR, the co-ordinates of the point in the original grid where the largest absolute error was achieved in C, an indicator of the success of the program in IPHSE, the number of outer iterations in MESH, and a summary of the working of the program in ICOUN and ANORM. (In particular, ICOUN (1,MESH) is the number of points in the last differential correction set, ANORM (1,MESH) is the largest (unmodified for constraints) absolute error on this set (i.e. AMESH.I,since ANORM (1,l)= A, = error norm on initial differential correction set), and ANORM (2,MESH) is the largest (modified) absolute error on the original grid.) In the first call to differential correction, the Loeb initialization in the differential correction subroutines is used to get an initial approximation (although the user can override this by changing INWTC and supplying an initial approximation in X ) ; after that, it is used only if differential correction fails when started with the approximation from the previous differential correction set. If upper or lower constraints are used, the numerator coefficients are perturbed a little at first to increase the size of the first error norm computed inside differential correction. This is done because the error norm does not take into account possible constraint violations, and its being smaller than or very close to the best (including constraints) error norm on this differential correction set would cause trouble. In our experience, this first attempt at differential correction has occasionally failed, but, in all the examples we have run, the second try (with

SHORT COMMUNICATIONS

1573

the Loeb initialization) has subsequently succeeded, thus allowing the program to run to a successful conclusion (assuming sufficient time was allotted). The six machine-dependent parameters in the differential correction subroutines are set near the beginning of the subroutines in which they appear; the remaining four machinedependent parameters, plus two parameters dependent on dimensioning and three other parameters which the user may fiddle with to modify the performance of the algorithm, are set at the beginning of subroutine MSHREF. Suggestions are given on setting all of the parameters. A FORTRAN V listing of this program, including output for the three examples discussed below, is available from the first author. TEST RESULTS So far, in our numerical experiments, the program has given excellent results and has allowed us to study some new problems (for example, the approximation by reciprocals of polynomials with non-negative coefficients on large data sets; note that the non-negativity restrictions require only minor additions to subroutines SETVl and SETV). The examples given below were run on a CDC Cyber 172, which has roughly 15 digits of accuracy in single precision; another example can be found in Reference 10.

Example 1 Weapproximate f(x,y, z ) = ~ ~ + y ~ f z ~ o n a 5 1subdivisionof ~ 5 1 ~ 4 1 [O, l]x[O, l]x[0, 13 (i.e. {(0.021,, 0.0212,0.025/3),0 4 lI 4 50, 0 s l2 G 50, 0 s 1, 4 4 0 ) for a total of 106,641 points) by a rational function of the form ( p1+ p 2 x + psy + p4z)/(ql+ q2x + q3y + q42).The computed result after 85 sec execution time is R ( x , y, z ) = (-0.00680+0.5491 lx +0*54911y+ 0-54911 z ) / ( 1 ~ 0 0 0 0 0 - 0 ~ 1 6 9 9-0.16994~ 4~ -0*16994z), with error norm 0.34667, achieved at the extreme points (1,0, O)+, (0, 1,0)+, (1, l , O ) + , (0.48,0.48,0.48)-, (0, 0, l)', (1,0, l ) + , (0, 1, l)+,(1, 1, 1)-. Here the sign indicates the sign of f - R. The largest set on which differential correction was applied had 19 points. The sequence of grids (with the number of outer iterations in parentheses) was 2 x 2 x 2( 1)-+ 3 x 3 x 3(2)+ 5 x 5 x 4(6) + 8 x 8 x 6(1)+ 14 x 14 x 11(1)-+26 x 26 x 21(0)-+51 x 51 x 41(1). By comparison, approximation on a 5 x 5 x 4 subdivision of [0, 11x [0, 11x [0, 11 required 1.8 sec, while straight differential correction required 7.0 sec on this grid. For a further comparison, approximation on a 2 5 1 ~ 1 2 6 ~ 1 0grid 1 (3,194,226 points) required 36min 42sec; the error norm was 0.34668; and the sequence of grids and outer iteration numbers was 2 x 2 x 2(1)-+ 3 x 2 x 2(1) + 5 x 3 x 3(1)+9 x 5 x 5(0)-+1 7 x 9 x8(1)+33 x 1 7 14(1)-+64 ~ x 33 x26(1)+ 126 ~ 6 ~ 45 1 ( 0 ) - + 251 x 126 x lOl(0). In spite of the tremendous number of points of approximation, the largest set on which differential correction was applied had only 19 points. We note that straight differential correction on the full 251 x 126 x 101 grid would have required the solution of a sequence of linear programming problems with nine variables and 6,388,460 constraints.

Example 2 This is a digital filter problem. The function f to be approximated has value 1 on [0, r / 2 ] (the pass band), is undefined on ( ~ / 2 , 3 ~ /(the 5 ) transition band), and has value 0 on [37r/5, T] (the stop band). We impose an equally spaced grid of 25,001 points on [0, T ]and approximate f by a rational function of the form R ( x )= ( p l + p 2 cos x ) / ( q l+ q2 cos x + q3 cos 2x) on those points of the grid which fall into the pass band or stop band, while weighting the error only

1574

SHORT COMMUNICATIONS

half as heavily on [0, n-/2] as on [3n-/5, n-] and imposing the additional conditions R d 1.02 on [O,n-/2] and R 2 0 on ( ~ / 2n-1. , The computed result after 54 sec execution time was R(x) = (0-24162+ 0.24162 cos x)/(l-00000- 0.70447 cos x + 0.56816 cos 2x), with error ~, 3n--/5, n-+, where norm 0.22025, achieved at the 'extreme points' O', 0 - 3 6 4 0 4 ~n-+/2, actually f - R was 0.44049 at 0 and m/2 (due to the weight function), -0.02000 at 0 . 3 6 4 0 4 ~ (R hit the upper constraint), -0.22025 at 3n-/5, and 0.00000 at n- (R hit the lower constraint). The largest set on which differential correction was applied had 10 points. The sequence of grids and outer iteration numbers was 5(1) -+ 8(2)-+ 14(0)-+ 26(1) -+ 50(1)-+ 99(1)+ 197(2)-+ 392(1)+ 783(1) -+ 1564(2)-+ 3126(1)+ 6251(1)--f 12501(1)-+ 25001(1). By comparison, using 101 points required 2.3 sec, while straight differential correction required 4.1 sec on this grid; using 5001 points required 13 sec, and using 5001 points and 10 basis functions (up to cos 4x terms in numerator and denominator) required 37 sec and produced an error norm of 0.00436.

Example 3 This is a the set of (0.1, r/4), (Note: the BOXIN in

scattered data problem in two dimensions. We approximate f(x, y ) = sin (x + y ) on 12 points with polar co-ordinates (0, O), (1,7r/4), (1, n-/2), . . . , (1,27r), (0.1, O), (0.1, n-/2) by a rational function of the form R(x, y ) = (pl+p2x+p3y)/(ql+q2e"). program still operates in rectangular co-ordinates.) The points were arranged by a 4 x 4 array of boxes, with L O C A T = [ 6 16 14 13 5 1 2 4 8 7 11 101

and INCRS=[6 7 8 5 1 10 9 12 11 4 3 21 (Thus, for example, the third non-empty box contains point 8 since INCRS (3)= 8, and this is the fourth box in the array since LOCAT (8) = 4; note that boxes 3, 9, 12 and 15 are empty.) The computed result after 0.75 sec was R(x, y ) = (0-00000+0~75769x + 0*75769y)/(1.00000+ O~OOOOOe"), with error norm 0,08378, achieved at the extreme points (in polar co-ordinates) ( 1 , 3 ~ / 4 ) ' , (1, 3n-/2)-, (1, T ) . , (1,27~)+,(1, n-/2)+, (1,n-/4)-. The largest set on which differential correction was applied had 10 points (i.e. non-empty boxes). The sequence of grids and iteration numbers was 3 ~ 3 ( 1 ) - + 4 ~ 4 ( the 1 ) ; boxes in the 3 x 3 grid were boxes 1, 3 , 4 , 9, 11, 12, 13, 15, 16 (the second row and column were removed from the 4 x 4 grid). By comparison, straight differential correction with this problem required 0.54 sec; thus, straight differential correction was a little faster with this small problem.

REFERENCES

I . I. Barrodale, M. J. D. Powell and F. D. K. Roberts, 'The differential correction algorithm for rational I , approximation', SIAM J. Num. Anal. 9,493-504 (1972). 2. E. W. Cheney and H. L. Loeb, 'Two new algorithms for rational approximation', Numer. Math. 4,124-127 (1962). 3. E. H. Kaufman, Jr. and G. D. Taylor, 'An application of linear programming to rational approximation', Rocky Mtn. J. Math. 4, 371-373 (1974). 4. E. H. Kaufman, Jr. and G. D. Taylor, 'Uniform rational approximation of functions of several variables', Int. J. nurn. Merh. Engng, 9, 297-323 (1975). 5. E. H. Kaufman, Jr. and G. D. Taylor, 'An application of a restricted range version of the differential correction algorithm to the design of digital systems', Int. Series Numer. Math. 30, 207-232 (1976). 6 . E. H. Kaufman, Jr., D. J. Leeming and G. D. Taylor, 'A combined Remes-differential correction algorithm for rational approximation', Math. Comp. 32, 233-242 (1978). 7. E. H. Kaufman, Jr., D. J. Leeming and G. D. Taylor, 'A combined Remes-differential correction algorithm for rational approximation: experimental results', Comp. Math. Appl. 6 , 155-160 (1980).

SHORT COMMUNICATIONS

1575

8. E. H. Kaufman. Jr. and G . D. Taylor, ‘Uniform rational approximation with restricted denominators’, J. Approx. Theory, 32, 9-26 (1981). 9. E. H. Kaufman, Jr., D. J. Leeming and G. D. Taylor, ‘Uniform rational approximation by differential correction and Remes-differential correction’, Int. J. num. Me&. Engng, 17,1273-1278 (1981). 10. E. H. Kaufman, Jr., S. F. McCormick and G. D. Taylor, ‘An adaptive differential correction algorithm’, submitted. 11. C. M. Lee and F. D. K. Roberts, ‘A comparison of algorithms for rational I , approximation’, Math. Comp. 27, 111-121 (1973).