curve fitting using splines, polynomials and rational ... - CiteSeerX

CURVE FITTING USING SPLINES, POLYNOMIALS AND RATIONAL APPROXIMATIONS: A COMPARATIVE STUDY Gerd Vandersteen* Vrije Universiteit Brussel (VUB / TW-ELEC), Pleinlaan 2 B-1050 Brussels, Belgium e-mail: [email protected] Abstract - Spline approximation is often preferred over polynomial approximation. They require less numerical operations, do not suffer from numerical conditioning problems and they can approximate functions with discontinuous derivatives. This paper presents a comparative study between cubic splines, polynomial and rational equi-ripple approximations in curve fitting. The cubic splines use an optimal knot placement method while the polynomial and rational approximations are computed using a modified Lawson algorithm. Numerical conditioning problems are avoided using Forsythe orthogonal polynomials.

1. INTRODUCTION Splines are widely used for interpolation and curve fitting [1], [2], [3]. The use of polynomials is often discouraged in the literature. Two major problems arise when using polynomial approximations. First, conditioning problems can occur when working with improper polynomial basis functions. This problem can be solved using Forsythe orthogonal polynomials [4]. Second, polynomials are not suitable for the approximation of functions with discontinuous derivatives. For functions with only a few singularities, rational functions may be very useful [5]. Although splines can handle these functions easily, they suffer from the (non-trivial) knot placement problem [1]. This paper presents a comparative study of the performance of cubic splines, polynomial and rational equi-ripple approximations for curve fitting. The cubic splines use an optimal knot placement method. The equi-ripple approximations for the polynomial and rational forms are obtained using a modified Lawson algorithm, which has been adapted to handle rational approximations. It solves a sequence of weighted linear least squares problems where the weighting is update iteratively. Forsythe orthogonal polynomials are used to avoid numerical problems. The paper is structured as follows. The computational methodologies used are given first. Second, five functions 4x 4 f (x) = tanh ( 5x ) , e ⁄ e , sin ( πx ) , 1 + x ⁄ 2 and x are approximated using the different methods. The conditioning of the problems and the maximum approximation errors over the range of interest are studied as function of the number of model parameters.

2. COMPARISON SETUP 2.1. Descriptions of the functions used Five fundamentally different functions are considered to compare the approximation capabilities.The first three func4x 4 tions f (x) = tanh ( 5x ) , e ⁄ e and sin ( πx ) are analytic functions with different asymptotic properties. These are chosen to illustrate the importance of the asymptotic behaviour of f (x) on the performance of the polynomial approximations. 1 + x ⁄ 2 and x are considered afterwards, both having discontinuous first derivatives. Without loss of generality, the functions f (x) are considered for x ∈ [ – 1, 1 ] and scaled such that max f (x) = 1 . *.This work is supported by the Belgian National Fund for Scientific Research (NFWO), the Flemish government (GOA-IMMI), and the Belgian government as a part of the Belgian programme on Interuniversity Poles of Attraction (IUAP50) initiated by the Belgian State, Prime Minister’s Office, Science Policy Programming.

2.2. Cubic Splines and the knot placement method Cubic Splines are used for the spline approximations. The knots are determined using the optimal knot placement method described in [1], [2]. The new break sequence is extracted out of the previous spline approximation, under the assumption that this last approximation contains enough information about the function. This implies that the positions of the knots can be seen as additional parameters which need to be optimized together with the coefficients of the spline. In this paper, however, we do not take the number of optimized knots into account into the number of free parameters. The reader should be aware that this favourites splines approximations.

2.3. The modified Lawson’s algorithm The Lawson’s algorithm [6] can be seen as a sequence of weighted linear least squares approximations, where the weighting is chosen such that, under suitable conditions, the solution converges towards an equi-ripple approximation. To apply this technique on rational approximations, consider e(x, α, β) = f (x) – N(x, α) ⁄ ( D(x, β) ) which needs to minimized in an equi-ripple sense with n

N(x, α) =

∑i = 0 α i p i(x)

D(x, β) =

∑i = 0 β i q i(x) .

(1)

(2)

and d

(3)

α i and β i represent the numerator and denominator coefficients, p i(x) and q i(x) describe the i -th order polynomial basis functions. The weighted linear least squares problem of the k -th iteration is then formulated as 2

k

k

k

arg min ∑ w (x)ε k(x, α , β ) k k α , β x ∈ [ – 1, 1 ]

(4)

with k

k

k

k

ε k(x, α , β ) = f (x)D(x, β ) – N(x, α ) .

(5)

2 k k The weighting w (x) is updated iteratively such that w (x)ε k(x, α , β ) become important where the error k k k

k

e(x, α , β ) is large. This is done by choosing w

k+1

k

k

k

w (x) e(x, α , β ) (x) = ----------------------------------------------------------------------- . k k k ∑ w (x) e(x, α , β )

(6)

x ∈ [ – 1, 1 ]

2.4. Forsythe Orthogonal Polynomials The use of polynomial functions is often discouraged in the literature due to the bad numerical conditioning of the least squares problem. This conditioning problem comes from the badly conditioned set of normal equations, i.e. a badly cont ditioned matrix J J , with J the Jacobian matrix. This conditioning problem can be solved using the correct polynomial basis functions. Well known polynomial basis functions are the Chebyshev polynomials which satisfy 〈p i(x)|p j(x)〉

Cheby

=

1

∫–1 p i(x)p j(x) dx = δ ij .

(7)

This means that the set of normal equations for polynomial approximations are best conditioned (i.e. the condition number κ equals 1) if the summation in (4) can be approximated by an integral over the range [ – 1, 1 ] and if the weighting k w (x) is constant for all x . Since this is not the case in general, it is appropriate to use Forsythe orthogonal polynomials [4]. These orthogonal polynomials are constructed using a 3 terms recurrence algorithm such that

〈p i(x)|p j(x)〉

Forythe

=

∑

x ∈ [ – 1, 1 ]

t

w(x)p i(x)p j(x) = δ ij .

(8)

Hence, J J equals the identity matrix by construction.

2.5. Rational Approximation A rational form is considered with Forsythe orthogonal polynomial based numerator and denominator [7]. Hence, its set of normal equations is best conditioned [8] since t

JJ =

In + 1 A

t

A (9) Id + 1

with I n a unity matrix of size n . The modified Lawson’s algorithm is used to obtain a rational equi-ripple approximation. Although not necessary, the degree of numerator and denominator are chosen equal in all cases ( n = d ). Rational models with real poles in the approximation interval [ – 1, 1 ] are rejected.

3. SIMULATION RESULTS All simulation results are obtained using the algorithms described above and on an equidistant grid in the range of x ∈ [ – 1, 1 ] with 50 times more uniformly distributed points than model parameters.

3.1. Approximation Capabilities The approximation rates of the different models are represented in Fig. 1 up to Fig. 5. These show log 10(max( e(x, α, β) )) , for x ∈ [ – 1, 1 ] as function of the numbers of free parameters. The symbols ‘x’, ‘+’ and ‘o’ denote the cubic splines, the polynomial and the rational approximations respectively.

log10(max(|e|))

0

E I D I I E D I D I D I D I D E D I I I E I I I E

-5

D

D

D

D

E E E

-10

E E E

-15 0

5

10 15 20 Nbr. of parameters

E

25

Fig. 1. f (x) = tanh ( 5x ) It is well known that polynomials produce large approximation errors when the original function has discontinuous derivatives or tends asymptotically towards a finite value. This is rather trivial since polynomials can not have discontinuous derivatives and tend asymptotically towards plus or minus infinity. Rational approximations solve both these problems since they can introduce a finite number of singularities and since their asymptotic values depend on the order of numerator and denominator. Only in the case of sin ( πx ) it can be seen that a polynomial approximation outperforms the rational approximation. This is because sin ( πx ) is analytic and does not tend towards an asymptotic value. For x it can be noticed that the

0

I E D I I D

log10(max(|e|))

E

I E

-5

I D I I I I I I I I D E

D D

E

-10

D

E

D E D

-15

E

0

5

Fig. 2. f (x) = e

0

D

10 15 20 Nbr. of parameters

4x

⁄e

25

4

E I D I I I D I

log10(max(|e|))

D

D

I I E I I I I I I

-5

D E D

E

D E

-10

D E D E D

D E D

-15 0

5

10 15 20 Nbr. of parameters

25

Fig. 3. f (x) = sin ( πx )

log10(max(|e|))

0

I I I E D D I I D I D D D D E I I E I I I I E I E

-5

D

D

D

D

E E E E

-10

E E E

-15 0

Fig. 4. f (x) =

5

10 15 20 Nbr. of parameters

25

1+x⁄ 2

cubic-splines perform equally well as the rational approximations. Taking into account all five functions, it can be concluded that the rational approximations outperform cubic splines and polynomial approximations.

3.2. Conditioning of the Problem The conditioning using splines remains approximately constant since every basis is active in a limited range of x . The

log10(max(|e|))

0

EI I DIDII II I D D D D D D D D D D D D D D D D D D D D D D D D E D DIDII I I I I I E I I I E I E I I E E I I E I I I E E I E E E E E E E

-5

-10

-15 0

20 40 Nbr. of parameters

Fig. 5. f (x) =

60

x

condition number using ordinary polynomials increases approximately exponentially with the degree. This limits the maximum usable degree. The set of normal equations is, however, always perfectly conditioned when using Forsythe orthogonal polynomials [4]. Although the conditioning of the rational approximation increases with the number of model parameters, conditioning problems did never occur in the presented examples. The problem only becomes badly conditioned when the approximation error becomes comparable with the precision of the numerical representation. A typical conditioning versus the approximation error is given in Fig. 6. The symbols ‘x’, ‘+’ and ‘o’ denote the cubic splines, the polynomial and the rational approximations respectively.

log10(Cond. Nbr.)

15

E E E

10

E E E

5

E E

0

E E E I III I I I I I II E DDDDDDDDDDDDDDD

0

-5 -10 log10(max(|e|))

-15

Fig. 6. log 10(κ) versus the log 10(max( e(x) )) for f (x) = tanh ( 5x )

4. CONCLUSIONS The performances of cubic splines, polynomial and rational equi-ripple approximations for curve fitting problems are discussed and experimentally verified. Conditioning problems are avoided using Forsythe orthogonal polynomials. The superiority of rational approximations for both analytic and non-analytic functions has been demonstrated experimentally. The rational approximations outperform both spline and polynomial approximations, i.e. it is the most robust approximation w.r.t. the properties of the function which needs to be approximated.

REFERENCE [1] Boor C., “A Practical Guide to Splines”, Springer-Verlag, New York, 1978.

[2] Boor C., “Spline Toolbox User’s Guide”, The Mathworks, Inc., 1992. [3] Ahlberg J.H., “The theory of splines and their applications”, Mathematics in science and engineering, Academic Press, Nilson and J.L. Walsh. - London, 1967. [4] Forsythe G.E., “Generation and use of Orthogonal Polynomials for Data Fitting with a Digital Computer”, J. Soc. Indust. Appl. Math., Vol. 5, pp. 74-88, 1957. [5] Lorentz G.G., “Approximation of Functions”, Chelsea Publishing Company, New York, 1986. [6] Rice, J.R. and K.H. Usow, “The Lawson Algorithm and extensions”, Mathematics of Computation, Vol. 22, pp. 118127, 1968. [7] Vandersteen G., Y. Rolain, J. Schoukens and R. Pintelon, “On the Use of System Identification for Accurate Parametric Modelling of Non-linear Systems using Noisy Measurements”, IEEE Transaction on Instrumentation and Measurement, April 1996, Vol. 45, No. 2, pp. 605-609. [8] Forsythe G.E. and E.G. Straus, “On Best Conditioned Matrices”, Proceedings of the American Mathematical Society, Vol.6, pp.340-345, 1955.

THE AUTHOR Gerd Vandersteen (°1968) received the degree of electrical engineer (burgerlijk ingenieur) in July 1991, from the Vrije Universiteit Brussel (VUB), Brussels, and an additional degree in telecommunication, in July 1992, from the Université Catholique de Louvain (UCL), Louvain-la-Neuve, Belgium. He is presently Aspirant of the National Fund for Scientific Research (NFWO) at the VUB in the Electrical Measurement Department (ELEC).