Oct 6, 1994 - It states that the error and the condition number cannot both be kept small. Bounds on the Lebesgue constants are obtained as a byproduct.
Error Estimates and Condition Numbers for Radial Basis Function Interpolation Robert Schaback Institut fur Numerische und Angewandte Mathematik Universitat Gottingen D { 37083 Gottingen Germany Revised for Advances in Computational Mathematics October 6, 1994
Abstract: For interpolation of scattered multivariate data by radial basis functions, an \uncertainty relation" between the attainable error and the condition of the interpolation matrices is proven. It states that the error and the condition number cannot both be kept small. Bounds on the Lebesgue constants are obtained as a byproduct. A variation of the Narcowich{Ward theory of upper bounds on the norm of the inverse of the interpolation matrix is presented in order to handle the whole set of radial basis functions that are currently in use.
1
Introduction
Interpolation by \radial" basis functions requires a function : IRd ! IR, a space IPmd of d{variate polynomials of degree less than m, and interpolates data values y ; . . . ; yN 2 IR at data locations (\centers") x ; . . . ; xN 2 IRd by solving the system 1
1
N X j =1 N
X j =1
j (xj ? xk ) + j pi(xj )
Q X `=1
+
1kN
`p` (xk ) = yk ;
1iQ
= 0;
0
(1:1)
m? 1 + d
where Q = . See Table 1 for the most commonly for a basis p ; . . . ; pQ of d used examples. In self{evident matrix formulation the system (1.1) reads as
IPmd ,
1
A P T y = ;
P 0 0 and solvability is usually guaranteed by the requirements rank (P ) = Q N and k k T A 2 2
(1:2) (1:3)
for all 2 IRN with P = 0, where is a positive constant. The latter condition is called \conditional positive de niteness of order m" if it holds for a speci c pair (m; ) and for arbitrary sets X = fx ; . . . ; xN g IRd . The condition rank P = Q N can be called \IPmd { nondegeneracy of X ", because on such sets polynomials from IPmd are uniquely determined by their values. Details can be found in review articles by M.J.D. Powell [14], N. Dyn [7] and M. Buhmann [5]. 1
If m = 0, then IPmd and P disappear. In this case kA? k ? holds in the spectral norm. More generally, as we shall see in the next section, the quantity ? controls the sensitivity of the solution vector with respect to variations in the data vector y. Thus one is interested in lower bounds on that are as tight as possible. Such bounds were obtained by Ball [2], Narcowich and Ward [11], [12], [13], Ball, Sivakumar, and Ward [3], Baxter [4] and Sun [17], while lower bounds were supplied by Schaback [16]. If the data are values of a function f , one usually considers a xed function space F and evaluates the error of the interpolant 1
sf =
N X j =1
j ( ? xj ) + 1
1
2
Q X l=1
1
` p`
(1:4)
de ned by a solution of (1.1) with yk = f (xk ). If F is de ned via itself in a natural way, the space F carries a speci c seminorm j:jF and the bound for the error f (x) ? sf (x) takes the form jf (x) ? sf (x)j jf jF P (x); where the power function P (x) just is the norm of the error functional on F evaluated at x. Note that P depends on x; X; , and F , but not on f . For sake of completeness, we note that F contains all functions f : IRd ! R with
f (x) = (2)?d where f^ is in L (IRd ) and
Z
IR
^(!)ei!T xd!; f d
x 2 IRd
Z jf^(!)j jf jF := '(!) d! < 1:
1
2
2
IRd
Here, ' denotes the d{variate generalized Fourier transform of . It is assumed to be a positive continuous function on IRd n f0g satisfying a variational equation
Z Z IR
d
IR
d
v(x)v(y)(x ? y)dxdy = (2)?d
Z
IR
'(!)jv(w)j d! 2
d
(1:5)
for all test functions v in the Schwartz space of tempered functions that additionally satisfy v^ (0) = 0 for all 2 IN d ; j j < m: Note that ' coincides in the sense of Jones [8] with the generalized Fourier transform of in the context of tempered distributions. This approach goes back to W. Madych and S.A. Nelson [9], where (1.5) explicitly occurs. Table 2 contains the functions ' corresponding to the various cases of . If ' decays at least algebraically at in nity, e.g.: 0 < '(!) < C (1 + k!k)?k ; then F contains Sobolew space ( )
Wk 2
0
(IRd ) =
Z k f jf^(!)j (1 + k!k ) d! < 1 : IR 2
2
d
However, if ' decays exponentially, e.g.: for (x) = e? kxk2 , then F consists of C 1 functions. Further details can be found in W. Madych and S. Nelson [9], [10], N. Dyn [6], and R. Schaback [15]. Numerical observations and theoretical results have revealed that the error and the sensitivity, described by P (x) and ? , seem to be intimately related. In particular, there is no case known where the error and the sensitivity are both reasonably small. There is a dichotomy: Either one goes for a small error and gets a bad sensitivity, or one wants a stable algorithm and has to take a comparably large error. This e ect is reminiscent the Uncertainty Principle in quantum mechanics, and here it will take the very simple form P (x) ? (x) 1 (1:6) where (x) is de ned via (1.3) for the matrix Ax that arises after adding an additional row and column for A of (1.2) for the location x = x . We shall prove (1.6) in section 2 and draw several conclusions. 1
2
1
0
2
To explain the latter, we have to introduce some additional notation. First, the known upper bounds for P (x) take the form P (x) F (h(x)) (1:7) where F : IR> ! IR> is decreasing and h(x) := kymax min ky ? xj k (1:8) ?xk j N 2
0
0
2
2
1
measures the density of centers xj from X around x. Note that h(x) depends on X and > 0 (which is kept xed), but not on . Equation (1.7) holds uniformly for all points x and sets X such that h(x) h ; the positive constant h being dependent on d; m; , and . It requires no additional hypotheses on exceeding those we made so far. The function F is dependent on the decay of the generalized Fourier transform ' of at in nity. This is equivalent to being dependent on the smoothness of . See the references in Table 1 for details concerning the construction of F . Second, the known lower bounds for (x) take the analogous form (x) G(q(x)); where q(x) is the separation distance within the set X [ fxg, i.e.: q(x) = 21 min kx ? xk k (1:9) j 0 which leaves room for further research. 0
0
0
0
2
2
3
For the reader's convenience, we list the known examples for functions F and G in Table 1. (x) = (r); r = kxk
F (h)
2
G(h)
r ; 2 IR> n 2IN
h
thin{plate splines
[18]
h [2]: d = = 1 [3], pg. 419: 2 (0; 2) [13], x VI: = m ? d=2; d odd
2 2IN thin{plate splines
h [18]
h [13], x VI: = m ? d=2; d even
( + r ) = ; 2 IR n 2IN
e? h
h e? h2 [11], pg. 90: = 1 = ; d = 2,
Multiquadrics
> 0. [10]
0
(?1)
=2r log r;
1+
2
2
2
0
6
2
he? 2hd [3], pgs. 422{423: = 1 = h exp(?12:76 d=h)
e? h2 h?d e? h2 [13], pg. 90: = 1 > 0 [10] h?d exp(?40:71d =( h ))
e? r2 ; > 0 Gaussians
2
2
2d= Kk?d= (r)(r=2)k?d= h k?d h k?d ?(k) as in [18] 2k > d, Sobolev splines Table 1: All entries are modulo factors that are independent of r and h, but possibly dependent on parameters of . Unreferenced cases for G are treated in section 3, where we included the constants. 2
2
2
2
2
Another useful result of our analysis will be a bound on the Lagrange functions u (x); . . . ; uN (x) corresponding to interpolation by (1.1) on X = fx ; . . . ; xN g. In the above context, we get 1
1
1+
N X j =1
uj (x) P((xx)) FG((hq((xx)))) ; 2
2
and this serves to prove that the Lagrange functions cannot grow too badly in regions where there are suciently many regularly distributed centers.
2
Basic results
First we assert that ? of (1.3) controls the sensitivity of the solution vector 2 IRN of (1.2) with respect to perturbations of the data vector y 2 IRN . In fact, 1
T A = T y 4
follows from (1.2) and implies
k k ? kyk ; 1
2
2
if (1.3) holds. Similarly, an upper bound T A k k ; > 0 (2:1) of the quadratic form induced by A yields ? kyk k k and in case of a perturbation y + y of y that leads to a perturbation + of , we get the condition{type estimate k k kyk : kyk k k Since the determination of the other solution vector 2 IRQ of (1.2) can be viewed as a problem of polynomial interpolation, the main part of the sensitivity analysis of \radial" basis function interpolation problems consists in nding good bounds for and . We now turn to the function P (x). Due to the special choice of the space F and the seminorm j:jF , the function P (x) can be explicitly written as 2 2
1
2
2
2
2
2
2
2
P (x) = (0) ? 2 2
N X
uj (x)(xj ? x)
j =1 N N
XX
+
j =1 k=1
uj (x)uk (x)(xj ? xk );
where u (x); . . . ; uN (x) are the Lagrange basis functions for interpolation, i.e. (1.6) equals 1
sf =
N X
f (xj )uj
j =1
(see e.g. Schaback [15]). We now set x := x and add a rst row and column to the N N matrix A in (1.2) to get a (N + 1) (N + 1) matrix Ax. With ux := (1; ?u (x); . . . ; ?uN (x))T 2 IRN we then have P (x) = uTx Axux and simply use (1.3) for Ax in the form (x)k k T Ax for all 2 IRN to get ! N X (x) 1 + uj (x) P (x): (2:2) 0
+1
1
2
2 2
+1
2
2
j =1
If we take (2.1) for A replaced by Ax, then (x) P (x) 2
1+
N X j =1
uj (x)
We now can read o a series of statements:
5
2
(x):
Theorem 2.1 For any IPmd {nondegenerate set X = fx ; . . . ; xN g and all x 2 IRd n X we 1
have
11+
If corresponds to A in (1.3), then
N X j =1
uj (x) P (x)? (x): 2
2
1
min Pj (xj ) X n fxj g is IPmd {nondegenerate; 1 j N 2
where Pj is the power function for X n fxj g.
2
In all practically relevant cases we have bounds of the form
P (x) F (h(x)) (2:3) (x) G(q(x)) with continuous and decreasing functions F and G for small arguments. The relation between q(x) and h(x), as de ned in (1.9) and (1.8), is 2
2q(x) min kx ? xj k h(x); j N 1
(2:4)
but in order to relate the bounds in (2.3) we have to look at conditions under which h(x) can be bounded from above in terms of q(x). De nition 2.2 We call a set of centers X = fx ; . . . ; xN g quasi{uniform for a compact set
IRd, if there are > 0 and h > 0 such that a) h(x) h for all x 2
b) 2q = min kx ? xk k h , j 0 satisfying
d
(3:6)
0 d 1 d +1 1 ? 2 + 1 @ A M 12 q 9
(3:7)
M 6:38q d :
(3:8)
2
or, a fortiori,
(3:5)
2 2
8
Proof:
We start with any M > 0 and the characteristic function
M (x) =
( 1 kxk M ) 2
0 else
of the L ball with radius M . Then we de ne 2
' (M )? d2 + 1 ( M M )(!) (!) := M (!) := 2dM d d= and use the calculations of [13] to get (3.2) via 0
2
supp (
M)
=
x 2 IRd : kxk 2M =: C (0) M 2
2
d d= k M M k1 vol(CM (0)) = M d 2d : 2
? 2 +1 The radial basis function M corresponding to M is obtained via the inverse Fourier transform d= M (x) = 2M Jd= (M kxk ) kxk 2
and the Convolution Theorem as
2
2
2
' (M )? 2d + 1 ( M M )_(x) M (x) = 2d M d d= ' (M )? d2 + 1 kxk?dJd= (M kxk ) = 2d d= with J denoting the Bessel function, satisfying d Jd= (z) 2z ; z > 0 lim z?d Jd= (z) = d 1d z! 2 ? 2 +1 as was proven in [13]. The second formula yields 0
2
0
2
2
2
2
0
2
2
2
+2
2
2
(3:9)
2
d p M (0) = 'd(M ) 4M ? 2 +1 and we assert diagonal dominance of the quadratic form in (3.3) by a suitable choice of M . We have 1 0 0
T B k k2 2
N X B M (xj ? xk )C A @ M (0) ? max j N 1
9
k =1
k6=j
by Gerschgorin's theorem, and the nal bound will be of the form
d
p ; 12 M (0) = ' d(M ) 4M 2? 2 + 1 because we shall choose M such that X max M (xj ? xk ) 21 M (0): j N 0
1
(3:10) (3:11)
k =1
k6=j
This is done by a tricky summation argument of Narcowich and Ward [13] that proves (3.11) for M satisfying (3.7). It remains to show that (3.8) implies (3.7). We use a variation of Stirling's formula in the form p ?(1 + x) 2xxxe?xe = x; x>0 to get ? d + 1 dd (2e)?d e = d; 9 2 9 1 d +1 1 d d +1 1 d ?d + 3 d ( d + 1) 1 d 9 e (2e) 9 ? 2 +1 d 3p2e e = d 0:531 such that M 6:q38 d is satisfactory for all cases. 2 Note that this technique completely ignores the additional conditions on that might lead to a larger lower bound. The advantage is that the result is fairly general and can be applied in all of the cases. We incorporated the results into Table 1 to supply a number of missing cases. This was done by application of Theorem 3.1 to the entries in Table 2. To keep the formulae short, we used (3.8) instead of (3.7), which would yield sharper, but much more complicated bounds. To treat multiquadrics as a speci c example, we have to evaluate ' (M ) via ' (R) := 0; = d +2 : We use equation 9.6.23 of Abramowitz and Stegun [1] to get 1 Z1 ? + 2 K (r) p(r=2) = e?rt(t ? 1)? = dt 1 12
2
2
+1
1 3
2
2
1 6
0
1
0
2
1 2
1
=
Z1 1
e?r
e?rt(t ? 1) ? dt 2
Z1 0
e?rss ? ds 2
= ?(2 )e?r r? 2
10
1
1
and
p ?(2 ) ?R ? ' (R) 1 2 e R : 2
? +2 By the doubling formula for the ? function this can be simpli ed to 1
' (R) 2? ?( )R? e?R: 1
1
2
Then we have
' (M ) = 0
2d= (2 ) ' (2 M ) 2
2
? ?2 d= 2 (2 ) 2? ?( )e? M (2 M )? ? ?2 d= ? ? M = ?()M e ? ?2 2
2
1
2
2
1
2
2
2
d + ? 2 1d 2 d M ? exp(?2 M ) ? ?2 ? 2 + 1 as incorporated into Table 2 with M = 6:38d=q. For < 0 and large values of q there is a better choice of M , but we leave this exotic case to the reader. Similar tricks can be done in the Gaussian and Sobolev cases. and
2 +1
Acknowledgement: The author thanks the referees for a series of helpful remarks
and suggestions.
11
12
References [1] M. Abramowitz and I.A. Stegun. Handbook of Mathematical Functions. Dover, New York, 1970. [2] K. Ball. Eigenvalues of Euclidean distance matrices. Journal of Approximation Theory, 68:74{82, 1992. [3] K. Ball, N. Sivakumar, and J.D. Ward. On the sensitivity of radial basis interpolation to minimal distance separation. Journal of Approximation Theory, 8:401{426, 1992. [4] B.J.C. Baxter, N. Sivakumar, and J.D. Ward. Regarding the p{norms of radial basis interpolation matrices. Technical Report 281, Department of Mathematics, Texas A & M University, 1992. [5] M.D. Buhmann. New developments in the theory of radial basis function interpolation. In Multivariate Approximations: From CAGD to Wavelets, pages 35{75. K. Jetter and F.I. Utreras, editors; World Scienti c, London, 1993. [6] N. Dyn. Interpolation and approximation by radial and related functions. In C.K. Chui, L.L. Schumaker, and J.D. Ward, editors, Approximation Theory VI. Academic Press, Boston, 1989. [7] N. Dyn, W.A. Light, and E.W. Cheney. Interpolation by piecewise{linear radial basis functions, I. Journal of Approximation Theory, 59:202{223, 1989. [8] D.S. Jones. The Theory of Generalized Functions. Cambridge University Press, 1982. [9] W.R. Madych and S.A. Nelson. Multivariate interpolation: a variational theory. Manuscript, 1983. [10] W.R. Madych and S.A. Nelson. Bounds on multivariate polynomials and exponential error estimates for multiquadric interpolation. Journal of Approximation Theory, 70:94{ 114, 1992. [11] F.J. Narcowich and J.D. Ward. Norm of inverses and condition numbers for matrices associated with scattered data. Journal of Approximation Theory, 64:69{94, 1991. [12] F.J. Narcowich and J.D. Ward. Norms of inverses for matrices associated with scattered data. In P.J. Laurent, A. Le Mehaute, and L.L. Schumaker, editors, Curves and Surfaces, pages 341{348. Academic Press, Boston, 1991. [13] F.J. Narcowich and J.D. Ward. Norm estimates for the inverses of a general class of scattered{data radial{function interpolation matrices. Journal of Approximation Theory, 69:84{109, 1992. [14] M.J.D. Powell. Univariate multiquadric interpolation: Some recent results. In P.J. Laurent, A. Le Mehaute, and L.L. Schumaker, editors, Curves and Surfaces, pages 371{382. Academic Press, 1991. [15] R. Schaback. Comparison of radial basis function interpolants. In Multivariate Approximation. From CAGD to Wavelets, pages 293{305. K. Jetter and F. Utreras, editors; World Scienti c, London, 1993. 13
[16] R. Schaback. Lower bounds for norms of inverses of interpolation matrices for radial basis functions. To appear in Journal of Approximation Theory, 1994. [17] X. Sun. Norm estimates for inverses of Euclidean distance matrices. Journal of Approximation Theory, 70:339{347, 1992. [18] Z. Wu and R. Schaback. Local error estimates for radial basis function interpolation of scattered data. IMA Journal of Numerical Analysis, 13:13{27, 1993.
14
Blatt bitte entfernen. Liegt nur bei, damit die Referenzierung fuer Tabelle 2 (die ja extra gedruckt wird) klappt. Table 2: These table entries explicitly contain the relevant constants.
15
