An Algorithm for the shape parameter in radial basis ...

An Algorithm for the shape parameter in radial basis functions interpolation Jafar Biazar1 , Mohammad Hosami Department of Applied Mathematics, Faculty of Mathematical Sciences, University of Guilan, P.O. Box 41335-1914 P.C.4193822697, Rasht, Iran

Abstract: Radial basis functions methods depend on an auxiliary parameter, called the shape parameter that plays an important role, in the convergence of the method and the accuracy of the results. Many studies have been devoted to searching for an optimal value for the shape parameter. In many cases, approaches are based on minimization of the error function, such as Rippa’s Algorithm. In this paper, a novel Algorithm will be introduced that determines an interval for suitable values of the shape parameter. To show the efficiency of the Algorithm, illustrative examples, including support points in one and two-dimensional spaces and also two-dimensional elliptic boundary value problems are presented, and the results are compared with those of Rippa’s Algorithm. Keywords: Radial basis functions; Shape parameter; Interpolation; boundary value problems Subject classification codes: 97N50, 65N35, 30E25

1. Introduction Radial basis functions (RBFs) approximation has been used in many practical problems, in science and engineering. Initial applications of RBFs commences from the interpolation of multivariate scattered data [ 1-4]. Many researchers have developed the theory and applications of the RBFs, in various scientific problems [5-12]. There is a parameter in most of the famous Radial basis functions called the shape parameter, which is usually user-defined. There are some Algorithms for optimal value of this parameter. Three of the most famous values have been introduced by Hardy [4], Franke [12], and Fasshauer [13]. These values are depending on the number of the center nodes and the position of them. There are some other methods for choosing a value for the shape parameter, by minimizing the interpolation error. Rippa is one of pioneers in this group [10]. Some other works in this topic is due to Schuerer [14], Roque and Ferreira [15], and Bayona et al. [16]. In all of these strategies, an optimal value of the shape parameter is obtained. In this paper, a new Algorithm for determining an interval for the good values of the shape parameter is presented. This paper is organized as the following. Section 2 deals with an introduction to Radial 1

Corresponding author; Emails: [email protected]; [email protected];

basis functions approximation method. In section 3, some strategies for selecting an optimal value for the shape parameter, is reviewed briefly. A novel Algorithm is introduced in section 4. There are five illustrative examples in section 5 .Three of them are about one-dimensional interpolation, and last two are examples of two-dimensional interpolation and a boundary value problem. Each of these examples includes a significant point. 2. Radial Basis Functions approximation 2.1 Radial basis function interpolation To interpolate a given scattered data, {( xi , ui )}in1 by RBFs, as a basis, interpolation will present as a combination of n, RBFs. n

u ( x) 

   ( r ), j

j

 j  ,

(1)

j 1

where r j || x  x j ||2 ,  ( r ) is a RBF, and  j is the coefficient that should be determined. Radial basis functions are real valued functions, which are dependent on the Euclidean distance r j between x j and other points. the commonly used RBFs are Multiquadric (MQ), Gaussian, Thinplate Spline (TPS), and compactly supported RBFs (CS-RBFs) [6,7,17]. MQ is defined as  ( r )  c 2  r 2 or  (r )  1  ( r ) 2 , where  or c are called the shape parameter that is usually defined by the user. Throughout this paper we take earlier form of the MQ. By collocating the interpolation conditions ( u ( x j )  f j , j  1,.., n ), a system of equations is obtained as the following matrix form (2)

A  f ,

where aij   (rij ) and rij || xi  x j ||2 . By solving the linear system (2), the coefficients  j ( j  1,..., n ) are determined.

2.2 RBF method to steady PDE problems Consider the following boundary value problem

L(u ( x))  f ( x ), B(u ( x))  g ( x),

x  , x  ,

where L is a linear differential operator, B is a boundary differential operator,  is a bounded and connected domain, and  is the boundary of  . The Kansa method to solve equation (3) is as follows:

(3)

n

Let X  X 1  X 2 be a set of n distinct centers where X 1   xi i 1 1 contains n1 interior points n  n n

in the domain  , and X 2   xi i 1 n 21 1

contains n2 boundary points.

Suppose that u ( x) is approximated in the form (1). For the interior and boundary centers, we have

L(u ( xi ))  f ( xi ),

i  1,..., n1

B(u ( xi ))  g ( xi ),

i  n1  1,..., n.

(4)

Substituting (1) into (4) results in

 

n j 1

 j L( (rij ))  f ( xi ),

n

 B ( (rij ))  g ( xi ), j 1 j

i  1,..., n1

(5)

i  n1  1,..., n.

In matrix form, equation (5) can be written as

 L  F   B     G  ,     where ( L ) ij  L( ( rij )) , ( B ) ij  B ( ( rij )) , ( F ) i  f ( xi ) , and (G )i  g ( xi ) . By solving equation (6), the vector of unknown coefficients  , will be determined. However, the Kansa method has been successfully applied to various physical problems [8,9,13,18,19 ], but the mathematical proof of its solvability is still not available. The accuracy of the interpolation by RBFs depends on various factors. Some of the most important ones are the family of RBFs, which is selected, nodes distribution, and the value of the shape parameter. As mentioned before, finding the optimal value for the shape parameter is still an open problem. Although a great amount researches have been done to determine some appropriate values for the shape parameter, but so far no interval for suitable values of the shape parameter introduced yet. An interval, that this article proposes. 3. Some strategies to select a shape parameter The value of the shape parameter, in radial basis functions such as Multiquadric, Gaussian, etc… has a direct influence on the accuracy of the method. In general, the choice of the value of the shape parameter is experimentally or user defined. Many scientists and mathematicians use a trial and error procedure to choose a value of shape parameter. Most famous strategy is plotting the error function respect to shape parameter and choosing a value with a minimum error. This is obvious that it is not always practical. Although for the shape parameter some methods are provided in the literatures. Hardy, introduced a value for the shape parameter, based on the distance of any data point to its nearest neighbour [4]

(6)

c  0.815d , where d is the average distance between nodes. Franke, proposed a value for the shape parameter based on the position and the number of the nodes in the data set [12]

c  1.25 D / N , where D is the diameter of the minimal circle including all the data points. Another well-known proposed value of the shape parameter is due to Fasshauer, which is just dependent on the number of the nodes in the data set [ 17].

c  2/ N, where N is the number of the nodes. There are some methods based on Leave-One-Out Cross validation (LOOCV) which is a wellknown method in statistics. In [10], Rippa proposed an Algorithm based on LOOCV that determines an optimal value for the shape parameter by minimizing a cost function as follows

E  [ E1 , E2 ,..., En ]t k where Ek  uk  s ( xk ) , and s k ( x ) is the approximate function to a reduced data set obtained

by removing the point xk from the original data set. To avoid high cost of calculating the cost vector E , Rippa derived a simple formula for the error at each data points as

Ek 

k , Akk1

where k is kth component of the vector  and Akk1 is kth diagonal element of the inverse of the coefficient matrix. The Algorithm finds an optimal value for the shape parameter such that the cost functions E is minimized. For further details see [10]. . Many researchers have followed the Rippa’s idea, more or less. For example, Tsai et al [18], Scheuerer [14]; Gherlone et al [20], and Luh [21].

In this paper, we introduce a new Algorithm for selecting a good value of the shape parameter. The Algorithm is not based on minimizing the error function, but considers the convergence property of the approximate function at each point in the domain. The Algorithm is introduced in the next section. 4. The New Algorithm The Algorithm is designed to determine an interval of suitable values for the shape parameter. Suppose that the radial basis functions are applied to approximate a function say u ( x ), over a

known domain D, with a given number of collocation points. The Algorithm determines an interval of suitable values of the shape parameters so that the approximate function u * has the best accuracy. Let R denote this interval. Note that the criterion, the best accuracy, means when the accuracy resulted for different values of the shape parameter, be compare. The Algorithm is based on this idea: Let n

u* ( x;  )   j 1  j  (rj ) interpolates the function u ( x ), over the domain D . Note that the approximate function

u* ( x;  ) depends on  . Thus for suitable values of  , as long as the approximate function (7) is convergent to exact solution, we have

x  D ,

u* ( x;  )  u( x).

For all valid values of  , keeping x*  D fixed, the corresponding values of u * ( x* ;  ) should converge to the same value u ( x * ) . Thus by plotting u * ( x* ;  ), as a function of  , the plot will be horizontal over the interval R , i.e. almost parallel to the  axis. Any value in this interval can be taken, as a suitable value for the shape parameter. This idea can be written as the following Algorithm, to find a good value of the shape parameter, without any information about the exact solution, or the error function.

Algorithm (The shape parameter Interval)

1. Let   [ 1 ,  2 ,...,  N ]t 2. For k  1 to N 3.

Do

evaluate the value of u* ( x* ;  k ) .

4. Plot the curve of u * versus  . 5. Select the subinterval R , over which the curve is almost parallel to the  axis. 6.

Choose any value of the selected interval, as a good value for the shape parameter,

 good .

Note that based on the property of meshless methods, this Algorithm does not depend on the dimension of the problem. Consequently, the Algorithm for approximation in one, two, or three dimensions works properly. 5. One, and Two-Dimensional Examples To demonstrate the role of the proposed Algorithm for selection a good value for the shape parameter, some numerical examples are presented. These examples attempt to approximate a

(7)

given function using MQ-RBF, and the proposed Algorithm. The central nodes are uniformly distributed. In each example, to show the efficiency of the Algorithm, maximum error and RMS error of the approximations obtained by applying the proposed value (called  good ), the value of the shape parameter introduced by the Rippa’s Algorithm (called  opt ) and a value of the shape parameter out of the obtained interval (called  ) are compared.

5.1 One-Dimensional Examples:

Example 1: The first example deals with the following one-dimensional function.

f1 ( x)  2 sec h 2 ( x),

x  [ 10, 40] .

First, to estimate a good value for the shape parameter,  is selected in [0.3, 5]. Figure 1 shows

f1* ( x* ;  ) , the curve of approximate function at some different test points x*  0.5 , x *  0 , and x*  2.5 versus the shape parameters. Note that the values of x * have been selected in the range that the function f1 ( x) have steepest gradient. By visual inspection it is obvious that the line segment in this plot is over the interval [0.5, 2]. This means f1* ( x* ;  ) as a function of the shape parameter  has the same value in the interval [0.5, 2]. Thus the values of the shape parameters in this range approximate the function better than others do. Thus the value of the shape parameter to approximate the function f1 can be chosen in the interval [0.5, 2]. Figure 2 shows the error of the approximate function by applying three values for the shape parameter; the proposed value by the Algorithm,  good =1,  opt =0.5468 (obtained by Rippa’s Algorithm) and a shape parameter out of the selected interval. 200 center nodes have been applied to approximate the function. According to Figure 2, the proposed value, results in the best accuracy for interpolating function f1 . Table 1 show that  opt is in the proposed interval (

 opt  R ).Moreover, the maximum error and RMS error of the various shape parameters have been shown in Table 1.

Example 2: Let us consider

f 2 ( x)  arctan(5 x), in the interval [-2,2]. In this example three different test points x *  0.5 , x *  0 , and x *  0.5 are applied. Figure 3, shows the curve of f 2* ( x* ,  ) in the interval [0.5, 5]. By visual inspection, the interval [1.5,5] can be selected as a range of good values for the shape parameter. In fact, this is the best interval to select a value for the shape parameter to evaluate the exact value of f 2 ( x* ) . By applying Rippa’s Algorithm the optimum shape parameter is 2.7931. To compare the error of applying different values of shape parameter, the error function

of the approximate function with  good ,  opt , and a shape parameter out of the selected interval have been plotted in Figure 4. Now a problem is arisen. Is this range optimum to evaluate all the points in the domain? We found that, the optimum range may be different to various test points. The empirical results show that the interval may not be the optimal range to evaluate the function in whole the domain, but it is a valid interval. The maximum error and RMS error of applying different values of the shape parameters are summarized in Table 1.

Example 3: Consider 3

f3 ( x)  e x  cos(2 x), that is defined on the domain [0,1]. Three top plots of Figure 5, show f3* ( x* ,  ) for x *  0.1 ,

x *  0.5 , and x *  0.9 versus the shape parameter  in the interval [0.2,5]. The curves show that the shape parameter can be selected in the interval [1,5]. To get shorter interval, one can apply the Algorithm in this obtained interval, once more. In almost all mathematical packages, it is possible to rerun the “plot command” on a subinterval. Thus there is no need to rerun the Algorithm. Three bottom plots of Figure 5 show the curve over this domain. The line segment in these plots is over the interval [4,5]. To show the reliability of the selected interval, three error function of the approximation with   4 (selected in proposed interval),   3.8663 (obtained by Rippa’s Algorithm), and   0.5 (a value out of the proposed interval) is shown in Figure 6. According to Table 1,  opt  R and  good produces the best accuracy for interpolating f 3 ( x ) .

5.2 Two-dimensional interpolation Example 4: Consider the Franke function in two-dimensional interpolation.

f 4 ( x, y )  franke( x, y ) 2

2

3 ((9 x  2) 4(9 y  2) ) 3  ( (9 x491)  e  e 4 4

2



(9 y 1) ) 10

1  ( (9 x 7)  e 4 2

2



(9 y 3)2 ) 4

2 2 1  e  ((9 x  4)  (9 y 7) ) , 5

Figure 7, demonstrate the plot of the value of the Franke function in three different evaluation point ( x* , y * )  D  [0,1.5]  [0,1.5]   2 with respect to shape parameters in the range of [0.5,5]. The plot shows that, by using the values for the shape parameter in the interval [2,5], the approximate values of f 4 ( x* , y* ) have a same value. By selecting three different values for the shape parameter, three approximation error functions are plotted, in Figure 8. Table 1 displays the numerical results of applying these three values.

5.3 Two-dimensional linear boundary value problems Example 5: Let us consider the following linear BVP

u xx  u yy  5e x  2 y ,

x  ,

u ( x, y )  e x  2 y ,

x  .

where   [0,1]  [0,1] and  is the boundary of the domain. The exact solution of equation (8) is u  e x 2 y . The plot of the value of the approximated function by MQRBF collocation method with 225 center nodes in an evaluation point with respect to shape parameter is shown in Figure 9. The plot show that the suitable interval is [0.6,1.5]. A good value in this range is 0.9 which has a good accuracy based on the plot of error function in Figure 10. In this Figure, the error of using the proposed value is compared to those of two other values out of the range. The plots show the impact of selecting a good value for the shape parameter.

6. Conclusion In this paper a novel Algorithm for choosing an optimal value for the shape parameter is proposed. This Algorithm works well for approximation of a function in one, two, and more dimensions and also boundary value problems. Despite of other strategies to select a value for the shape parameter, the aim of the proposed Algorithm is not introducing the optimal value, but the Algorithm suggests a range of good values for the shape parameter. Especially when the exact function is unknown, the Algorithm introduces the range of the best values for the shape parameters without any information about error function. It should be noted that a range of shape parameter is obtained for approximating a function on a set of predetermined constant number of central nodes, while the optimal range for a different number of central nodes may vary. The comparison of the results of applying the proposed Algorithm and Rippa’s Algorithm illustrated that the proposed interval includes optimum value obtained by Rippa’s Algorithm. So the new Algorithm works well as Rippa’s Algorithm. This indicates that the proposed Algorithm can be reliable to select a range of good values of the shape parameters.

References

[1] E.J. Kansa, Multiquadrics-A scattered data approximation scheme with applications to computational fluid dynamics I. Surface approximations and partial derivative estimates, Comput. Math. Appl. 19 (1990), pp. 127–145. [2] E.J. Kansa, Multiquadrics- A scattered data approximation scheme with applications to computational fluid dynamics II. Solution to parabolic, hyperbolic and elliptic partial differential equations, Comput. Math. Appl. 19 (1990), pp. 147–161.

(8)

[3] N. Dyn, W.A. Light and E.W. Cheney, Interpolation by piecewise-linear radial basis functions, J. Approx. Theory, 59 (2) (1989), pp. 202-213. [4] R.L. Hardy, Multiquadratic equations for topography and other irregular surfaces. J. Geophys. Res. 176 (1971), pp. 1905–15. [5] G.E. Fasshauer, Solving differential equations with radial basis functions: multilevel methods and smoothing, Adv. Comput. Math. 11(1999), pp. 139-159. [6] H. Wendland, Guassian interpolation revisited, Trends in Approximation Theory, K. Kopotun, T. Lyche, M. Neamtu (eds.), Vanderbilt University Press, 2001, 417-426. [7] M. D. Buhman, Radial Basis Functions, Acta. Numer. 9 (2000), pp. 1-38. [8] M. Dehghan and A. Shokri, Numerical solution of the nonlinear Klein–Gordon equation using radial basis functions. J. Comput. Appl. Math. 230 (2009), 400–410.

[9] Q. Shen, A meshless method of line for the numerical solution of KdV equation using radial basis functions, Eng. Anal. Bound. Elem. 33(2009), pp. 1171-1180. [10] S. Rippa, An algorithm for selecting a good value for the parameter c in radial basis function interpolation. Adv. Comput. Math. 11(1999), pp. 193–210. [11] Z. Wu, R. Schaback, Local error estimates for radial basis function interpolation of scattered data, IMA J. Numer. Anal. 13(1993), pp. 13-27. [12] R. Franke, Scattered data interpolation tests of some methods. Math. Comp. 38 (1982), pp. 181–200. [13] G.E. Fasshauer, Newton iteration with multiquadratics for the solution of nonlinear pdes, Comput. Math. Appl. 43 (2002), pp. 4234–438. [14] M. Scheuerer, An alternative procedure for selecting a good value for the parameter c in RBFinterpolation, Adv. Comput. Math. 34 (2011), pp. 105–126. [15] C. M. C. Roque and A. J. M. Ferreira, Numerical experiments on optimal shape parameters for radial basis functions, Numer. methods for partial differential equations 26 (3) (2010), pp. 675-689. [16] V. Bayona, M. Miguel, and M. Kindelan, Optimal constant shape parameter for multiquadric based RBF-FD method, J. Comput. Phys. 230 (2011), pp. 7384-7399. [17] G.E. Fasshauer, Meshfree Approximation Methods with MATLAB. World Scientific Co, Pte. Ltd., Singapore, 2007. [18] C.H. Tsai, J. Kolibal and M. Li, The golden section search algorithm for finding a good shape parameter for meshless collocation methods, Eng. Anal. Bound. Elem. 34 (2010), pp. 738–746. [19] S.A. Sarra, Adaptive radial basis function method for time dependent partial differential equations. Appl. Numer. Math. 54 (2005), pp. 79–94. [20] M. Gherlone, L. Iurlaro and M. Di Sciuva, A novel algorithm for shape parameter selection in radial basis functions collocation method, Compos. Struct. 94 (2012), pp. 453–461. [21] L. T. Luh, The shape parameter in the Gaussian function, Comput. Math. Appl. 63(2012), pp. 687– 694.

1.5731 1.5731 1.573 1.573 1.5729 1.5729 1.5728 0

1

2

3

4

5

6

2

0.0532

2

0.0532

1.9999

0.0532

1.9999

0.0532

1.9998

0.0532

1.9998

0.0532

1.9997

0.0531

1.9997

0.0531

1.9996 0

1

2

3

4

5

6

0.0531 0

1

2

3

4

5

6

Figure 1: Plot of f1* ( x*;  ) versus the shape parameter  with x*  0.5 , x*  0 , and x*  2.5 (respectively left to right) using 200 center nodes (solid), and f1 ( x * ) (dashed)

10

0

shape parameter = 1 shape parameter = 0.5498 shape parameter = 4

10

10

10

10

-5

-10

-15

-20

-10

0

10

20

30

40

Figure 2: The error function for three values of the shape parameters  =1,  =0.5498, and  =4 (Example 1)

-1.17

16

-1.175

-1.18

-1.185

-1.19

-1.195 0

1

2

3

4

5

6

x 10

-3

1.206

14

1.204

12

1.202

10

1.2

8

1.198

6

1.196

4

1.194

2

1.192

0

1.19

-2 0

1

2

3

4

5

6

1.188 0

1

2

3

4

5

6

Figure 3: Plot of f 2* ( x* ;  ) versus the shape parameter  with x*  0.5 , x*  0 , and x*  0.5 (respectively left to right) using 60 center nodes (solid), and f 2 ( x* ) (dotted)

10

10

10

10

Example 2

0

-5

-10

shape parameter = 1.5 shape parameter = 2.7931 shape parameter = 1

-15

-2

-1

0

1

2

/epsilon

Figure 4: The error function with shape parameters  =1.5,  =2.7931, and  =1 (example 2)

1.984

1.984

1.984

1.9835

1.9835

1.9835

1.983

1.983

1.983

1.9825

1.9825

1.9825

1.982

1.982

1.982

1.9815

1.9815

1.9815

1.981

1.981

1.981

1.9805

1.9805

1.9805

1.98 0

1

2

3

4

5

1.98 0

1

2

3

4

5

1.98 0

1.9811

1.9811

1.9811

1.9811

1.9811

1.9811

1.9811

1.9811

1.9811

1.9811

1.9811

1.9811

1.9811

1.9811

1.9811

1.9811

1.9811

1.9811

1.9811

1

2

3

4

5

1.9811

1

2

3

4

5

1.9811

1

1

2

2

3

3

4

4

5

5

Figure 5: Plot of f3* ( x* ;  ) versus the shape parameter  in [0.2,5] (top), and [1,5] (bottom) with x*  0.1 , x*  0.5 , and x*  0.9 (respectively left to right) using 60 center nodes (solid), and f 3 ( x* ) (dotted)

Example 3

shape parameter = 4 shape parameter = 3.8663 shape parameter = 0.5

-2

10

-4

10

-6

10

-8

10

-10

10

0

0.2

0.4

0.6

0.8

1

/epsilon

Figure 6: The error function with shape parameters  =4,  =3.8663, and  =0.5 (example 3)

0.2

0.38

0.05

0.36 0.15

0 0.34 0.32

0.1

-0.05

0.3 0.05

-0.1

0.28 0.26

0

-0.15 0.24

-0.05 0

1

2

3

4

5

6

0.22 0

1

2

3

4

5

-0.2 0

6

1

2

3

4

5

6

Figure 7: Plot of f 4* ( x* , y* ;  ) versus the shape parameter  in [0.5,5] with three different ( x* , y * ) using 400 center nodes (solid), and f 4 ( x* , y * ) (dotted)

-4

-4

x 10

x 10

8

8

0.2

6

6

0.15

4

0.1

2

0.05

0 2

0 2

4 2 0 2

1

1 0 0

0.5

1

1.5

0 0

1 0.5

1

1.5

0 0

0.5

1

1.5

Figure 8: The error function with shape parameters  =2,  =2.2925, and  =1.2 (left to right) (example 4)

3.45 3.44 3.43 3.42 3.41 3.4 3.39 3.38 0

1

2

3

4

5

Figure 9: Plot of u * ( x* , y* ;  ) versus the shape parameter  , in [0.2,5] using 225 center nodes (solid), and u ( x* , y * ) (dotted)

x 10

-4

0.1

4

0.03

2

0.05

0 1

0 1

0.02 0.01

1

0.5

0.5 0 0

1

0.5

0.5 0 0

0 1

1

0.5

0.5 0 0

Figure 10: The error function with shape parameters  =0.9,  =0.2 and  =4 (left to right) (example 5)

Table 1: The results of applying different values of the shape parameter in MQ-RBF to approximate the functions Good interval

Example 1

Example 2

Example 3

Example 4

[0.5,2]

[1.5,5]

[4,5]

[2,5]

f1  f 4

Shape parameter

Max error

RMS error

New Algorithm

1

3.85e-8

1.14e-8

Rippa’s Algorithm

0.5468

5.28e-8

1.44e-8

Out of interval

4

4.38e-5

1.03e-5

New Algorithm Rippa’s Algorithm Out of interval New Algorithm Rippa’s Algorithm Out of interval New Algorithm Rippa’s Algorithm Out of interval

2

3.21e-6

1.60e-6

2.7931

2.44e-6

1.06e-6

1 4

2.30e-3

9.47e-4

1.08e-8

7.31e-9

3.8663

2.25e-7

8.50e-8

0.5 2 2.2925 1.2

1.93e-4

8.90e-5

6.83e-4 7.24e-4 1.91e-1

6.02e-4 5.51e-4 2.63e-1