MATH 590: Meshfree Methods

Transcription

1 MATH 590: Meshfree Methods Chapter 33: Adaptive Iteration Greg Fasshauer Department of Applied Mathematics Illinois Institute of Technology Fall 2010 MATH 590 Chapter 33 1

2 Outline 1 A Greedy Adaptive Algorithm 2 The Faul-Powell Algorithm fasshauer@iit.edu MATH 590 Chapter 33 2

3 The two adaptive algorithms discussed in this chapter both yield an approximate solution to the RBF interpolation problem. MATH 590 Chapter 33 3

4 The two adaptive algorithms discussed in this chapter both yield an approximate solution to the RBF interpolation problem. The algorithms have some similarity with some of the omitted material from Chapters 21, 31 and 32. MATH 590 Chapter 33 3

5 The two adaptive algorithms discussed in this chapter both yield an approximate solution to the RBF interpolation problem. The algorithms have some similarity with some of the omitted material from Chapters 21, 31 and 32. The contents of this chapter are based mostly on the papers [Faul and Powell (1999), Faul and Powell (2000), Schaback and Wendland (2000a), Schaback and Wendland (2000b)] and the book [Wendland (2005a)]. MATH 590 Chapter 33 3

6 The two adaptive algorithms discussed in this chapter both yield an approximate solution to the RBF interpolation problem. The algorithms have some similarity with some of the omitted material from Chapters 21, 31 and 32. The contents of this chapter are based mostly on the papers [Faul and Powell (1999), Faul and Powell (2000), Schaback and Wendland (2000a), Schaback and Wendland (2000b)] and the book [Wendland (2005a)]. As always, we concentrate on systems for strictly positive definite kernels (variations for strictly conditionally positive definite kernels also exist). MATH 590 Chapter 33 3

7 Outline A Greedy Adaptive Algorithm 1 A Greedy Adaptive Algorithm 2 The Faul-Powell Algorithm fasshauer@iit.edu MATH 590 Chapter 33 4

8 One of the central ingredients is the use of the native space inner product discussed in Chapter 13. MATH 590 Chapter 33 5

9 One of the central ingredients is the use of the native space inner product discussed in Chapter 13. As always, we assume that our data sites are X = {x 1,..., x N }. fasshauer@iit.edu MATH 590 Chapter 33 5

10 One of the central ingredients is the use of the native space inner product discussed in Chapter 13. As always, we assume that our data sites are X = {x 1,..., x N }. We also consider a second set Y X. fasshauer@iit.edu MATH 590 Chapter 33 5

11 One of the central ingredients is the use of the native space inner product discussed in Chapter 13. As always, we assume that our data sites are X = {x 1,..., x N }. We also consider a second set Y X. Let P Y f be the interpolant to f on Y X. fasshauer@iit.edu MATH 590 Chapter 33 5

12 One of the central ingredients is the use of the native space inner product discussed in Chapter 13. As always, we assume that our data sites are X = {x 1,..., x N }. We also consider a second set Y X. Let P Y f be the interpolant to f on Y X. Then the first orthogonality lemma from Chapter 18 (with g = f ) yields f P Y f, P Y f NK (Ω) = 0. fasshauer@iit.edu MATH 590 Chapter 33 5

13 One of the central ingredients is the use of the native space inner product discussed in Chapter 13. As always, we assume that our data sites are X = {x 1,..., x N }. We also consider a second set Y X. Let P Y f be the interpolant to f on Y X. Then the first orthogonality lemma from Chapter 18 (with g = f ) yields f P Y f, P Y f NK (Ω) = 0. This leads to the energy split (see Chapter 18) f 2 N K (Ω) = f PY f 2 N K (Ω) + PY f 2 N K (Ω). fasshauer@iit.edu MATH 590 Chapter 33 5

14 We now consider an iteration on residuals. MATH 590 Chapter 33 6

15 We now consider an iteration on residuals. We pretend to start with our desired interpolant r 0 = Pf X set X. on the entire fasshauer@iit.edu MATH 590 Chapter 33 6

16 We now consider an iteration on residuals. We pretend to start with our desired interpolant r 0 = Pf X set X. on the entire We also pick an appropriate sequence of sets Y k X, k = 0, 1,... (we will discuss some possible heuristics for choosing these sets later). fasshauer@iit.edu MATH 590 Chapter 33 6

17 We now consider an iteration on residuals. We pretend to start with our desired interpolant r 0 = Pf X set X. on the entire We also pick an appropriate sequence of sets Y k X, k = 0, 1,... (we will discuss some possible heuristics for choosing these sets later). Then we iteratively define the residual functions r k+1 = r k P Y k r k, k = 0, 1,.... (1) fasshauer@iit.edu MATH 590 Chapter 33 6

18 We now consider an iteration on residuals. We pretend to start with our desired interpolant r 0 = Pf X set X. on the entire We also pick an appropriate sequence of sets Y k X, k = 0, 1,... (we will discuss some possible heuristics for choosing these sets later). Then we iteratively define the residual functions r k+1 = r k P Y k r k, k = 0, 1,.... (1) Remark In the actual algorithm below we will only deal with discrete vectors. Thus the vector r 0 will be given by the data values (since P f is supposed to interpolate f on X ). fasshauer@iit.edu MATH 590 Chapter 33 6

19 Now, the energy splitting identity with f = r k gives us r k 2 N K (Ω) = r k P Y k r k 2 N K (Ω) + PY k r k 2 N K (Ω) (2) fasshauer@iit.edu MATH 590 Chapter 33 7

20 Now, the energy splitting identity with f = r k gives us r k 2 N K (Ω) = r k P Y k r k 2 N K (Ω) + PY k r k 2 N K (Ω) (2) or, using the iteration formula (1), r k 2 N K (Ω) = r k+1 2 N K (Ω) + r k r k+1 2 N K (Ω). (3) fasshauer@iit.edu MATH 590 Chapter 33 7

21 We have the following telescoping sum for the partial sums of the norm of the residual updates P Y k r k : M k=0 P Y k r k 2 N K (Ω) (1) = M r k r k+1 2 N K (Ω) k=0 fasshauer@iit.edu MATH 590 Chapter 33 8

22 We have the following telescoping sum for the partial sums of the norm of the residual updates P Y k r k : M k=0 P Y k r k 2 N K (Ω) (1) = (3) = M r k r k+1 2 N K (Ω) k=0 M k=0 { } r k 2 N K (Ω) r k+1 2 N K (Ω) fasshauer@iit.edu MATH 590 Chapter 33 8

23 We have the following telescoping sum for the partial sums of the norm of the residual updates P Y k r k : M k=0 P Y k r k 2 N K (Ω) (1) = (3) = M r k r k+1 2 N K (Ω) k=0 M k=0 { } r k 2 N K (Ω) r k+1 2 N K (Ω) = r 0 2 N K (Ω) r M+1 2 N K (Ω) r 0 2 N K (Ω). fasshauer@iit.edu MATH 590 Chapter 33 8

24 We have the following telescoping sum for the partial sums of the norm of the residual updates P Y k r k : M k=0 P Y k r k 2 N K (Ω) (1) = (3) = M r k r k+1 2 N K (Ω) k=0 M k=0 { } r k 2 N K (Ω) r k+1 2 N K (Ω) = r 0 2 N K (Ω) r M+1 2 N K (Ω) r 0 2 N K (Ω). Remark This estimate shows that the sequence of partial sums is monotone increasing and bounded, and therefore convergent even for a poor choice of the sets Y k. fasshauer@iit.edu MATH 590 Chapter 33 8

25 If we can show that the residuals r k converge to zero, fasshauer@iit.edu MATH 590 Chapter 33 9

26 If we can show that the residuals r k converge to zero, then we would have that the iteratively computed approximation u M+1 = M k=0 P Y k r k fasshauer@iit.edu MATH 590 Chapter 33 9

27 If we can show that the residuals r k converge to zero, then we would have that the iteratively computed approximation u M+1 = M k=0 P Y k r k = M (r k r k+1 ) k=0 fasshauer@iit.edu MATH 590 Chapter 33 9

28 If we can show that the residuals r k converge to zero, then we would have that the iteratively computed approximation u M+1 = M k=0 P Y k r k = M (r k r k+1 ) = r 0 r M+1 (4) k=0 converges to the original interpolant r 0 = P X f. fasshauer@iit.edu MATH 590 Chapter 33 9

29 If we can show that the residuals r k converge to zero, then we would have that the iteratively computed approximation u M+1 = M k=0 P Y k r k = M (r k r k+1 ) = r 0 r M+1 (4) k=0 converges to the original interpolant r 0 = P X f. Remark The omitted chapters contain iterative methods by which we approximate the interpolant by iterating an approximation method on the full data set. fasshauer@iit.edu MATH 590 Chapter 33 9

30 If we can show that the residuals r k converge to zero, then we would have that the iteratively computed approximation u M+1 = M k=0 P Y k r k = M (r k r k+1 ) = r 0 r M+1 (4) k=0 converges to the original interpolant r 0 = P X f. Remark The omitted chapters contain iterative methods by which we approximate the interpolant by iterating an approximation method on the full data set. Here we are approximating the interpolant by iterating an interpolation method on nested (increasing) adaptively chosen subsets of the data. fasshauer@iit.edu MATH 590 Chapter 33 9

31 Remark The present method also has some similarities with the (omitted) multilevel algorithms of Chapter 32. MATH 590 Chapter 33 10

32 Remark The present method also has some similarities with the (omitted) multilevel algorithms of Chapter 32. However, here: we compute the interpolant Pf X single kernel K on the set X based on a fasshauer@iit.edu MATH 590 Chapter 33 10

33 Remark The present method also has some similarities with the (omitted) multilevel algorithms of Chapter 32. However, here: we compute the interpolant Pf X single kernel K on the set X based on a Chapter 32: the final interpolant is given as the result of using the spaces M k=1 N K k (Ω), where K k is an appropriately scaled version of the kernel K. fasshauer@iit.edu MATH 590 Chapter 33 10

34 Remark The present method also has some similarities with the (omitted) multilevel algorithms of Chapter 32. However, here: we compute the interpolant Pf X single kernel K on the set X based on a Chapter 32: the final interpolant is given as the result of using the spaces M k=1 N K k (Ω), where K k is an appropriately scaled version of the kernel K. Moreover, the goal in Chapter 32 is to approximate f, not P f. fasshauer@iit.edu MATH 590 Chapter 33 10

35 To prove convergence of the residual iteration, we assume that we can find sets of points Y k such that at step k at least some fixed percentage of the energy of the residual is picked up by its interpolant, i.e., fasshauer@iit.edu MATH 590 Chapter 33 11

36 To prove convergence of the residual iteration, we assume that we can find sets of points Y k such that at step k at least some fixed percentage of the energy of the residual is picked up by its interpolant, i.e., with some fixed γ (0, 1]. P Y k r k 2 N K (Ω) γ r k 2 N K (Ω) (5) fasshauer@iit.edu MATH 590 Chapter 33 11

37 To prove convergence of the residual iteration, we assume that we can find sets of points Y k such that at step k at least some fixed percentage of the energy of the residual is picked up by its interpolant, i.e., with some fixed γ (0, 1]. Then (3) and the iteration formula (1) imply P Y k r k 2 N K (Ω) γ r k 2 N K (Ω) (5) r k+1 2 N K (Ω) = r k 2 N K (Ω) PY k r k 2 N K (Ω), fasshauer@iit.edu MATH 590 Chapter 33 11

38 To prove convergence of the residual iteration, we assume that we can find sets of points Y k such that at step k at least some fixed percentage of the energy of the residual is picked up by its interpolant, i.e., with some fixed γ (0, 1]. Then (3) and the iteration formula (1) imply P Y k r k 2 N K (Ω) γ r k 2 N K (Ω) (5) r k+1 2 N K (Ω) = r k 2 N K (Ω) PY k r k 2 N K (Ω), and therefore r k+1 2 N K (Ω) r k 2 N K (Ω) γ r k 2 N K (Ω) fasshauer@iit.edu MATH 590 Chapter 33 11

39 To prove convergence of the residual iteration, we assume that we can find sets of points Y k such that at step k at least some fixed percentage of the energy of the residual is picked up by its interpolant, i.e., with some fixed γ (0, 1]. Then (3) and the iteration formula (1) imply P Y k r k 2 N K (Ω) γ r k 2 N K (Ω) (5) r k+1 2 N K (Ω) = r k 2 N K (Ω) PY k r k 2 N K (Ω), and therefore r k+1 2 N K (Ω) r k 2 N K (Ω) γ r k 2 N K (Ω) = (1 γ) r k 2 N K (Ω). fasshauer@iit.edu MATH 590 Chapter 33 11

40 Applying the bound A Greedy Adaptive Algorithm recursively yields r k+1 2 N K (Ω) (1 γ) r k 2 N K (Ω) Theorem If the choice of sets Y k satisfies P Y k r k 2 N K (Ω) γ r k 2 N K (Ω), then the residual iteration (see (4)) u M = M 1 k=0 P Y k r k = r 0 r M, r k+1 = r k P Y k r k, k = 0, 1,... converges linearly in the native space norm. fasshauer@iit.edu MATH 590 Chapter 33 12

41 Applying the bound A Greedy Adaptive Algorithm recursively yields r k+1 2 N K (Ω) (1 γ) r k 2 N K (Ω) Theorem If the choice of sets Y k satisfies P Y k r k 2 N K (Ω) γ r k 2 N K (Ω), then the residual iteration (see (4)) u M = M 1 k=0 P Y k r k = r 0 r M, r k+1 = r k P Y k r k, k = 0, 1,... converges linearly in the native space norm. After M steps of iterative refinement there is an error bound P X f u M 2 N K (Ω) = r 0 u M 2 N K (Ω) = r M 2 N K (Ω) (1 γ)m r 0 2 N K (Ω). fasshauer@iit.edu MATH 590 Chapter 33 12

42 Remark This theorem has various limitations: MATH 590 Chapter 33 13

43 Remark This theorem has various limitations: The norm involves the kernel K which makes it difficult to find sets Y k that satisfy (5). fasshauer@iit.edu MATH 590 Chapter 33 13

44 Remark This theorem has various limitations: The norm involves the kernel K which makes it difficult to find sets Y k that satisfy (5). The native space norm of the initial residual r 0 is not known. fasshauer@iit.edu MATH 590 Chapter 33 13

45 Remark This theorem has various limitations: The norm involves the kernel K which makes it difficult to find sets Y k that satisfy (5). The native space norm of the initial residual r 0 is not known. A way around these problems is to use an equivalent discrete norm on the set X. fasshauer@iit.edu MATH 590 Chapter 33 13

46 Schaback and Wendland establish an estimate of the form r 0 u M 2 X C c (1 δ c2 C 2 ) M/2 r 0 2 X, where c and C are constants denoting the norm equivalence, i.e., c u X u NK (Ω) C u X for any u N K (Ω), and where δ is a constant analogous to γ (but based on use of the discrete norm X in (5)). fasshauer@iit.edu MATH 590 Chapter 33 14

47 Schaback and Wendland establish an estimate of the form r 0 u M 2 X C c (1 δ c2 C 2 ) M/2 r 0 2 X, where c and C are constants denoting the norm equivalence, i.e., c u X u NK (Ω) C u X for any u N K (Ω), and where δ is a constant analogous to γ (but based on use of the discrete norm X in (5)). In fact, any discrete l p norm on X can be used. fasshauer@iit.edu MATH 590 Chapter 33 14

48 Schaback and Wendland establish an estimate of the form r 0 u M 2 X C c (1 δ c2 C 2 ) M/2 r 0 2 X, where c and C are constants denoting the norm equivalence, i.e., c u X u NK (Ω) C u X for any u N K (Ω), and where δ is a constant analogous to γ (but based on use of the discrete norm X in (5)). In fact, any discrete l p norm on X can be used. In the implementation below we will use the maximum norm. fasshauer@iit.edu MATH 590 Chapter 33 14

49 In [Schaback and Wendland (2000b)] a basic version of this algorithm where the sets Y k consist of a single point is described and tested. fasshauer@iit.edu MATH 590 Chapter 33 15

50 In [Schaback and Wendland (2000b)] a basic version of this algorithm where the sets Y k consist of a single point is described and tested. The resulting approximation yields the best M-term approximation to the interpolant. fasshauer@iit.edu MATH 590 Chapter 33 15

51 In [Schaback and Wendland (2000b)] a basic version of this algorithm where the sets Y k consist of a single point is described and tested. The resulting approximation yields the best M-term approximation to the interpolant. Remark This idea is related to the concepts of greedy approximation algorithms (see, e.g., [Temlyakov (1998)]) and sparse approximation (see, e.g., [Girosi (1998)]). fasshauer@iit.edu MATH 590 Chapter 33 15

52 If the set Y k consists of only a single point y k, then the partial interpolant P Y k r k is particularly simple: fasshauer@iit.edu MATH 590 Chapter 33 16

53 If the set Y k consists of only a single point y k, then the partial interpolant P Y k r k is particularly simple: with P Y k r k = βk (, y k ) β = r k(y k ) K (y k, y k ) fasshauer@iit.edu MATH 590 Chapter 33 16

54 If the set Y k consists of only a single point y k, then the partial interpolant P Y k r k is particularly simple: with P Y k r k = βk (, y k ) β = r k(y k ) K (y k, y k ) This follows immediately from the usual RBF expansion (which consists of only one term here) and the interpolation condition P Y k r k (y k ) = r k (y k ). fasshauer@iit.edu MATH 590 Chapter 33 16

55 The point y k is picked to be the point in X where the residual is largest, i.e., r k (y k ) = r k. fasshauer@iit.edu MATH 590 Chapter 33 17

56 The point y k is picked to be the point in X where the residual is largest, i.e., r k (y k ) = r k. This choice of set Y k certainly satisfies the constraint (5): P Y k r k 2 N K (Ω) = βk (, y k ) 2 N K (Ω) fasshauer@iit.edu MATH 590 Chapter 33 17

57 The point y k is picked to be the point in X where the residual is largest, i.e., r k (y k ) = r k. This choice of set Y k certainly satisfies the constraint (5): P Y k r k 2 N K (Ω) = βk (, y k ) 2 N K (Ω) = r k (y k ) K (y k, y k ) K (, y k) 2 N K (Ω) fasshauer@iit.edu MATH 590 Chapter 33 17

58 The point y k is picked to be the point in X where the residual is largest, i.e., r k (y k ) = r k. This choice of set Y k certainly satisfies the constraint (5): P Y k r k 2 N K (Ω) = βk (, y k ) 2 N K (Ω) = r k (y k ) K (y k, y k ) K (, y k) 2 N K (Ω) γ r k 2 N K (Ω), 0 < γ 1. fasshauer@iit.edu MATH 590 Chapter 33 17

59 The point y k is picked to be the point in X where the residual is largest, i.e., r k (y k ) = r k. This choice of set Y k certainly satisfies the constraint (5): P Y k r k 2 N K (Ω) = βk (, y k ) 2 N K (Ω) = r k (y k ) K (y k, y k ) K (, y k) 2 N K (Ω) γ r k 2 N K (Ω), 0 < γ 1. Here we require K (, y k ) K (y k, y k ) which is certainly true for positive definite translation invariant kernels (cf. Chapter 3). However, in general we only know that K (x, y) 2 K (x, x)k (y, y) (see [Berlinet and Thomas-Agnan (2004)]). The interpolation problem is (approximately) solved without having to invert any linear systems. fasshauer@iit.edu MATH 590 Chapter 33 17

60 Algorithm (Greedy one-point) Input data locations X, associated values f of f, tolerance tol > 0 Set initial residual r 0 = P X f X = f, initialize u 0 = 0, e =, k = 0 Choose starting point y k X While e > tol do Set β = r k(y k ) K (y k, y k ) For 1 i N do r k+1 (x i ) = r k (x i ) βk (x i, y k ) u k+1 (x i ) = u k (x i ) + βk (x i, y k ) end Find e = max r k+1 and the point y k+1 where it occurs X Increment k = k + 1 end fasshauer@iit.edu MATH 590 Chapter 33 18

61 Remark It is important to realize that in our MATLAB implementation we never actually compute the initial residual r 0 = P X f. fasshauer@iit.edu MATH 590 Chapter 33 19

62 Remark It is important to realize that in our MATLAB implementation we never actually compute the initial residual r 0 = P X f. All we require are the values of r 0 on the grid X of data sites. fasshauer@iit.edu MATH 590 Chapter 33 19

63 Remark It is important to realize that in our MATLAB implementation we never actually compute the initial residual r 0 = P X f. All we require are the values of r 0 on the grid X of data sites. However, since Pf X X = f X the values r 0 (x i ) are given by the interpolation data f (x i ) (see line 5 of the code). fasshauer@iit.edu MATH 590 Chapter 33 19

64 Remark It is important to realize that in our MATLAB implementation we never actually compute the initial residual r 0 = P X f. All we require are the values of r 0 on the grid X of data sites. However, since Pf X X = f X the values r 0 (x i ) are given by the interpolation data f (x i ) (see line 5 of the code). Moreover, since the sets Y k are subsets of X the value r k (y k ) required to determine β is actually one of the current residual values (see line 10 of the code). fasshauer@iit.edu MATH 590 Chapter 33 19

65 Remark We use DistanceMatrix together with rbf to compute both K (y k, y k ) (on lines 9 and 10) and K (x i, y k ) needed for the updates of the residual r k+1 and the approximation u k+1 on lines fasshauer@iit.edu MATH 590 Chapter 33 20

66 Remark We use DistanceMatrix together with rbf to compute both K (y k, y k ) (on lines 9 and 10) and K (x i, y k ) needed for the updates of the residual r k+1 and the approximation u k+1 on lines Note that the matrices DM_data, IM, DM_res, RM, DM_eval, EM are only column vectors since only one center, y k, is involved. fasshauer@iit.edu MATH 590 Chapter 33 20

67 Remark The algorithm demands that we compute the residuals r k on the data sites. fasshauer@iit.edu MATH 590 Chapter 33 21

68 Remark The algorithm demands that we compute the residuals r k on the data sites. The partial approximants u k to the interpolant can be evaluated anywhere. fasshauer@iit.edu MATH 590 Chapter 33 21

69 Remark The algorithm demands that we compute the residuals r k on the data sites. The partial approximants u k to the interpolant can be evaluated anywhere. If we do this also at the data sites, then we are required to use a plotting routine that differs from our usual one (such as trisurf built on a triangulation of the data sites obtained with the help of delaunayn). fasshauer@iit.edu MATH 590 Chapter 33 21

70 Remark The algorithm demands that we compute the residuals r k on the data sites. The partial approximants u k to the interpolant can be evaluated anywhere. If we do this also at the data sites, then we are required to use a plotting routine that differs from our usual one (such as trisurf built on a triangulation of the data sites obtained with the help of delaunayn). We instead follow the same procedure as in all of our other programs, i.e., to evaluate u k on a grid of equally spaced points. This has been implemented on lines of the program. fasshauer@iit.edu MATH 590 Chapter 33 21

71 Remark The algorithm demands that we compute the residuals r k on the data sites. The partial approximants u k to the interpolant can be evaluated anywhere. If we do this also at the data sites, then we are required to use a plotting routine that differs from our usual one (such as trisurf built on a triangulation of the data sites obtained with the help of delaunayn). We instead follow the same procedure as in all of our other programs, i.e., to evaluate u k on a grid of equally spaced points. This has been implemented on lines of the program. Note that the updating procedure has been vectorized in MATLAB allowing us to avoid the for-loop over i in the algorithm. fasshauer@iit.edu MATH 590 Chapter 33 21

72 Program (RBFGreedyOnePoint2D.m) 1 rbf exp(-(e*r).^2); ep = 5.5; 2 N = 16641; dsites = CreatePoints(N,2, h ); 3 neval = 40; epoints = CreatePoints(neval^2,2, u ); 4 tol = 1e-5; kmax = 1000; 5 res = testfunctionsd(dsites); u = 0; 6 k = 1; maxres(k) = ; 7 ykidx = (N+1)/2; yk(k,:) = dsites(ykidx,:); 8 while (maxres(k) > tol && k < kmax) 9 DM_data = DistanceMatrix(yk(k,:),yk(k,:)); 10 IM = rbf(ep,dm_data); beta = res(ykidx)/im; 11 DM_res = DistanceMatrix(dsites,yk(k,:)); 12 RM = rbf(ep,dm_res); 13 DM_eval = DistanceMatrix(epoints,yk(k,:)); 14 EM = rbf(ep,dm_eval); 15 res = res - beta*rm; u = u + beta*em; 16 [maxres(k+1), ykidx] = max(abs(res)); 17 yk(k+1,:) = dsites(ykidx,:); k = k + 1; 18 end 19 exact = testfunctionsd(epoints); 20 rms_err = norm(u-exact)/neval fasshauer@iit.edu MATH 590 Chapter 33 22

73 To illustrate the greedy one-point algorithm we perform two experiments. Both tests use data obtained by sampling Franke s function at Halton points in [0, 1] 2. Test 1 is based on Gaussians, Test 2 uses inverse multiquadrics. For both tests we use the same shape parameter ε = 5.5. fasshauer@iit.edu MATH 590 Chapter 33 23

74 Figure: 1000 selected points and residual for greedy one point algorithm with Gaussian RBFs and N = data points. fasshauer@iit.edu MATH 590 Chapter 33 24

75 Figure: Fits of Franke s function for greedy one point algorithm with Gaussian RBFs and N = data points. Top left to bottom right: 1 point, 2 points, 4 points, final fit with 1000 points. fasshauer@iit.edu MATH 590 Chapter 33 25

76 Remark In order to obtain our approximate interpolants we used a tolerance of 10 5 along with an additional upper limit of kmax=1000 on the number of iterations. fasshauer@iit.edu MATH 590 Chapter 33 26

77 Remark In order to obtain our approximate interpolants we used a tolerance of 10 5 along with an additional upper limit of kmax=1000 on the number of iterations. For both tests the algorithm uses up all 1000 iterations. fasshauer@iit.edu MATH 590 Chapter 33 26

78 Remark In order to obtain our approximate interpolants we used a tolerance of 10 5 along with an additional upper limit of kmax=1000 on the number of iterations. For both tests the algorithm uses up all 1000 iterations. The final maximum residual is maxres = for Gaussians, and maxres = for inverse MQs. fasshauer@iit.edu MATH 590 Chapter 33 26

79 Remark In order to obtain our approximate interpolants we used a tolerance of 10 5 along with an additional upper limit of kmax=1000 on the number of iterations. For both tests the algorithm uses up all 1000 iterations. The final maximum residual is maxres = for Gaussians, and maxres = for inverse MQs. In both cases there occurred several multiple point selections. fasshauer@iit.edu MATH 590 Chapter 33 26

80 Remark In order to obtain our approximate interpolants we used a tolerance of 10 5 along with an additional upper limit of kmax=1000 on the number of iterations. For both tests the algorithm uses up all 1000 iterations. The final maximum residual is maxres = for Gaussians, and maxres = for inverse MQs. In both cases there occurred several multiple point selections. Contrary to interpolation problems based on the solution of a linear system, multiple point selections do not pose a problem here. fasshauer@iit.edu MATH 590 Chapter 33 26

81 Figure: 1000 selected points and residual for greedy one point algorithm with IMQ RBFs and N = data points. fasshauer@iit.edu MATH 590 Chapter 33 27

82 Figure: Fits of Franke s function for greedy one point algorithm with IMQ RBFs and N = data points. Top left to bottom right: 1 point, 2 points, 4 points, final fit with 1000 points. fasshauer@iit.edu MATH 590 Chapter 33 28

83 Remark We note that the inverse multiquadrics have a more global influence than the Gaussians (for the same shape parameter). fasshauer@iit.edu MATH 590 Chapter 33 29

84 Remark We note that the inverse multiquadrics have a more global influence than the Gaussians (for the same shape parameter). This effect is clearly evident in the first few approximations to the interpolants in the figures. fasshauer@iit.edu MATH 590 Chapter 33 29

85 Remark We note that the inverse multiquadrics have a more global influence than the Gaussians (for the same shape parameter). This effect is clearly evident in the first few approximations to the interpolants in the figures. From the last figure we see that the greedy algorithm enforces interpolation of the data only on the most recent set Y k (i.e., for the one-point algorithm studied here only at a single point). fasshauer@iit.edu MATH 590 Chapter 33 29

86 Remark We note that the inverse multiquadrics have a more global influence than the Gaussians (for the same shape parameter). This effect is clearly evident in the first few approximations to the interpolants in the figures. From the last figure we see that the greedy algorithm enforces interpolation of the data only on the most recent set Y k (i.e., for the one-point algorithm studied here only at a single point). If one wants to maintain the interpolation achieved in previous iterations, then the sets Y k should be nested. fasshauer@iit.edu MATH 590 Chapter 33 29

87 Remark We note that the inverse multiquadrics have a more global influence than the Gaussians (for the same shape parameter). This effect is clearly evident in the first few approximations to the interpolants in the figures. From the last figure we see that the greedy algorithm enforces interpolation of the data only on the most recent set Y k (i.e., for the one-point algorithm studied here only at a single point). If one wants to maintain the interpolation achieved in previous iterations, then the sets Y k should be nested. This, however, would have a significant effect on the execution time of the algorithm since the matrices at each step would increase in size. fasshauer@iit.edu MATH 590 Chapter 33 29

88 Remark One advantage of this very simple algorithm is that no linear systems need to be solved. MATH 590 Chapter 33 30

89 Remark One advantage of this very simple algorithm is that no linear systems need to be solved. This allows us to approximate the interpolants for large data sets even for globally supported kernels, MATH 590 Chapter 33 30

90 Remark One advantage of this very simple algorithm is that no linear systems need to be solved. This allows us to approximate the interpolants for large data sets even for globally supported kernels, and also with small values of ε (and therefore an associated ill-conditioned interpolation matrix). MATH 590 Chapter 33 30

91 Remark One advantage of this very simple algorithm is that no linear systems need to be solved. This allows us to approximate the interpolants for large data sets even for globally supported kernels, and also with small values of ε (and therefore an associated ill-conditioned interpolation matrix). One should not expect too much in this case, however, as the results in the following figure show where we used a value of ε = 0.1 for the shape parameter. fasshauer@iit.edu MATH 590 Chapter 33 30

92 Remark One advantage of this very simple algorithm is that no linear systems need to be solved. This allows us to approximate the interpolants for large data sets even for globally supported kernels, and also with small values of ε (and therefore an associated ill-conditioned interpolation matrix). One should not expect too much in this case, however, as the results in the following figure show where we used a value of ε = 0.1 for the shape parameter. A lot of smoothing occurs so that the convergence to the RBF interpolant is very slow. fasshauer@iit.edu MATH 590 Chapter 33 30

93 Figure: 1000 selected points (only 20 of them distinct) and fit of Franke s function for greedy one point algorithm with flat Gaussian RBFs (ε = 0.1) and N = data points. fasshauer@iit.edu MATH 590 Chapter 33 31

94 Remark In the pseudo-code of the algorithm matrix-vector multiplications are not required. MATH 590 Chapter 33 32

95 Remark In the pseudo-code of the algorithm matrix-vector multiplications are not required. However, MATLAB allows for a vectorization of the for-loop which does result in two matrix-vector multiplications. fasshauer@iit.edu MATH 590 Chapter 33 32

96 Remark In the pseudo-code of the algorithm matrix-vector multiplications are not required. However, MATLAB allows for a vectorization of the for-loop which does result in two matrix-vector multiplications. For practical situations, e.g., for smooth kernels and densely distributed points in X the convergence can be rather slow. fasshauer@iit.edu MATH 590 Chapter 33 32

97 Remark In the pseudo-code of the algorithm matrix-vector multiplications are not required. However, MATLAB allows for a vectorization of the for-loop which does result in two matrix-vector multiplications. For practical situations, e.g., for smooth kernels and densely distributed points in X the convergence can be rather slow. The simple greedy algorithm described above is extended in [Schaback and Wendland (2000b)] to a version that adaptively uses kernels of varying scales. fasshauer@iit.edu MATH 590 Chapter 33 32

98 Outline The Faul-Powell Algorithm 1 A Greedy Adaptive Algorithm 2 The Faul-Powell Algorithm fasshauer@iit.edu MATH 590 Chapter 33 33

99 The Faul-Powell Algorithm Another iterative algorithm was suggested in [Faul and Powell (1999), Faul and Powell (2000)]. MATH 590 Chapter 33 34

100 The Faul-Powell Algorithm Another iterative algorithm was suggested in [Faul and Powell (1999), Faul and Powell (2000)]. From our earlier discussions we know that it is possible to express a kernel interpolant in terms of cardinal functions uj, j = 1,..., N, i.e., P f (x) = N f (x j )uj (x). j=1 fasshauer@iit.edu MATH 590 Chapter 33 34

101 The Faul-Powell Algorithm Another iterative algorithm was suggested in [Faul and Powell (1999), Faul and Powell (2000)]. From our earlier discussions we know that it is possible to express a kernel interpolant in terms of cardinal functions uj, j = 1,..., N, i.e., P f (x) = N f (x j )uj (x). j=1 The basic idea of the Faul-Powell algorithm is to use approximate cardinal functions Ψ j instead. fasshauer@iit.edu MATH 590 Chapter 33 34

102 The Faul-Powell Algorithm Another iterative algorithm was suggested in [Faul and Powell (1999), Faul and Powell (2000)]. From our earlier discussions we know that it is possible to express a kernel interpolant in terms of cardinal functions uj, j = 1,..., N, i.e., P f (x) = N f (x j )uj (x). j=1 The basic idea of the Faul-Powell algorithm is to use approximate cardinal functions Ψ j instead. Of course, this will only give an approximate value for the interpolant, and therefore an iteration on the residuals is suggested to improve the accuracy of this approximation. fasshauer@iit.edu MATH 590 Chapter 33 34

103 The Faul-Powell Algorithm The approximate cardinal functions Ψ j, j = 1,..., N, are determined as linear combinations of the basis functions K (, x l ) for the interpolant, i.e., Ψ j = l L j b jl K (, x l ), (6) where L j is an index set consisting of n (n 50) indices that are used to determine the approximate cardinal function. fasshauer@iit.edu MATH 590 Chapter 33 35

104 The Faul-Powell Algorithm The approximate cardinal functions Ψ j, j = 1,..., N, are determined as linear combinations of the basis functions K (, x l ) for the interpolant, i.e., Ψ j = l L j b jl K (, x l ), (6) where L j is an index set consisting of n (n 50) indices that are used to determine the approximate cardinal function. Example The n nearest neighbors of x j will usually do. fasshauer@iit.edu MATH 590 Chapter 33 35

105 The Faul-Powell Algorithm The approximate cardinal functions Ψ j, j = 1,..., N, are determined as linear combinations of the basis functions K (, x l ) for the interpolant, i.e., Ψ j = l L j b jl K (, x l ), (6) where L j is an index set consisting of n (n 50) indices that are used to determine the approximate cardinal function. Example The n nearest neighbors of x j will usually do. Remark The basic philosophy of this algorithm is very similar to that of the omitted fixed level iteration of Chapter 31 where approximate MLS generating functions were used as approximate cardinal functions. fasshauer@iit.edu MATH 590 Chapter 33 35

106 The Faul-Powell Algorithm The approximate cardinal functions Ψ j, j = 1,..., N, are determined as linear combinations of the basis functions K (, x l ) for the interpolant, i.e., Ψ j = l L j b jl K (, x l ), (6) where L j is an index set consisting of n (n 50) indices that are used to determine the approximate cardinal function. Example The n nearest neighbors of x j will usually do. Remark The basic philosophy of this algorithm is very similar to that of the omitted fixed level iteration of Chapter 31 where approximate MLS generating functions were used as approximate cardinal functions. The Faul-Powell algorithm can be interpreted as a Krylov subspace method. fasshauer@iit.edu MATH 590 Chapter 33 35

107 The Faul-Powell Algorithm Remark In general, the choice of index sets allows much freedom, and this is the reason why we include the algorithm in this chapter on adaptive iterative methods. MATH 590 Chapter 33 36

108 The Faul-Powell Algorithm Remark In general, the choice of index sets allows much freedom, and this is the reason why we include the algorithm in this chapter on adaptive iterative methods. As pointed out at the end of this section, there is a certain duality between the Faul-Powell algorithm and the greedy algorithm of the previous section. fasshauer@iit.edu MATH 590 Chapter 33 36

109 The Faul-Powell Algorithm For every j = 1,..., N, the coefficients b jl are found as solution of the (relatively small) n n linear system Ψ j (x i ) = δ jk, i L j. (7) fasshauer@iit.edu MATH 590 Chapter 33 37

110 The Faul-Powell Algorithm For every j = 1,..., N, the coefficients b jl are found as solution of the (relatively small) n n linear system Ψ j (x i ) = δ jk, i L j. (7) These approximate cardinal functions are computed in a pre-processing step. fasshauer@iit.edu MATH 590 Chapter 33 37

111 The Faul-Powell Algorithm For every j = 1,..., N, the coefficients b jl are found as solution of the (relatively small) n n linear system Ψ j (x i ) = δ jk, i L j. (7) These approximate cardinal functions are computed in a pre-processing step. In its simplest form the residual iteration can be formulated as u (0) (x) = N f (x j )Ψ j (x) j=1 u (k+1) (x) = u (k) (x) + N j=1 [ ] f (x j ) u (k) (x j ) Ψ j (x), k = 0, 1,.... fasshauer@iit.edu MATH 590 Chapter 33 37

112 The Faul-Powell Algorithm Instead of adding the contribution of all approximate cardinal functions at the same time, this is done in a three-step process in the Faul-Powell algorithm. fasshauer@iit.edu MATH 590 Chapter 33 38

113 The Faul-Powell Algorithm Instead of adding the contribution of all approximate cardinal functions at the same time, this is done in a three-step process in the Faul-Powell algorithm. To this end, we choose index sets L j, j = 1,..., N n, such that while making sure that j L j. L j {j, j + 1,..., N} fasshauer@iit.edu MATH 590 Chapter 33 38

114 The Faul-Powell Algorithm Instead of adding the contribution of all approximate cardinal functions at the same time, this is done in a three-step process in the Faul-Powell algorithm. To this end, we choose index sets L j, j = 1,..., N n, such that while making sure that j L j. L j {j, j + 1,..., N} Remark If one wants to use this algorithm to approximate the interpolant based on conditionally positive definite kernels of order m, then one needs to ensure that the corresponding centers form an (m 1)-unisolvent set and append a polynomial to the local expansion (6). fasshauer@iit.edu MATH 590 Chapter 33 38

115 Step 1 The Faul-Powell Algorithm We define u (k) 0 = u (k), and then iterate u (k) j = u (k) j 1 + θ(k) j Ψ j, j = 1,..., N n, (8) fasshauer@iit.edu MATH 590 Chapter 33 39

116 Step 1 The Faul-Powell Algorithm We define u (k) 0 = u (k), and then iterate u (k) j = u (k) j 1 + θ(k) j Ψ j, j = 1,..., N n, (8) with θ (k) j = P f u (k) j 1, Ψ j NK (Ω). (9) Ψ j, Ψ j NK (Ω) fasshauer@iit.edu MATH 590 Chapter 33 39

117 Step 1 The Faul-Powell Algorithm We define u (k) 0 = u (k), and then iterate u (k) j = u (k) j 1 + θ(k) j Ψ j, j = 1,..., N n, (8) with θ (k) j = P f u (k) j 1, Ψ j NK (Ω). (9) Ψ j, Ψ j NK (Ω) Remark The stepsize θ (k) j is chosen so that the native space best approximation to the residual P f u (k) j 1 from the space spanned by the approximate cardinal functions Ψ j is added. fasshauer@iit.edu MATH 590 Chapter 33 39

118 Step 1 (cont.) The Faul-Powell Algorithm Using the representation Ψ j = l L j b jl K (, x l ), the reproducing kernel property of K, and the (local) cardinality property Ψ j (x i ) = δ jk, i L j we can calculate the denominator of (9) as fasshauer@iit.edu MATH 590 Chapter 33 40

119 Step 1 (cont.) The Faul-Powell Algorithm Using the representation Ψ j = l L j b jl K (, x l ), the reproducing kernel property of K, and the (local) cardinality property Ψ j (x i ) = δ jk, i L j we can calculate the denominator of (9) as Ψ j, Ψ j NK (Ω) = Ψ j, l L j b jl K (, x l ) NK (Ω) fasshauer@iit.edu MATH 590 Chapter 33 40

120 Step 1 (cont.) The Faul-Powell Algorithm Using the representation Ψ j = l L j b jl K (, x l ), the reproducing kernel property of K, and the (local) cardinality property Ψ j (x i ) = δ jk, i L j we can calculate the denominator of (9) as Ψ j, Ψ j NK (Ω) = Ψ j, l L j b jl K (, x l ) NK (Ω) = l L j b jl Ψ j, K (, x l ) NK (Ω) fasshauer@iit.edu MATH 590 Chapter 33 40

121 Step 1 (cont.) The Faul-Powell Algorithm Using the representation Ψ j = l L j b jl K (, x l ), the reproducing kernel property of K, and the (local) cardinality property Ψ j (x i ) = δ jk, i L j we can calculate the denominator of (9) as Ψ j, Ψ j NK (Ω) = Ψ j, l L j b jl K (, x l ) NK (Ω) = l L j b jl Ψ j, K (, x l ) NK (Ω) = l L j b jl Ψ j (x l ) fasshauer@iit.edu MATH 590 Chapter 33 40

122 Step 1 (cont.) The Faul-Powell Algorithm Using the representation Ψ j = l L j b jl K (, x l ), the reproducing kernel property of K, and the (local) cardinality property Ψ j (x i ) = δ jk, i L j we can calculate the denominator of (9) as Ψ j, Ψ j NK (Ω) = Ψ j, l L j b jl K (, x l ) NK (Ω) = l L j b jl Ψ j, K (, x l ) NK (Ω) = l L j b jl Ψ j (x l ) = b jj since we have j L j by construction of the index set L j. fasshauer@iit.edu MATH 590 Chapter 33 40

123 Step 1 (cont.) The Faul-Powell Algorithm Similarly, we get for the numerator P f u (k) j 1, Ψ j NK (Ω) = P f u (k) j 1, l L j b jl K (, x l ) NK (Ω) fasshauer@iit.edu MATH 590 Chapter 33 41

124 Step 1 (cont.) The Faul-Powell Algorithm Similarly, we get for the numerator P f u (k) j 1, Ψ j NK (Ω) = P f u (k) j 1, l L j b jl K (, x l ) NK (Ω) = l L j b jl P f u (k) j 1, K (, x l) NK (Ω) fasshauer@iit.edu MATH 590 Chapter 33 41

125 Step 1 (cont.) The Faul-Powell Algorithm Similarly, we get for the numerator P f u (k) j 1, Ψ j NK (Ω) = P f u (k) j 1, l L j b jl K (, x l ) NK (Ω) = l L j b jl P f u (k) j 1, K (, x l) NK (Ω) = ( ) b jl P f u (k) (x l ) l L j j 1 fasshauer@iit.edu MATH 590 Chapter 33 41

126 Step 1 (cont.) The Faul-Powell Algorithm Similarly, we get for the numerator P f u (k) j 1, Ψ j NK (Ω) = P f u (k) j 1, l L j b jl K (, x l ) NK (Ω) = l L j b jl P f u (k) j 1, K (, x l) NK (Ω) = ( ) b jl P f u (k) (x l ) l L j j 1 = ( ) b jl f (x l ) u (k) j 1 (x l). l L j fasshauer@iit.edu MATH 590 Chapter 33 41

127 Step 1 (cont.) The Faul-Powell Algorithm Similarly, we get for the numerator P f u (k) j 1, Ψ j NK (Ω) = P f u (k) j 1, l L j b jl K (, x l ) NK (Ω) = l L j b jl P f u (k) j 1, K (, x l) NK (Ω) = ( ) b jl P f u (k) (x l ) l L j j 1 = ( ) b jl f (x l ) u (k) j 1 (x l). l L j Therefore (8) and (9) can be written as u (k) j = u (k) j 1 + Ψ ( ) j f (x l ) u (k) b j 1 (x l), j = 1,..., N n. jj l L j b jl fasshauer@iit.edu MATH 590 Chapter 33 41

128 Step 2 The Faul-Powell Algorithm Next we interpolate the residual on the remaining n points (collected via the index set L ). fasshauer@iit.edu MATH 590 Chapter 33 42

129 Step 2 The Faul-Powell Algorithm Next we interpolate the residual on the remaining n points (collected via the index set L ). Thus, we find a function v (k) in span{k (, x j ) : j L } such that v (k) (x i ) = f (x i ) u (k) N n (x i), i L, fasshauer@iit.edu MATH 590 Chapter 33 42

130 Step 2 The Faul-Powell Algorithm Next we interpolate the residual on the remaining n points (collected via the index set L ). Thus, we find a function v (k) in span{k (, x j ) : j L } such that v (k) (x i ) = f (x i ) u (k) N n (x i), i L, and the approximation is updated, i.e., u (k+1) = u (k) N n + v (k). fasshauer@iit.edu MATH 590 Chapter 33 42

131 Step 3 The Faul-Powell Algorithm Finally, the residuals are updated, i.e., r (k+1) i = f (x i ) u (k+1) (x i ), i = 1,..., N. (10) fasshauer@iit.edu MATH 590 Chapter 33 43

132 The Faul-Powell Algorithm Step 3 Finally, the residuals are updated, i.e., r (k+1) i = f (x i ) u (k+1) (x i ), i = 1,..., N. (10) Remark The outer iteration (on k) is now repeated unless the largest of these residuals is small enough. fasshauer@iit.edu MATH 590 Chapter 33 43

133 The Faul-Powell Algorithm Algorithm (Pre-processing step) Choose n For 1 j N n do Determine the index set L j Find the coefficients b jl of the approximate cardinal function Ψ j by solving Ψ j (x i ) = δ jk, i L j end fasshauer@iit.edu MATH 590 Chapter 33 44

134 The Faul-Powell Algorithm Algorithm (Faul-Powell) Input data locations X, associated values of f, tolerance tol > 0 Perform pre-processing step Initialize: k = 0, u (k) 0 = 0, r (k) i = f (x i ), i = 1,..., N, e = max i=1,...,n While e > tol do Update u (k) j = u (k) j 1 + Ψ j b jj l L j b jl Solve the interpolation problem v (k) (x i ) = f (x i ) u (k) N n (x i), Update the approximation Compute new residuals r (k+1) i Set new value for e = Increment k = k + 1 r (k) i ( ) f (x l ) u (k) j 1 (x l), 1 j N n u (k+1) 0 = u (k) N n + v (k) max i=1,...,n i L = f (x i ) u (k+1) 0 (x i ), i = 1,..., N (k+1) r i end fasshauer@iit.edu MATH 590 Chapter 33 45

135 The Faul-Powell Algorithm Remark Faul and Powell prove that this algorithm converges to the solution of the original interpolation problem. MATH 590 Chapter 33 46

136 The Faul-Powell Algorithm Remark Faul and Powell prove that this algorithm converges to the solution of the original interpolation problem. One needs to make sure that the residuals are evaluated efficiently by using, e.g., a fast multipole expansion, fast Fourier transform, or compactly supported kernels. fasshauer@iit.edu MATH 590 Chapter 33 46

137 The Faul-Powell Algorithm Remark In its most basic form the Krylov subspace algorithm of Faul and Powell can also be explained as a dual approach to the greedy residual iteration algorithm of Schaback and Wendland. fasshauer@iit.edu MATH 590 Chapter 33 47

138 The Faul-Powell Algorithm Remark In its most basic form the Krylov subspace algorithm of Faul and Powell can also be explained as a dual approach to the greedy residual iteration algorithm of Schaback and Wendland. Instead of defining appropriate sets of points Y k, in the Faul and Powell algorithm one picks certain subspaces U k of the native space. fasshauer@iit.edu MATH 590 Chapter 33 47

139 The Faul-Powell Algorithm Remark In its most basic form the Krylov subspace algorithm of Faul and Powell can also be explained as a dual approach to the greedy residual iteration algorithm of Schaback and Wendland. Instead of defining appropriate sets of points Y k, in the Faul and Powell algorithm one picks certain subspaces U k of the native space. In particular, if U k is the one-dimensional space U k = span{ψ k } (where Ψ k is a local approximation to the cardinal function) we get the Schaback-Wendland algorithm described above. fasshauer@iit.edu MATH 590 Chapter 33 47

140 The Faul-Powell Algorithm Remark In its most basic form the Krylov subspace algorithm of Faul and Powell can also be explained as a dual approach to the greedy residual iteration algorithm of Schaback and Wendland. Instead of defining appropriate sets of points Y k, in the Faul and Powell algorithm one picks certain subspaces U k of the native space. In particular, if U k is the one-dimensional space U k = span{ψ k } (where Ψ k is a local approximation to the cardinal function) we get the Schaback-Wendland algorithm described above. For more details see [Schaback and Wendland (2000b)]. fasshauer@iit.edu MATH 590 Chapter 33 47

141 The Faul-Powell Algorithm Remark In its most basic form the Krylov subspace algorithm of Faul and Powell can also be explained as a dual approach to the greedy residual iteration algorithm of Schaback and Wendland. Instead of defining appropriate sets of points Y k, in the Faul and Powell algorithm one picks certain subspaces U k of the native space. In particular, if U k is the one-dimensional space U k = span{ψ k } (where Ψ k is a local approximation to the cardinal function) we get the Schaback-Wendland algorithm described above. For more details see [Schaback and Wendland (2000b)]. Implementation of this algorithm is omitted. fasshauer@iit.edu MATH 590 Chapter 33 47

142 References I Appendix References Berlinet, A., Thomas-Agnan, C. (2004). Reproducing Kernel Hilbert Spaces in Probability and Statistics. Kluwer, Dordrecht. Buhmann, M. D. (2003). Radial Basis Functions: Theory and Implementations. Cambridge University Press. Fasshauer, G. E. (2007). Meshfree Approximation Methods with MATLAB. World Scientific Publishers. Higham, D. J. and Higham, N. J. (2005). MATLAB Guide. SIAM (2nd ed.), Philadelphia. Iske, A. (2004). Multiresolution Methods in Scattered Data Modelling. Lecture Notes in Computational Science and Engineering 37, Springer Verlag (Berlin). fasshauer@iit.edu MATH 590 Chapter 33 48

143 References II Appendix References G. Wahba (1990). Spline Models for Observational Data. CBMS-NSF Regional Conference Series in Applied Mathematics 59, SIAM (Philadelphia). Wendland, H. (2005a). Scattered Data Approximation. Cambridge University Press (Cambridge). Faul, A. C. and Powell, M. J. D. (1999). Proof of convergence of an iterative technique for thin plate spline interpolation in two dimensions. Adv. Comput. Math. 11, pp Faul, A. C. and Powell, M. J. D. (2000). Krylov subspace methods for radial basis function interpolation. in Numerical Analysis 1999 (Dundee), Chapman & Hall/CRC (Boca Raton, FL), pp MATH 590 Chapter 33 49

144 References III Appendix References Girosi, F. (1998). An equivalence between sparse approximation and support vector machines. Neural Computation 10, pp Schaback, R. and Wendland, H. (2000a). Numerical techniques based on radial basis functions. in Curve and Surface Fitting: Saint-Malo 1999, A. Cohen, C. Rabut, and L. L. Schumaker (eds.), Vanderbilt University Press (Nashville, TN), Schaback, R. and Wendland, H. (2000b). Adaptive greedy techniques for approximate solution of large RBF systems. Numer. Algorithms 24, pp Temlyakov, V. N. (1998). The best m-term approximation and greedy algorithms. Adv. in Comp. Math. 8, pp MATH 590 Chapter 33 50