Radial basis function approximation

Transcription

1 Radial basis function approximation PhD student course in Approximation Theory Elisabeth Larsson E. Larsson, (1 : 77)

2 Global RBF approximation Stable evaluation methods RBF partition of unity methods E. Larsson, (2 : 77)

3 Short introduction to (global) RBF methods Basis functions: φ j (x) = φ( x x j ). Translates of one single function rotated around a center point. Example: Gaussians φ(εr) = exp( ε 2 r 2 ) Approximation: s ε (x ) = N j=1 λ jφ j (x ) Collocation: s ε (x i ) = f i Aλ = f ε=1 ε=1/3 ε=3 Advantages: Flexibility with respect to geometry. As easy in d dimensions. Spectral accuracy / exponential convergence. Continuosly differentiable approximation. E. Larsson, (3 : 77)

4 Commonly used RBFs Global infinitely smooth Gaussian exp( ε 2 r 2 ), ε > 0 (Inverse) multiquadric (1 + ε 2 r 2 ) β/2, ε > 0, β N Global piecewise smooth Polyharmonic spline (odd) r 2m 1, m N Polyharmonic spline (even) r 2m log(r), m N Matérn/Sobolev r ν K ν (r), ν > 0 C 2 Matérn (1 + r) exp( r), v = 3/2 Compactly supported Wendland functions C 2 and pos def for d 3, (1 r ρ )4 +(4 r ρ + 1), ρ > 0 C 2 and pos def for d 5, (1 r ρ )4 +(5 r ρ + 1), ρ > 0 E. Larsson, (4 : 77)

5 demo1.m (RBF interpolation in 1-D) E. Larsson, (5 : 77)

6 Observations from the results of demo1.m As N grows for fixed ε, convergence stagnates. As ε decreases for fixed N, the error blows up. λ min = λ max means cancellation. Coefficients λ means that cond(a). For small ε, the RBFs are nearly flat, and almost linearly dependent. That is, they form a bad basis. Max error Max error E. Larsson, (6 : 77) ε = N N = ε log 10 (max/min(λ)) log 10 (max/min(λ)) ε = N N = ε

7 Why is it interesting to use small values of ε? Driscoll & Fornberg [DF02] Somewhat surprisingly, in 1-D for small ε s(x, ε) = P N 1 (x) + ε 2 P N+1 (x) + ε 4 P N+3 (x) +, where P j is a polynomial of degree j and P N 1 (x) is the Lagrange interpolant. Implications It can be shown that cond(a) O(Nε 2(N 1) ), but the limit interpolant is well behaved. It is the intermediate step of computing λ that is ill-conditioned. By choosing the corresponding nodes, the flat RBF limit reproduces pseudo-spectral methods. This is a good approximation space. E. Larsson, (7 : 77)

8 The multivariate flat RBF limit Larsson & Fornberg [LF05], Schaback [Sch05] In n-d the flat limit can either be s(x, ε) = P K (x) + ε 2 P K+2 (x) + ε 4 P K+4 (x) +, ( ) ( ) (K 1) + d K + d where < N and P d d K is a polynomial interpolant or s(x, ε) = ε 2q P M 2q (x) + ε 2q+2 P M 2q+2 (x) + + P M (x) + ε 2 P M+2 (x) + ε 4 P M+4 (x) +. The questions of uniqueness and existence are connected with multivariate polynomial uni-solvency. Schaback [Sch05] Gaussian RBF limit interpolants always converge to the de Boor/Ron least polynomial interpolant. E. Larsson, (8 : 77)

9 The multivariate flat RBF limit: Divergence Necessary condition: Q(x) of degree N 0 such that Q(x j ) = 0, j = 1,..., N. Then divergence as ε 2q may occur, where q = (M N 0 )/2 and M = min non-degenerate degree. Points Q N 0 Basis M q x y 1 1, x, x 2, 5 2 x 3, x 4, x 5 x 2 y 1 2 1, x, y, xy, y 2 xy x 2 + y , x, y, x 2, xy, x 3, x 2 y, x Divergence actually only occurs for the first case as ε 2. E. Larsson, (9 : 77)

10 The multivariate flat RBF limit, contd Schaback [Sch05], Fornberg & Larsson [LF05] Example: In two dimensions, the eigenvalues of A follow a pattern: µ 1 O(ε 0 ), µ 2,3 O(ε 2 ), µ 4,5,6 O(ε 4 ),... ( ) k + n 1 In general, there are = (k+1) (k+n 1) n 1 (n 1)! eigenvalues µ j O(ε 2k ) in n dimensions. Implications There is an opportunity for pseudo-spectral-like methods in n-d. There is no amount of variable precision that will save us. For smooth functions, a small ε can lead to very high accuracy. E. Larsson, (10 : 77)

11 Multivariate interpolation Theorem (Mairhuber Curtis) For a domain Ω R d, d 2 that has an interior point, there is no basis of continuous functions f 1 (x ),... f N (x ), N 2 such that an interpolation matrix A = {f j (x i )} N i,j=1 is guaranteed to be non-singular (no Haar basis). Proof. Let two of the points x i and x k change places along a closed continuous path in Ω. When the two points have changed places, two rows in A are interchanged, and det(a) has changed sign. Then det(a) = 0 somewhere along the path. For RBF approximation, A = {φ( x i x j )} N i,j=1. If two points change place, two rows and two columns are swapped. Determinant does not change sign. E. Larsson, (11 : 77)

12 Positive definite functions Definition (Positive definite function) A real valued continuous function Φ is positive definite on R d it is even and N j=1 k=1 N c j c k Φ(x j x k ) 0 for any parwise distinct points x 1,..., x N R d, c j R. Theorem (Bochner 1933) A function Φ C(R d ) is positive definite on R d it is the Fourier transform of a finite non-negative Borel measure µ on R d 1 Φ(x ) = e ix ω dµ(ω). (2π) d R d E. Larsson, (12 : 77)

13 Bochner Theorem contd Partial proof N j=1 k=1 = = = N c j c k Φ(x j x k ) = 1 (2π) d 1 (2π) d N N ) (c j c k e i(x j x k ) ω dµ(ω) j=1 k=1 R d N N c j e ix j ω ix c k e k ω dµ(ω) R d j=1 1 N (2π) d c j e ix j ω R d j=1 2 k=1 dµ(ω) 0. E. Larsson, (13 : 77)

14 Example The Gaussian is positive definite in any dimension e ε2 x 2 = 1 1 (2π) d R d ( /(4ε2) 2ε) d e ω 2 e ix ω dω Theorem (Schoenberg 1938) A cont function ϕ : [0, ) R is strictly pos def and radial on R d for all d ϕ(r) = 0 e r 2 t 2 dµ(t), where µ is a finite non-negative Borel measure not concentrated at the origin. E. Larsson, (14 : 77)

15 Results and consequences for RBF approximation Non-singularity of RBF interpolation is guaranteed for distinct node points and strictly pos def functions such as the Gaussian and the inverse multiquadric. There are no oscillatory or compactly supported RBFs that are strictly pos def for all d. Because φ(r 0) = 0 breaks theorem, cf. Bessel and Wendland. Non-singularity/positive definiteness of interpolation matrix holds also for conditionally positive definite RBFs augmented with polynomials. Micchelli [Mic86], cf. multiquadric RBFs E. Larsson, (15 : 77)

16 Tensor product vs multivariate basis Tensor product basis s(x ) = n i=0 j=0 n c ij p i (x 1 )p j (x 2 ) Number of unknowns N T = (n + 1) d. Multivariate basis Thinking in terms of polynomials, a multivariate polynomial space of degree n has dimension N M = ( n + d d ) = (n + 1) (n + d) d! Degrees of freedom for n = 8: E. Larsson, (16 : 77) d N T N M

17 demo2.m (Conditioning and errors) E. Larsson, (17 : 77)

18 Comments on the results of demo2 N Error is small where condition is high and vice versa. Interesting region only reachable with stable method. Best results for small ε. RBF Direct: log 10 ( cond 2 (A) ) log 10 (ε) N RBF QR: log 10 s(x) f(x) log 10 (ε) Teaser: Conditioning for is perfect... E. Larsson, (18 : 77)

19 The Contour-Padé method Fornberg & Wright [FW04] Think of ε as a complex variable. The limit ε = 0 is a removable singularity. Complex ε for which A is singular lead to poles. Pole location only depend on the location of nodes. Example Evaluate f (ε) = 1 cos(ε) ε 2 Numerically unstable. Removable singularity at 0. Compute f (0) as average of f (ε) around safe path. Bad region Safe path Target point Im ε Re ε E. Larsson, (19 : 77)

20 The Contour-Padé method: Algorithm Compute s(x, ε) = A e A 1 f at M points around a safe path (circle). Inverse FFT of the M values gives a Laurent expansion u(x) =... + s 2 (x)ε 4 + s 1 (x)ε 2 +s 0 (x)+s 1 (x)ε 2 +s 2 (x)ε }{{} Needs to be converted Convert the negative power expansion into Padé form and find the correct number of poles and their locations s 1 ε 2 + s 2 ε = p 1ε p m ε 2m 1 + q 1 ε q n ε 2n. Evaluate u(x) using Taylor + Padé for any ε inside the circle. E. Larsson, (20 : 77)

21 The Contour-Padé method: Results max s(x) f(x) 10 0 N= Usual method 1e 9 New method 1e ε max s(x) f(x) 10 0 N= Quad precision 9e 12 Usual method 1e 9 New method 1e ε Stable computation for all ε with Contour-Padé. Limited number of nodes, otherwise general. Expensive to compute A 1 at M points. Tricky to find poles. Modern efficient version RBF-RA [WF17]. E. Larsson, (21 : 77)

22 Expansions of (Gaussian) RBFs On the surface of the sphere Hubbert & Baxter [BH01] For different RBFs there are expansions φ( x x k ) = j=0 ε 2j j m= j Cartesian space, polynomial expansion For Gaussians c j,m Y m j (x ) φ( x x k ) = e ε2 (x x k ) (x x k ) E. Larsson, (22 : 77) = e ε2 (x x ) e ε2 (x k x ) k e 2ε2 (x x k ) = e ε2 (x x ) e ε2 (x k x k ) 2j 2j ε j! (x x k) j j=0

23 Expansions of (Gaussian) RBFs contd Mercer expansion (Mercer 1909) For a positive definite kernel K(x, x k ) = φ( x x k ), there is an expansion φ( x x k ) = λ j ϕ j (x )ϕ j (x k ), j=0 where λ j are positive eigenvalues, and ϕ j (x ) are eigenfunctions of an associated compact integral operator. E. Larsson, (23 : 77)

24 The RBF-QR method on the sphere Fornberg & Piret [FP07] φ( x x k ) = j=0 ε 2j j m= j c j,m Y m j (x ) The number of SPH functions/power matches the RBF eigenvalue pattern on the sphere. If we collect RBFs and expansion functions in vectors, and coefficients in the matrix B, we have a relation Φ(x ) = B Y = Q E R Y (x ) The new basis Ψ(x ) = R Y (x ) spans the same space as Φ(x ), but the ill-conditioning has been absorbed in E. E. Larsson, (24 : 77)

25 The RBF-QR method in Cartesian space Fornberg, Larsson, Flyer [FLF11] The expansion of the Gaussian φ( x x k ) = e ε2 (x x ) e ε2 (x k x k ) j=0 2j 2j ε j! (x x k) j + The number of expansion functions for each power of ε matches the eigenvalue pattern in A. The expansion functions are the monomials. Better expansion functions in 2-D Change to polar coordinates. Trigs in the angular direction are perfect. Necessary to preserve powers of ε Partial conversion to Chebyshev polynomials. E. Larsson, (25 : 77)

26 The RBF-QR method in Cartesian space contd New expansion functions { T c j,m (x) = e ε2 r 2 r 2m T j 2m (r) cos((2m + p)θ), T s j,m (x) = e ε2 r 2 r 2m T j 2m (r) sin((2m + p)θ), Matrix form of factorized expansion Express Φ(x ) = (φ( x x 1 ),..., φ( x x N )) T in terms of expansion functions T (x ) = (T c 0,0, T c 1,0,...)T as. Φ(x ) = C D T (x ), where c ij is O(1) and D = diag(o(ε 0, ε 2, ε 2, ε 4,...)). Note that C has an infinite number of columns etc. E. Larsson, (26 : 77)

27 The RBF-QR method in Cartesian space contd The QR part The coefficient matrix C is QR-factorized so that Φ(x ) = Q [ ] [ ] D R 1 R T (x ), where R 0 D 1 and 2 D 1 are of size (N N). The change of basis Make the new basis (same space) close to T Ψ(x ) = D 1 1 R 1 1 QH Φ(x ) = [ I R ] T (x ). Analytical scaling of R = D 1 1 R 1 1 R 2D 2 Any power of ε in D 1 any power of ε in D 2 Scaling factors O(ε 0 ) or smaller, truncation is possible. E. Larsson, (27 : 77)

28 demo3.m (RBF interpolation in 2-D with and without ) E. Larsson, (28 : 77)

29 Stable computation as ε 0 and N The RBF-QR method allows stable computations for small ε. (Fornberg, Larsson, Flyer [FLF11]) Consider a finite non-periodic domain. Theorem (Platte, Trefethen, and Kuijlaars [PTK11]): Exponential convergence on equispaced nodes exponential ill-conditioning. Solution #1: Cluster nodes towards the domain boundaries. E. Larsson, (29 : 77)

30 An RBF-QR example with clustered nodes in a non-trivial domain f (x, y) = exp( (x 0.1) 2 0.5y 2 ) N=793 node points Cosine-stretching towards each boundary Maximum error 2.2e-10 E. Larsson, (30 : 77)

31 demo4.m (RBF interpolation in 2-D with clustered nodes) E. Larsson, (31 : 77)

32 Brief survey of Mercer based methods Fasshauer & McCourt [FM12] Eigenvalues and eigenfunctions in 1-D can be chosen as α λ n = 2 ( ε 2 ) n 1 α 2 + δ 2 + ε 2 α 2 + δ 2 + ε 2, ( where β = φ n = γ n e δ2 x 2 H n 1 (αβx), ) ) 1 2 4, γ n = 1 + ( 2ε α β 2 n 1 Γ(n), δ2 = α2 2 (β2 1). Eigenfunctions are orthogonal in a weighted norm. The QR-step is similar to that of previous methods. Tensor product form is used in higher dimensions The powers of ε do not match the eigenvalues of A. New parameter α to tune. E. Larsson, (32 : 77)

33 Brief survey of Mercer based methods contd De Marchi & Santin [DMS13] Discrete numerical approximation of eigenfunctions. W diagonal matrix with cubature weights. Perform SVD W A W = Q Σ 2 Q T. The eigenbasis is given by W 1 Q Σ. Rapid decay of singular values Basis can be truncated Low rank approximation of A. De Marchi & Santin [DMS15] Faster: Lanczos algorithm on Krylov space K(A, f ). Eigenfunctions through SVD of H m from Lanczos. Computationally efficient. Basis depends on f. Potential trouble for f N K (X ) For details it is a good idea to ask the authors :-) E. Larsson, (33 : 77)

34 Differentiation matrices and RBF-QR Larsson, Lehto, Heryudono, Fornberg [LLHF13] Let u X be an RBF approximation evaluated at the nodes. To compute u Y evaluated at the set of points Y, we use Aλ = u X λ = A 1 u X to get u Y = A Y λ = A Y A 1 u X where A Y (i, j) = φ j (y i ). To instead evaluate a differential operator applied to u, where A L Y (i, j) = Lφ j(y i ). u Y = A L Y A 1 u X, To do the same thing using, replace φ j with ψ j. E. Larsson, (34 : 77)

35 Solving PDEs with RBFs/RBF-QR Domain defined by: r b (θ) = (sin(6θ) + sin(3θ)). { u=f (x ), x Ω, PDE: u=g(x ), x on Ω, Solution: u(x ) = sin(x x 2 2 ) sin(2x (x 2 0.5) 2 ). Collocation: ( A A 1 X i X I ) ( ) u i X u b X ( ) f i = X g b X Evaluation: u Y = A Y A 1 X u X E. Larsson, (35 : 77) Domain + nodes

36 demo5.m (Solving the Poisson problem in 2-D using RBFs) E. Larsson, (36 : 77)

37 Reproducing Kernel Hilbert spaces and optimality Let N (Ω) be a real Hilbert space of functions u : Ω R, where Ω R d with inner product (, ) N (Ω). Consider an RBF as a kernel K(x, y). The following holds (i) K(, x ) N (Ω) for all x Ω. (ii) (u, K(, x )) N (Ω) = u(x ). Let I (u) be the interpolant of u N (Ω). Then I (u) N (Ω) u N (Ω) Consider a finite dimensional subspace N (X ) of the native space N (Ω). Then (I (u) u, v) N (Ω) = 0 for all v N (X ). E. Larsson, (37 : 77)

38 Ingredients for exponential convergence estimates Dependence on geometry through interior cone conditions. Approximation quality depends on boundary shape. General sampling inequalities based on polynomial approximation. These tell us how much a smooth error can grow between nodes. Embedding constants relating Native spaces to Sobolev spaces. These are needed to go from algebraic to exponential estimates. E. Larsson, (38 : 77)

39 Interior cone conditions Definition (Interior cone condition) A domain Ω R d satisfies an interior cone condition with radius r and angle θ if every x Ω is the vertex of such a cone that is contained entirely within Ω. Definition (Star shaped) A domain Ω R d is star shaped with respect to B(x c, r) if for every x Ω, the convex hull of x and B(x c, r) is entirely enclosed in Ω. Example A star shaped domain wrt B(x c, r), enclosed by B(x c, R) satisfies an interior cone condition with radius r and angle θ = 2 arcsin( r 2R ). Narcowich, Ward, Wendland [NWW05] E. Larsson, (39 : 77) r R θ

40 General sampling inequalitites Rieger & Zwicknagl [RZ10] Bound derivatives of u through polynomial bounds D α u(x ) D α u(x ) D α p(x ) + D α p(x ) Detailed computations with averaged Taylor polys 1 p 1 q ) D α u Lq(Ω) C S kδk d( Ω (k α )! (δ α Ω d q + 2δΩ h α u l (X ), + h α ) u W k p (Ω) where δ Ω is the diameter of Ω, h is the fill distance (largest ball empty of nodes from X ), and the constant C S depends on d, p, and θ, 1 p <, 1 q. Fill distance must be small enough and k large enough. E. Larsson, (40 : 77)

41 Embedding constants Rieger & Zwicknagl [RZ10] Assume there are embedding constants, for all k such that u W k p (Ω) E(k) u H(Ω) for some space H(Ω) of smooth functions. Further assume that E(k) C k E k(1 ɛ)k, for ɛ, C E > 0. Then, the general sampling inequality can be rewritten as D α u Lq(Ω) e C log(h) h d q u H(Ω) + 2δΩ h α u l (X ), where C = ɛ r sin θ c 0 /4 and c 0 = min{1, 4(1+sinθ) }. We are still considering star shaped domains. E. Larsson, (41 : 77)

42 Lipschitz domains Rieger & Zwicknagl [RZ10] For a general Lipschitz domain that satisfies a uniform interior cone condition, we create a cover of Ω consisting of star shaped subdomains. This affects the terms in front of the norms, but not the essentials. For E(k) C k E k(1 ɛ)k D α u Lq(Ω) e C log(h) h u H(Ω) + C 2 h α u l (X ). For E(k) C k E ksk, s 1 C D α u Lq(Ω) e h 1/(1+s) u H(Ω) + C 2 h α u l (X ). E. Larsson, (42 : 77)

43 Embedding constants for kernel spaces Fourier characterization of spaces { } Rd N K (R d ) = u C(R d ) L 2 (R d ) : u 2 û(ω) 2 N K = dω < ˆK(ω) W k 2 (R d ) = { } u L 2 (R d ) : û(ω) 2 (1 + ω 2 2) k dω < R d Finding a specific embedding constant For a particular kernel function K find E(k) such that where y = ω 2 2. E. Larsson, (43 : 77) (1 + y) k E(k)2 ˆK(y),

44 Embedding constants for kernel spaces contd Rieger & Zwicknagl [RZ10] For the Gaussian ˆK(y) = (2ε 2 ) d 2 e y 4ε 2 E(k) = C k k k 2. and For the inverse multiquadric ( y ) β ˆK(y) = 21 β Γ(β) ε (ε y) d/2 y K d/2 β ( ε ), where K is a modified Bessel function of the third kind, leading to E(k) = C k k k. Using the embedding constants We finally assume that there is an extension operator E such that Eu N (R d ) u N (Ω). Then u W k 2 (Ω) Eu W k 2 (Rd ) E(k) Eu N (R d ) E(k) u N (Ω) Wendland [Wen05, Theorem 10.46] E. Larsson, (44 : 77)

45 Implications for interpolation errors Rieger & Zwicknagl [RZ10] The interpolant I (u) is zero at the node set X (discrete term goes away). Together with the optimality property I (u) N (Ω) u N (Ω), we get for the Gaussian D α (I (u) u) Lq(Ω) e C log(h) h u N (Ω), and for the inverse multiquadric D α (I (u) u) Lq(Ω) e C h u N (Ω). These estimates can be improved, e.g., for a compact cube. E. Larsson, (45 : 77)

46 Cost of global method Global RBF approximations of smooth functions are very efficent. A small number of node points per dimension are needed. However N = 15 in 1-D becomes N = in 4-D. Up to three dimensions can be handled on a laptop, but not more. Furthermore, for less smooth functions, the number of nodes per dimension grows quickly. For a dense linear system: Direct solution O(N 3 ), storage O(N 2 ). Move to localized methods. E. Larsson, (46 : 77)

47 Motivation for Global RBF approximation + Ease of implementation in any dimension. + Flexibility with respect to geometry. + Potentially spectral convergence rates. Computationally expensive for large problems. RBF partition of unity methods Local RBF approximations on patches are blended into a global solution using a partition of unity. Provides spectral or high-order convergence. Solves the computational cost issues. Allows for local adaptivity. [Wen02, Fas07, HL12, Cav12, CDR14, CDR15, CDRP16, CRP16], [SVHL15, HLRvS16, SL16, LSH17] E. Larsson, (47 : 77)

48 The RBF partition of unity method Global approximation P ũ(x) = w j (x)ũ j (x) j=1 PU weight functions Generate weight functions from compactly supported C 2 Wendland functions ψ(ρ) = (4ρ + 1)(1 ρ) 4 + using Shepard s method w i (x) = ψ i (x) M j=1 ψ j (x). Ω j Cover Each x Ω must be in the interior of at least one Ω j. Patches that do not contain unique points are pruned. E. Larsson, (48 : 77)

49 Differentiating approximations Applying an operator globally M ũ = w i ũ i + 2 w i ũ i + w i ũ i i=1 Local differentiation matrices Let u i be the vector of nodal values in patch Ω i, then u i = Aλ i, where A ij = φ j (x i ) Lu i = A L A 1 u i, where A L ij = Lφ j (x i ). The global differentiation matrix Local contributions are added into the global matrix E. Larsson, (49 : 77) nz = 10801

50 demo6.m (Solving a Poisson problem in 2-D with RBF PUM) E. Larsson, (50 : 77)

51 An collocation method Choices & Implications Nodes and evaluation points coincide. Square matrix, iterative solver available (Heryudono, Larsson, Ramage, von Sydow [HLRvS16]). Global node set. Solutions ũ i (x k ) = ũ j (x k ) for x k in overlap regions. Patches are cut by the domain boundary. Potentially strange shapes and lowered local order. E. Larsson, (51 : 77)

52 An least squares method Choices & Implications Each patch has an identical node layout. Computational cost for setup is drastically reduced. Evaluation nodes are uniform. Easy to generate both local and global high quality node sets. Patches have nodes outside the domain. Good for local order, but requires denser evaluation points. E. Larsson, (52 : 77)

53 The interpolation error E α = D α (I (u) u) = M j=1 β α The weight functions For C k weight functions and α k D α w j L (Ω j ) C α H α j ( ) α D β w j D α β (I (u j ) u j ) β, H j = diam(ω j ). The local RBF interpolants (Gaussians) Define the local fill distance h j (Rieger, Zwicknagl [RZ10]) D α (I (u j ) u j ) L ( Ω j ) c α,jh m j d 2 α j u j N ( Ω j ), D α (I (u j ) u j ) L ( Ω j ) eγ α,j log(h j )/ h j u j N ( Ωj ). E. Larsson, (53 : 77)

54 interpolation error estimates Algebraic estimate for H j /h j = c E α L (Ω) K max 1 j M C j H m j d 2 α j u N ( Ωj ) K Maximum # of Ω j overlapping at one point m j Related to the local # of points Ω j Ω j Ω Spectral estimate for fixed partitions E α L (Ω) K max 1 j M Ce γ j log(h j )/ h j u N ( Ωj ) Implications Bad patch reduces global order. Two refinement modes. Guidelines for adaptive refinement. E. Larsson, (54 : 77)

55 Error estimate for PDE approximation Larsson, Shcherbakov, Heryudono [LSH17] The PDE estimate ũ u L (Ω) C P E L + C P L,X L + Y,X (C M δ M + E L ), where C P is a well-posedness constant and C M δ M is a small multiple of the machine precision. Implications Interpolation error E L provides convergence rate. Norm of inverse/pseudoinverse can be large. Matrix norm better with oversampling. Finite precision accuracy limit involves matrix norm. Follows strategies from Schaback [Sch07, Sch16] E. Larsson, (55 : 77)

56 Does require stable methods? In order to achieve convergence we have two options Refine patches such that diameter H decreases. Increase node numbers such that N j increases. In both cases, theory assumes ε fixed. The effect of patch refinement H = 1, ε = 4 H = 0.5, ε = 4 H = 0.25, ε = H 0 0 H 0 0 H The method: Stable as ε 0 for N 1 Effectively a change to a stable basis. Fornberg, Piret [FP07], Fornberg, Larsson, Flyer [FLF11], Larsson, Lehto, Heryudono, Fornberg [LLHF13] E. Larsson, (56 : 77)

57 Effects on the local matrices Local contribution to a global Laplacian L j = (Wj A j + 2Wj A j + W j A j )A 1 j. Typically: A j ill-conditioned, L j better conditioned. RBF-QR for accuracy Stable for small RBF shape parameters ε Change of basis à = AQR1 T D1 T Same result in theory à L à 1 = A L A 1 More accurate in practice Relative error in A j A 1 j without RBF-QR N=10 N=20 N= ε E. Larsson, (57 : 77)

58 Poisson test problems in 2-D Domain Ω = [ 2, 2] 2. Uniform nodes in the collocation case. u R (x, y) = 1 25x 2 +25y 2 +1 u T (x, y) = sin(2(x 0.1) 2 ) cos((x 0.3) 2 )+sin 2 ((y 0.5) 2 ) E. Larsson, (58 : 77)

59 Error results with and without Error Least squares Fixed shape ε = 0.5 or scaled such that εh = c Left: 5 5 patches Spectral mode RBF QR Direct Scaled Points per dimension Right: 55 points per patch Error Algebraic mode p= 7 RBF QR Direct Scaled p= Patches per dimension With better results for H/h large. Scaled approach good until saturation. E. Larsson, (59 : 77)

60 Convergence as a function of patch size Runge Trig Error H Error H Collocation (dashed lines) and Least Squares (solid lines). Points per patch n = 28, 55, 91. Theoretical rates p = 4, 7, 10. Numerical rates p 3.9, 6.9, 9.8. E. Larsson, (60 : 77)

61 Spectral convergence for fixed patches Error Runge Error Trig /h /h Collocation (dashed lines) and Least Squares (solid lines). LS-RBF-PU is significantly more accurate due to the constant number of nodes per patch. E. Larsson, (61 : 77)

62 Robustness and large scale problems The global error estimate ũ u L (Ω) C P E L + C P L,X L + Y,X (C M δ M + E L ) The dark horse is the stability matrix norm The stability norm is related to conditioning. In the collocation case, L 1 X,X grows with N. How does it behave with least squares? E. Larsson, (62 : 77)

63 Stability norm: Patch size Fixed number of points per patch n = 28, 55, 91 Results as a function of patch diameter H Stability(H) Error(H) Stability norm H Error H Collocation (dashed) and LS (solid) The norm does not grow for LS- (!) E. Larsson, (63 : 77)

64 RBF-generated finite differences RBF FD Flyer et al. [FW09, FLB + 12] Approximate Lu(x c ) using the n nearest nodes by n Lu(x c ) w k u(x k ) k=1 Find weights w k by asking exactness for RBF-interpolants. φ 1 (x 1 ) φ 1 (x 2 ) φ 1 (x n ) φ 2 (x 1 ) φ 2 (x 2 ) φ 2 (x n ) φ n (x 1 ) φ n (x 2 ) φ n (x n ) w 1 w 2. w n Lφ 1 (x c ) = Lφ 2 (x c ).. Lφ n (x c ) E. Larsson, (64 : 77)

65 Is RBF-QR needed with RBF FD? Approximation of u with n = 56. Magenta lines are with added polynomial terms p = 0,..., 3. l error ε = 1.5, direct ε = 1.5, RBF-QR ε = 0 (polynomial) εh = 0.3, direct h Scaled ε: No ill-conditioning, but saturation/stagnation. [LLHF13] Fixed ε: RBF-QR is needed. Added terms: Compromise with partial recovery. E. Larsson, (65 : 77)

66 with PHS and polynomials Combine polyharmonic splines, e.g, φ(r) = r 7 with polynomial terms 1, x, y,..., x 2,... such that the number of polynomial terms the number of nodes. Contains both smooth and piecewise smooth components, that have different roles in the approximation. No shape parameter to tune. Heuristically, skewed stencils seem to behave well near boundaries. Bayona, Flyer, Fornberg, Barnett [FFBB16, BFFB17] E. Larsson, (66 : 77)

67 References I Victor Bayona, Natasha Flyer, Bengt Fornberg, and Gregory A. Barnett, On the role of polynomials in approximations: II. Numerical solution of elliptic PDEs, J. Comput. Phys. 332 (2017), Brad J. C. Baxter and Simon Hubbert, Radial basis functions for the sphere, Recent progress in multivariate approximation (Witten-Bommerholz, 2000), Internat. Ser. Numer. Math., vol. 137, Birkhäuser, Basel, 2001, pp MR Roberto Cavoretto, Partition of unity algorithm for two-dimensional interpolation using compactly supported radial basis functions, Commun. Appl. Ind. Math. 3 (2012), no. 2, e 431, 13. MR E. Larsson, (67 : 77)

68 References II Roberto Cavoretto and Alessandra De Rossi, A meshless interpolation algorithm using a cell-based searching procedure, Comput. Math. Appl. 67 (2014), no. 5, MR , A trivariate interpolation algorithm using a cube-partition searching procedure, SIAM J. Sci. Comput. 37 (2015), no. 4, A1891 A1908. MR R. Cavoretto, A. De Rossi, and E. Perracchione, Efficient computation of partition of unity interpolants through a block-based searching technique, Comput. Math. Appl. 71 (2016), no. 12, MR E. Larsson, (68 : 77)

69 References III Roberto Cavoretto, Alessandra De Rossi, and Emma Perracchione, RBF-PU interpolation with variable subdomain sizes and shape parameters, AIP Conference Proceedings 1776 (2016), no. 1, T. A. Driscoll and B. Fornberg, Interpolation in the limit of increasingly flat radial basis functions, Comput. Math. Appl. 43 (2002), no. 3-5, , Radial basis functions and partial differential equations. MR Stefano De Marchi and Gabriele Santin, A new stable basis for radial basis function interpolation, J. Comput. Appl. Math. 253 (2013), MR E. Larsson, (69 : 77)

70 References IV, Fast computation of orthonormal basis for RBF spaces through Krylov space methods, BIT 55 (2015), no. 4, MR Gregory E. Fasshauer, Meshfree approximation methods with MATLAB, Interdisciplinary Mathematical Sciences, vol. 6, World Scientific Publishing Co. Pte. Ltd., Hackensack, NJ, MR Natasha Flyer, Bengt Fornberg, Victor Bayona, and Gregory A. Barnett, On the role of polynomials in approximations: I. Interpolation and accuracy, J. Comput. Phys. 321 (2016), MR E. Larsson, (70 : 77)

71 References V Natasha Flyer, Erik Lehto, Sébastien Blaise, Grady B. Wright, and Amik St-Cyr, A guide to RBF-generated finite differences for nonlinear transport: Shallow water simulations on a sphere, J. Comput. Phys. 231 (2012), no. 11, MR Bengt Fornberg, Elisabeth Larsson, and Natasha Flyer, Stable computations with Gaussian radial basis functions, SIAM J. Sci. Comput. 33 (2011), no. 2, MR Gregory E. Fasshauer and Michael J. McCourt, Stable evaluation of Gaussian radial basis function interpolants, SIAM J. Sci. Comput. 34 (2012), no. 2, A737 A762. MR E. Larsson, (71 : 77)

72 References VI Bengt Fornberg and Cécile Piret, A stable algorithm for flat radial basis functions on a sphere, SIAM J. Sci. Comput. 30 (2007), no. 1, MR B. Fornberg and G. Wright, Stable computation of multiquadric interpolants for all values of the shape parameter, Comput. Math. Appl. 48 (2004), no. 5-6, MR Natasha Flyer and Grady B. Wright, A radial basis function method for the shallow water equations on a sphere, Proc. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 465 (2009), no. 2106, MR Alfa Heryudono and Elisabeth Larsson, FEM-RBF: A geometrically flexible, efficient numerical solution technique for partial differential equations with mixed regularity, Tech. report, Marie Curie FP7, E. Larsson, (72 : 77)

73 References VII Alfa Heryudono, Elisabeth Larsson, Alison Ramage, and Lina von Sydow, Preconditioning for radial basis function partition of unity methods, J. Sci. Comput. 67 (2016), no. 3, MR E. Larsson and B. Fornberg, Theoretical and computational aspects of multivariate interpolation with increasingly flat radial basis functions, Comput. Math. Appl. 49 (2005), no. 1, MR Elisabeth Larsson, Erik Lehto, Alfa Heryudono, and Bengt Fornberg, Stable computation of differentiation matrices and scattered node stencils based on Gaussian radial basis functions, SIAM J. Sci. Comput. 35 (2013), no. 4, A2096 A2119. MR E. Larsson, (73 : 77)

74 References VIII Elisabeth Larsson, Victor Shcherbakov, and Alfa Heryudono, A least squares radial basis function partition of unity method for solving pdes, SIAM J. Sci. Comp. (2017). Charles A. Micchelli, Interpolation of scattered data: distance matrices and conditionally positive definite functions, Constr. Approx. 2 (1986), no. 1, Francis J. Narcowich, Joseph D. Ward, and Holger Wendland, Sobolev bounds on functions with scattered zeros, with applications to radial basis function surface fitting, Math. Comp. 74 (2005), no. 250, MR E. Larsson, (74 : 77)

75 References IX Rodrigo B. Platte, Lloyd N. Trefethen, and Arno B. J. Kuijlaars, Impossibility of fast stable approximation of analytic functions from equispaced samples, SIAM Rev. 53 (2011), no. 2, MR Christian Rieger and Barbara Zwicknagl, Sampling inequalities for infinitely smooth functions, with applications to interpolation and machine learning, Adv. Comput. Math. 32 (2010), no. 1, MR Robert Schaback, Multivariate interpolation by polynomials and radial basis functions, Constr. Approx. 21 (2005), no. 3, MR , Convergence of unsymmetric kernel-based meshless collocation methods, SIAM J. Numer. Anal. 45 (2007), no. 1, (electronic). MR E. Larsson, (75 : 77)

76 References X, All well-posed problems have uniformly stable and convergent discretizations, Numer. Math. 132 (2016), no. 3, MR Victor Shcherbakov and Elisabeth Larsson, Radial basis function partition of unity methods for pricing vanilla basket options, Comput. Math. Appl. 71 (2016), no. 1, MR Ali Safdari-Vaighani, Alfa Heryudono, and Elisabeth Larsson, A radial basis function partition of unity collocation method for convection-diffusion equations arising in financial applications, J. Sci. Comput. 64 (2015), no. 2, MR E. Larsson, (76 : 77)

77 References XI Holger Wendland, Fast evaluation of radial basis functions: methods based on partition of unity, Approximation theory, X (St. Louis, MO, 2001), Innov. Appl. Math., Vanderbilt Univ. Press, Nashville, TN, 2002, pp MR , Scattered data approximation, Cambridge Monographs on Applied and Computational Mathematics, vol. 17, Cambridge University Press, Cambridge, MR Grady B. Wright and Bengt Fornberg, Stable computations with flat radial basis functions using vector-valued rational approximations, J. Comput. Phys. 331 (2017), no. C, E. Larsson, (77 : 77)