Consistency Estimates for gfd Methods and Selection of Sets of Influence

Transcription

1 Consistency Estimates for gfd Methods and Selection of Sets of Influence Oleg Davydov University of Giessen, Germany Localized Kernel-Based Meshless Methods for PDEs ICERM / Brown University 7 11 August 2017 Oleg Davydov Consistency Estimates for gfd 1

2 Outline 1 Generalized Finite Difference Methods 2 Error of Polynomial and Kernel Numerical Differentiation Numerical Differentiation Polynomial Formulas Kernel-Based Formulas Least Squares Formulas 3 Selection of Sets of Influence 4 Conclusion Oleg Davydov Consistency Estimates for gfd 2

3 Outline 1 Generalized Finite Difference Methods 2 Error of Polynomial and Kernel Numerical Differentiation Numerical Differentiation Polynomial Formulas Kernel-Based Formulas Least Squares Formulas 3 Selection of Sets of Influence 4 Conclusion Oleg Davydov Consistency Estimates for gfd 3

4 Generalized Finite Difference Methods Model problem: Poisson equation with Dirichlet boundary u = f on Ω, u Ω = g. Localized numerical differentiation (Ξ Ω, Ξ i Ξ small): u(ξ i ) ξ j Ξ i w i,j u(ξ j ) for all ξ i Ξ\ Ω Find a discrete approximate solution û defined on Ξ s.t. ξ j Ξ i w i,j û(ξ j ) = f(ξ i ) for ξ i Ξ\ Ω û(ξ i ) = g(ξ i ) for ξ i Ω Sparse system matrix [w i,j ] ξi,ξ j Ξ\ Ω. Oleg Davydov Consistency Estimates for gfd 3

5 Generalized Finite Difference Methods Sets of influence: Select Ξ i for each ξ i Ξ\ Ξ ξ i Ξ i ξ i Ξ i is the star or set of influence of ξ i Oleg Davydov Consistency Estimates for gfd 4

6 Generalized Finite Difference Methods Consistency and Stability = Convergence : û u Ξ S [ u(ξ }{{} i ) ] w i,j u(ξ j ) ξ solution error j Ξ i ξ i Ξ\ Ω } {{ } consistency error S := [w i,j ] 1 ξ i,ξ j Ξ\ Ω stability constant a vector norm, e.g. (max) or quadratic mean (rms), respectively a matrix norm, or 2 If S is bounded, then the convergence order for a sequence of discretisations Ξ n is determined by the consistency error: u(ξ i ) ξ j Ξ i w i,j u(ξ j ) (numerical differentiation error) Oleg Davydov Consistency Estimates for gfd 5

7 Outline 1 Generalized Finite Difference Methods 2 Error of Polynomial and Kernel Numerical Differentiation Numerical Differentiation Polynomial Formulas Kernel-Based Formulas Least Squares Formulas 3 Selection of Sets of Influence 4 Conclusion Oleg Davydov Consistency Estimates for gfd 6

8 Outline 1 Generalized Finite Difference Methods 2 Error of Polynomial and Kernel Numerical Differentiation Numerical Differentiation Polynomial Formulas Kernel-Based Formulas Least Squares Formulas 3 Selection of Sets of Influence 4 Conclusion Oleg Davydov Consistency Estimates for gfd 6

9 Numerical Differentiation Given a finite set of points X = {x 1,..., x N } R d and function values f j = f(x j ), we want to approximate the values Df(z) at arbitrary points z, where D is a linear differential operator Df(z) = a α (z) α f(z) α Z d +, α k k is the order of D, α := α 1 + +α d, a α (z) R. Oleg Davydov Consistency Estimates for gfd 6

10 Numerical Differentiation Approximation approach Df(z) Dp(z), where p is an approximation of f, e.g., least squares fit from a finite dimensional space P partition of unity interpolant moving least squares fit RBF / kernel interpolant If p = m a i φ i and the coefficients a i depend linearly on i=1 f(x j ), i.e. a = Af X, then p = φ a = φaf X, Dp(z) = Dφ(z)A f }{{} X = w j f(x j ). w This leads to a numerical differentiation formula Df(z) w j f(x j ), w: a weight vector. Oleg Davydov Consistency Estimates for gfd 7

11 Numerical Differentiation Exactness approach Require exactness of the numerical differentiation formula for all elements of a space P: Dp(z) = w j p(x j ) for all p P. Notation: w D P. E.g., exactness for polynomials of certain order q: P = Π d q, the space of polynomials of total degree < q in d variables. (Polynomial numerical differentiation.) Example: five point star (exact for Π 2 4 ) x 3 x 2 x 0 x 1 x 4 u(x 0 ) 1 h 2 ( u(x1 )+u(x 2 )+u(x 3 )+u(x 4 ) 4u(x 0 ) ) Oleg Davydov Consistency Estimates for gfd 8

12 Numerical Differentiation Exactness approach A classical method for computing weights w D Π d q is truncation of Taylor expansion of local error f p near z (as in the Finite Difference Method). Instead, we can look at Dp(z) = w j p(x j ) for all p Π d q. as an underdetermined linear system w.r.t. w, and pick solutions with desired properties. Similar to quadrature rules (Gauss formulas), there are special point sets that admit weights with particularly high exactness order for a given N (five point star). Oleg Davydov Consistency Estimates for gfd 9

13 Numerical Differentiation Joint work with Robert Schaback O. Davydov and R. Schaback, Error bounds for kernel-based numerical differentiation, Numer. Math., 132 (2016), O. Davydov and R. Schaback, Minimal numerical differentiation formulas, preprint. arxiv: O. Davydov and R. Schaback, Optimal stencils in Sobolev spaces, preprint. arxiv: Oleg Davydov Consistency Estimates for gfd 10

14 Outline 1 Generalized Finite Difference Methods 2 Error of Polynomial and Kernel Numerical Differentiation Numerical Differentiation Polynomial Formulas Kernel-Based Formulas Least Squares Formulas 3 Selection of Sets of Influence 4 Conclusion Oleg Davydov Consistency Estimates for gfd 11

15 Polynomial Formulas: General Error Bound Theorem If w is exact for polynomials of order q > k (the order of D), then N Df(z) w j f(x j ) f,q,ω w j x j z q 2, ( 1 where f,q,ω := q! α =q 1 α! α f 2,Ω ) 1/2. Oleg Davydov Consistency Estimates for gfd 11

16 Polynomial Formulas: General Error Bound Theorem If w is exact for polynomials of order q > k (the order of D), then N Df(z) w j f(x j ) f,q,ω w j x j z q 2, ( 1 where f,q,ω := q! α =q 1 α! α f 2,Ω ) 1/2. Proof: Let R(x) := f(x) T q,z f(x) be the remainder of the Taylor polynomial or order q. Recall the integral representation R(x) = q α =q (x z) α α! 1 Since q > k, it follows that DR(z) = 0. 0 (1 t) q 1 α f(z+t(x z)) dt. Oleg Davydov Consistency Estimates for gfd 11

17 Polynomial Formulas: General Error Bound Thus, we have for R(x) := f(x) T q,z f(x): DR(z) = 0, R(x) = q (x z) α α! α =q 1 0 (1 t) q 1 α f(z+t(x z)) dt. Hence R(x j ) α =q α =q (x j z) α α f α! C(Ω) ( (x j z) 2α α! = x j z q 2 α =q α f 2 ) C(Ω) 1/2 α! ) 1/2 ( 1 1 q! α! α f 2 C(Ω) α =q }{{} = f,q,ω Oleg Davydov Consistency Estimates for gfd 12

18 Polynomial Formulas: General Error Bound With R(x) := f(x) T q,z f(x), DR(z) = 0 and R(x j ) x j z q 2 f,q,ω, polynomial exactness implies Df(z) w j f(x j ) = DR(z) w j R(x j ) w j R(x j ) N = f,q,ω w j x j z q 2. Oleg Davydov Consistency Estimates for gfd 13

19 Polynomial Formulas: General Error Bound If w is exact for polynomials of order q > k (the order of D), then N Df(z) w j f(x j ) f,q,ω w j x j z q 2. Gives in particular an error bound in terms of Lebesgue (stability) constant w 1 := N w j : where Df(z) w j f(x j ) f,q,ω w 1 h q z,x, h z,x := max 1 j N x j z 2 is the radius of the set of influence. Applicable in particular to polyharmonic formulas. Oleg Davydov Consistency Estimates for gfd 14

20 Polynomial Formulas: General Error Bound If w is exact for polynomials of order q > k (the order of D), then N Df(z) w j f(x j ) f,q,ω w j x j z q 2. The best bound is obtained if w j x j z q 2 is minimized over all weights w satisfying the exactness condition Dp(z) = N w j p(x j ), p Π d q. (w D Π d q) We call them 1,q -minimal weights. Oleg Davydov Consistency Estimates for gfd 14

21 Polynomial Formulas: 1,µ -minimal weights An 1,µ -minimal (µ 0) weight vector w satisfies w j x j z µ 2 = inf w R N w D Π d q w j x j z µ 2. Oleg Davydov Consistency Estimates for gfd 15

22 Polynomial Formulas: 1,µ -minimal weights An 1,µ -minimal (µ 0) weight vector w satisfies w j x j z µ 2 = inf w R N w D Π d q w j x j z µ 2. 1,µ -minimal weights can be found by linear programming (e.g. simplex algorithm if N is small). Oleg Davydov Consistency Estimates for gfd 15

23 Polynomial Formulas: 1,µ -minimal weights An 1,µ -minimal (µ 0) weight vector w satisfies w j x j z µ 2 = inf w R N w D Π d q w j x j z µ 2. 1,µ -minimal weights can be found by linear programming (e.g. simplex algorithm if N is small). µ = 0: formulas with optimal stability constant N w j Oleg Davydov Consistency Estimates for gfd 15

24 Polynomial Formulas: 1,µ -minimal weights An 1,µ -minimal (µ 0) weight vector w satisfies w j x j z µ 2 = inf w R N w D Π d q w j x j z µ 2. 1,µ -minimal weights can be found by linear programming (e.g. simplex algorithm if N is small). µ = 0: formulas with optimal stability constant N w j Our error bound suggests the choice µ = q. Oleg Davydov Consistency Estimates for gfd 15

25 Polynomial Formulas: 1,µ -minimal weights An 1,µ -minimal (µ 0) weight vector w satisfies w j x j z µ 2 = inf w R N w D Π d q w j x j z µ 2. 1,µ -minimal weights can be found by linear programming (e.g. simplex algorithm if N is small). µ = 0: formulas with optimal stability constant N w j Our error bound suggests the choice µ = q. w is sparse in the sense that the number of nonzero w j s does not exceed dimπ d q. Oleg Davydov Consistency Estimates for gfd 15

26 Polynomial Formulas: 1,µ -minimal weights An 1,µ -minimal (µ 0) weight vector w satisfies w j x j z µ 2 = inf w R N w D Π d q w j x j z µ 2. 1,µ -minimal weights can be found by linear programming (e.g. simplex algorithm if N is small). µ = 0: formulas with optimal stability constant N w j Our error bound suggests the choice µ = q. w is sparse in the sense that the number of nonzero w j s does not exceed dimπ d q. Considered by Seibold (2006) for D = under additional condition of positivity, which restricts exactness to q 4. Oleg Davydov Consistency Estimates for gfd 15

27 Polynomial Formulas: 1,µ -minimal weights Influence of µ on the location of nonzero weights w j (a) µ = 0 (b) µ = q = 7 (c) µ = 15 1,µ -minimal weights (µ = 0, 7, 15) of exactness order q = 7 computed for the Laplacian at the origin from the data at 150 points. Locations of 28 points x j for which wj 0 are shown. Oleg Davydov Consistency Estimates for gfd 16

28 Polynomial Formulas: 1,µ -minimal weights Scalability 1,µ -minimal formulas are scalable in the sence that the weight vector w can be computed by scaling z, X into 0, Y in the unit circle by y j = h 1 z,x (x j z), obtaining weight vector v for the mapped differential operator, and scaling back by w j = h k z,x v j. This allows for stable computation of this formulas for any small radius h z,x by upscaling preconditioning. Any scalable differentiation formulas admit error bounds of the type Ch s k z,x for sufficiently smooth functions f, where C depends on the Lebesque constant of the upscaled formula v. Oleg Davydov Consistency Estimates for gfd 17

29 Polynomial Formulas: Growth Function Duality: inf w D Π d q w i x i z q 2 = i=1 = sup { Dp(z) : p Π d q, p(x i ) x i z q 2, i} =: ρ q,d (z, X) A special case of Fenchel s duality theorem, but can be also proved directly by using extension of linear functionals. We call ρ q,d (z, X) the growth function. More general, for any seminorm on R N, inf w = sup { Dp(z) : p Π d q, p X 1 } w D Π d q =: ρ q,d (z, X, ). Oleg Davydov Consistency Estimates for gfd 18

30 Polynomial Formulas: Growth Function Theorem For any 1,q -minimal formula, Df(z) w j f(x j ) ρ q,d (z, X) f,q,ω. As we will see, similar estimates involving ρ q,d (z, X) hold for kernel methods as well! Oleg Davydov Consistency Estimates for gfd 19

31 Polynomial Formulas: Growth Function Default behavior of growth function ρ q,d (z, X) := sup { Dp(z) : p Π d q, p(x i) x i z q 2, i}, h z,x := max 1 j N z x j 2 If X is a good set for Π d q ( norming set ), then max p(x) C max p(x i ) Ch q x z 2 h z,x /2 i z,x, hence Dp(z) Ch q k z,x and ρ q,d(z, X) Ch q k z,x, so that we get an error bound of order h q k z,x : Df(z) w j f(x j ) Ch q k z,x f,q,ω. X does not have to a norming set. Example: five point star. Oleg Davydov Consistency Estimates for gfd 20

32 Polynomial Formulas: Growth Function Example: Five point stencil for Laplace operator in 2D u(z) 5 i=1 w iu(x i ), {x 1,...,x 5 } = X h = {z, z±(h, 0), z±(0, h)} Ω h The classical FD formula with weights w 2 = w 3 = w 4 = w 5 = 1/h 2, w 1 = 4/h 2 is exact for Π 2 4. It is the only formula on X h with this exactness order, hence it is 1,4 -minimal. It is easy to show that ρ 4, (z, X) = 4h 2 Hence, f(z) 5 w i u(x i ) 4h 2 f,4,ω h, i=1 similar to classical error estimates for the five point stencil. Oleg Davydov Consistency Estimates for gfd 21

33 Polynomial Formulas: Growth Function A lower bound for well separated centers Theorem Given z and X, let γ 1 be such that x j z 2 γ dist(x j, X\{x j }), j = 1,...,N. For any w and q > k there exists a function f C (R d ) s. t. N Df(z) w j f(x j ) C f,q,ω w j x j z q 2, where C depends only on q, k, N, d and γ. In particular, if w is exact for polynomials of order q, then Df(z) w j f(x j ) Cρ q,d (z, X) f,q,ω. Oleg Davydov Consistency Estimates for gfd 22

34 Outline 1 Generalized Finite Difference Methods 2 Error of Polynomial and Kernel Numerical Differentiation Numerical Differentiation Polynomial Formulas Kernel-Based Formulas Least Squares Formulas 3 Selection of Sets of Influence 4 Conclusion Oleg Davydov Consistency Estimates for gfd 23

35 Kernel-Based Formulas Let K : R d R d R be a symmetric kernel, conditionally positive definite (cpd) of order s 0 on R d (positive definite when s = 0). Π d s : polynomials of order s. For a Π d s -unisolvent X, the kernel interpolant r X,K,f in the form r X,K,f = M a j K(, x j )+ b j p j, a j, b j R, M = dim(π d s ), is uniquely determined from the positive definite linear system M r X,K,f (x k ) = a j K(x k, x j )+ b j p j (x k ) = f k, 1 k N, a j p i (x j ) = 0, 1 i M. Oleg Davydov Consistency Estimates for gfd 23

36 Kernel-Based Formulas Examples. K(x, y) = φ( x y ) (φ : R + R is then a radial basis function (RBF)) s 0: Any φ with positive Fourier transform of Φ(x) = φ( x ) Gaussian φ(r) = e r2 inverse quadric 1/(1+r 2 ) inverse multiquadric 1/ 1+r 2 Matérn kernel K ν (r)r ν, ν > 0 (K ν (r) modified Bessel function of second kind) s 1: multiquadric 1+r 2 s ν/2 +1: polyharmonic / thin plate spline r ν {log r} K(εx, εy) are also cpd kernels (ε > 0: shape parameter) Oleg Davydov Consistency Estimates for gfd 24

37 Kernel-Based Formulas Optimal Recovery r X,K,f depends linearly on the data f j = f(x j ), r X,K,f (z) = w j f(x j ), w j R, j = 1,...,N. (w j = w j (z) depends on the evaluation point z R d ) The weights w = {wj } N provide optimal recovery of f(z) for f in the native space F K associated with K, i.e., inf w R N w Π d s sup f FK 1 f(z) w j f(x j ) = sup f FK 1 f(z) j f(x j ), w w Π d s : exactness for polynomials in Π d s (empty if s = 0). Oleg Davydov Consistency Estimates for gfd 25

38 Kernel-Based Formulas Native Space" F K If K is positive definite, then F K is just the reproducing kernel Hilbert space associated with K ; in the c.p.d. case a generalization of it (a semi-hilbert space). In the translation-invariant case K(x, y) = Φ(x y) on R d, F K = {f L 2 (R d ) : f FK := ˆf/ Φ < }. L2 (R d ) Matérn kernel K(x, y) = K ν ( x y ) x y ν : Φ(ω) = c ν,d (1+ ω 2 ) ν d/2 = f FK = c ν,d f H ν+d/2 (R d ) (Sobolev space) Polyharmonics: f FK equlivalent to Sobolev seminorm C kernels: spaces of infinitely differentiable functions Oleg Davydov Consistency Estimates for gfd 26

39 Kernel-Based Formulas A kernel-based numerical differentiation formula is obtained by applying D to the kernel interpolant (approximation approach): Df(z) Dr X,K,f (z) = wj f(x j ). The weights wj can be calculated by solving the system M wj K(x k, x j ) + c j p j (x k ) = [DK(, x k )](z), 1 k N, w j p i (x j ) + 0 = Dp i (z), 1 i M. Oleg Davydov Consistency Estimates for gfd 27

40 Kernel-Based Formulas: Optimal recovery Kernel-based weights w = {wj } N provide optimal recovery of Df(z) from f(x j ), j = 1,..., N, for f F K, inf w R N w D Π d s sup f FK 1 Df(z) w j f(x j ) = sup f FK 1 Df(z) F K is the native space of K w D Π d s : exactness of numerical differentiation for polynomials in Π d s, j f(x j ), w Oleg Davydov Consistency Estimates for gfd 28

41 Kernel-Based Formulas: Optimal recovery Kernel-based weights w = {wj } N provide optimal recovery of Df(z) from f(x j ), j = 1,..., N, for f F K, inf w R N w D Π d s sup f FK 1 Df(z) w j f(x j ) = sup f FK 1 Df(z) F K is the native space of K w D Π d s : exactness of numerical differentiation for polynomials in Π d s, For example, the formula obtained with Matérn kernel K(x, y) = K ν ( x y ) x y ν, ν > 0 (s = 0), gives the best possible estimate of Df(z) if we only know that f belongs to the Sobolev space F K = H ν+d/2 (R d ) j f(x j ), w Oleg Davydov Consistency Estimates for gfd 28

42 Kernel-Based Formulas: Optimal recovery Optimal recovery error P X (z) := sup f FK 1 Df(z) j f(x j ) w is called power function and can be evaluated as P X (z) = ǫ x w ǫy w K(x, y), ǫ wf := Df(z) w j f(x j ) can be used to optimize the choice of the local set X. Oleg Davydov Consistency Estimates for gfd 29

43 Kernel-Based Formulas: Error Bounds Theorem For any q max{s, k + 1}, Df(z) Dr X,K,f (z) ρ q,d (z, X)C K,q f FK, f F K, as soon as α,β K(x, y) C(Ω Ω) for α, β q, where ρ q,d (z, X) is the growth function, C K,q := 1 ( ( )( ) q q 1/4 α,β K 2 q! α β C(Ω Ω)) <. α, β =q Oleg Davydov Consistency Estimates for gfd 30

44 Kernel-Based Formulas: Error Bounds Theorem For any q max{s, k + 1}, Df(z) Dr X,K,f (z) ρ q,d (z, X)C K,q f FK, f F K, as soon as α,β K(x, y) C(Ω Ω) for α, β q, where ρ q,d (z, X) is the growth function, C K,q := 1 ( ( )( ) q q 1/4 α,β K 2 q! α β C(Ω Ω)) <. α, β =q To compare with the optimal error bound of polynomial approximation: Df(z) N w j f(x j ) ρ q,d (z, X) f,q,ω. Oleg Davydov Consistency Estimates for gfd 30

45 Kernel-Based Formulas: Error Bounds Theorem For any q max{s, k + 1}, Df(z) Dr X,K,f (z) ρ q,d (z, X)C K,q f FK, f F K, as soon as α,β K(x, y) C(Ω Ω) for α, β q, where ρ q,d (z, X) is the growth function, C K,q := 1 ( ( )( ) q q 1/4 α,β K 2 q! α β C(Ω Ω)) <. α, β =q To compare with the optimal error bound of polynomial approximation: Df(z) N w j f(x j ) ρ q,d (z, X) f,q,ω. Robustness: Prior knowledge of the approximation order attainable on X is not needed since estimate holds for all q. Oleg Davydov Consistency Estimates for gfd 30

46 Kernel-Based Formulas: Error Bounds Theorem For any q max{s, k + 1}, Df(z) Dr X,K,f (z) ρ q,d (z, X)C K,q f FK, f F K, as soon as α,β K(x, y) C(Ω Ω) for α, β q, where ρ q,d (z, X) is the growth function, C K,q := 1 ( ( )( ) q q 1/4 α,β K 2 q! α β C(Ω Ω)) <. α, β =q Growth function ρ q,d (z, X) can be evaluated on repeated patterns, to get estimates without unknown constants. E.g., ρ 4, (z, X) = 4h 2 for the five point star, leading to the estimate f(z) r X h,k,f (z) 4h2 C K,4 f FK if the kernel K is sufficiently smooth. Oleg Davydov Consistency Estimates for gfd 30

47 Kernel-Based Formulas: Polyharmonic Formulas Polyharmonic formulas with φ(r) = r ν {log r} and s ν/2 +1 If m := (ν+ d)/2 is integer and s m, they provide optimal recovery in Beppo-Levi space BL m (R d ), the semi-hilbert space generated by m-th order Sobolev seminorm, among all formulas with polynomial exactness order s, and admit error bounds of the type C 1 h ν/2 k z,x for f BL m (R d ). Oleg Davydov Consistency Estimates for gfd 31

48 Kernel-Based Formulas: Polyharmonic Formulas Polyharmonic formulas with φ(r) = r ν {log r} and s ν/2 +1 If m := (ν+ d)/2 is integer and s m, they provide optimal recovery in Beppo-Levi space BL m (R d ), the semi-hilbert space generated by m-th order Sobolev seminorm, among all formulas with polynomial exactness order s, and admit error bounds of the type C 1 h ν/2 k z,x for f BL m (R d ). They are scalable and can therefore be stably computed by upscaling preconditioning for any small radius h z,x. The constant C 1 depends on the power function of the upscaled formula v. Oleg Davydov Consistency Estimates for gfd 31

49 Kernel-Based Formulas: Polyharmonic Formulas Polyharmonic formulas with φ(r) = r ν {log r} and s ν/2 +1 If m := (ν+ d)/2 is integer and s m, they provide optimal recovery in Beppo-Levi space BL m (R d ), the semi-hilbert space generated by m-th order Sobolev seminorm, among all formulas with polynomial exactness order s, and admit error bounds of the type C 1 h ν/2 k z,x for f BL m (R d ). They are scalable and can therefore be stably computed by upscaling preconditioning for any small radius h z,x. The constant C 1 depends on the power function of the upscaled formula v. As any scalable differentiation formulas of exactness order s, they also admit error bounds of the type C 2 h s k z,x for sufficiently smooth f, where C 2 depends on on the Lebesgue constant of v. Oleg Davydov Consistency Estimates for gfd 31

50 Kernel-Based Formulas: Polyharmonic Formulas Polyharmonic formulas with φ(r) = r ν {log r} and s ν/2 +1 If m := (ν+ d)/2 is integer and s m, they provide optimal recovery in Beppo-Levi space BL m (R d ), the semi-hilbert space generated by m-th order Sobolev seminorm, among all formulas with polynomial exactness order s, and admit error bounds of the type C 1 h ν/2 k z,x for f BL m (R d ). They are scalable and can therefore be stably computed by upscaling preconditioning for any small radius h z,x. The constant C 1 depends on the power function of the upscaled formula v. As any scalable differentiation formulas of exactness order s, they also admit error bounds of the type C 2 h s k z,x for sufficiently smooth f, where C 2 depends on on the Lebesgue constant of v. Robust kernel-based estimates are however not applicable. Oleg Davydov Consistency Estimates for gfd 31

51 Outline 1 Generalized Finite Difference Methods 2 Error of Polynomial and Kernel Numerical Differentiation Numerical Differentiation Polynomial Formulas Kernel-Based Formulas Least Squares Formulas 3 Selection of Sets of Influence 4 Conclusion Oleg Davydov Consistency Estimates for gfd 32

52 Least Squares Formulas Discrete Least Squares Let X = {x 1,...,x N } be unisolvent for Π d q (N dimπ d q). The weighted least squares polynomial L θ X,q f Πd q is uniquely defined by the condition (L θ X,q f f) X 2,θ = min { (p f) X 2,θ : p Π d q}, where ( N ) 1/2, v 2,θ := θ j vj 2 θ = [θ1,...,θ N ] T, θ j > 0. Exact for polynomials: L θ X,q p = p for all p Πd q Num. differentiation: Df(z) DL θ X,q f(z) = N w 2,θ j f(x j ) The weights w 2,θ j are scalable. Oleg Davydov Consistency Estimates for gfd 32

53 Least Squares Formulas Dual formulation The weight vector w 2,θ of Df(z) DL θ X,q f(z) = N solves the quadratic minimization problem w 2,θ 2 2,θ 1 = inf w R N w D Π d q w 2 2,θ 1, where θ 1 := [θ 1 1,...,θ 1 N ]T, w 2,θ 1 = It follows that w 2,θ 2,θ 1 = sup ( w 2,θ j f(x j ) w 2 j θ j ) 1/2. { } Dp(z) : p Π d q, p X 2,θ 1 =: ρ q,d (z, X, 2,θ 1), with a generalized growth function. Oleg Davydov Consistency Estimates for gfd 33

54 Least Squares Formulas Theorem Df(z) DL θ X,q f(z) ( N ) ρ q,d (z, X, 2,θ 1) θ j x j z 2q 1/2 f,q,ω 2. In particular, for θ j = x j z 2q 2, Df(z) DL q X,qf(z) Nρ q,d (z, X, 2) f,q,ω, where ( N ρ q,d (z, X, 2) = w 2,q 2,q := (w 2,q j ) ) 2 x j z 2q 1/2 2 Oleg Davydov Consistency Estimates for gfd 34

55 Least Squares Formulas Connection between ρ q,d (z, X) and ρ q,d (z, X, 2) We have ρ q,d (z, X, 2) ρ q,d (z, X) Nρ q,d (z, X, 2). Oleg Davydov Consistency Estimates for gfd 35

56 Least Squares Formulas Connection between ρ q,d (z, X) and ρ q,d (z, X, 2) We have ρ q,d (z, X, 2) ρ q,d (z, X) Nρ q,d (z, X, 2). This implies for the least squares formulas with θ j = x j z 2q 2 an error bound in terms of ρ q,d (z, X): Df(z) DL q X,q f(z) Nρ q,d (z, X) f,q,ω, which is only by factor N worse than the error bound for the 1,q -minimal formula. Oleg Davydov Consistency Estimates for gfd 35

57 Least Squares Formulas Connection between ρ q,d (z, X) and ρ q,d (z, X, 2) We have ρ q,d (z, X, 2) ρ q,d (z, X) Nρ q,d (z, X, 2). This implies for the least squares formulas with θ j = x j z 2q 2 an error bound in terms of ρ q,d (z, X): Df(z) DL q X,q f(z) Nρ q,d (z, X) f,q,ω, which is only by factor N worse than the error bound for the 1,q -minimal formula. We can estimate ρ q,d (z, X) with the help of ρ q,d (z, X, 2), which is cheaper to compute by quadratic minimization or orthogonal decompositions instead of l 1 minimization. Oleg Davydov Consistency Estimates for gfd 35

58 Outline 1 Generalized Finite Difference Methods 2 Error of Polynomial and Kernel Numerical Differentiation Numerical Differentiation Polynomial Formulas Kernel-Based Formulas Least Squares Formulas 3 Selection of Sets of Influence 4 Conclusion Oleg Davydov Consistency Estimates for gfd 36

59 Selection of Sets of Influence Sets of influence: Select Ξ i for each ξ i Ξ\ Ξ ξ i Ξ i ξ i Ξ i is the star or set of influence of ξ i Oleg Davydov Consistency Estimates for gfd 36

60 Selection of Sets of Influence Selection is non-trivial on non-uniform points, especially near domain s boundary (a) adaptive points 12 x x 10-3 (b) 6 nearest points 12 x x 10-3 (c) better selection Points in (a) are obtained by DistMesh (Persson & Strang, 2004) using a theoretically justified (Wahlbin, 1991) density function. Oleg Davydov Consistency Estimates for gfd 37

61 Selection of Sets of Influence: Geometric Selection Geometric selection for Laplacian Choose points in a rather uniform way around ξ i. Four quadrant criterium (Liszka & Orkisz, 1980) (Image from Lyszka, Duarte & Tworzydlo, 1996) Oleg Davydov Consistency Estimates for gfd 38

62 Selection of Sets of Influence: Geometric Selection Choose n = 6 points around ξ i as close as possible to the vertices of an equilateral hexagon (D. & Dang, 2011; Dang, D. & Hoang, 2017): discrete optimization successful for low order methods (O(h 2 ) for Poisson eq.) (n = 6 gives a fair comparison to linear FEM where the rows of the system matrix have 7 nonzeros on average) too complicated to extend to higher order gfd methods Oleg Davydov Consistency Estimates for gfd 39

63 Selection of Sets of Influence Selection via polynomial formulas For a given approximation order smaller sets of influence are preferred since they lead to sparser system matrices Oleg Davydov Consistency Estimates for gfd 40

64 Selection of Sets of Influence Selection via polynomial formulas For a given approximation order smaller sets of influence are preferred since they lead to sparser system matrices This makes a case for 1,µ -minimal formulas Oleg Davydov Consistency Estimates for gfd 40

65 Selection of Sets of Influence Selection via polynomial formulas For a given approximation order smaller sets of influence are preferred since they lead to sparser system matrices This makes a case for 1,µ -minimal formulas It is possible to employ 1,µ -minimal formulas just as a method to select sets of influence, and compute the more robust kernel-based weights on these sets (Bayona, Moscoso & Kindelan, 2011: for Seibold s positive minimal formulas) Oleg Davydov Consistency Estimates for gfd 40

66 Selection of Sets of Influence Selection via polynomial formulas For a given approximation order smaller sets of influence are preferred since they lead to sparser system matrices This makes a case for 1,µ -minimal formulas It is possible to employ 1,µ -minimal formulas just as a method to select sets of influence, and compute the more robust kernel-based weights on these sets (Bayona, Moscoso & Kindelan, 2011: for Seibold s positive minimal formulas) New idea: least squares thresholding Oleg Davydov Consistency Estimates for gfd 40

67 Selection of Sets of Influence: LS Thresholding Least squares thresholding: Compute a least squares numerical differentiation formula, and pick the positions of n largest weights. Example: compare (a) 6 nearest points, (b) 6 positions of nonzero 1,3 -minimal weights of exactness order 3, (c) positions of n = 6 largest weights of a least squares formula of exactness order (a) 6 nearest points (b) 1,3 -minimal (c) LS thresholding Oleg Davydov Consistency Estimates for gfd 41

68 Selection of Sets of Influence: LS Thresholding Test Problem Dirichlet problem for the Helmholz equation u 1 u = f, (α+r) 4 r = x 2 + y 2 in the domain Ω = (0, 1) 2. RHS and the boundary conditions chosen such that the exact solution is sin( 1 α+r Exact solution: ), where α = 1 50π. Oleg Davydov Consistency Estimates for gfd 42

69 Selection of Sets of Influence: LS Thresholding Adaptive nodes from a FEM triangulation by PDE Toolbox x x 10-3 Oleg Davydov Consistency Estimates for gfd 43

70 Selection of Sets of Influence: LS Thresholding RMS Errors of FEM and RBF-FD solutions with Gauss-QR and different selection methods for 6 neighbors 10 1 rms error on interior nodes 10 0 FEM geometric LS threshoding nearest X-axis: number of interior nodes Y-axis: RMS error Oleg Davydov Consistency Estimates for gfd 44

71 Outline 1 Generalized Finite Difference Methods 2 Error of Polynomial and Kernel Numerical Differentiation Numerical Differentiation Polynomial Formulas Kernel-Based Formulas Least Squares Formulas 3 Selection of Sets of Influence 4 Conclusion Oleg Davydov Consistency Estimates for gfd 45

72 Conclusion Polynomial and kernel-based numerical differentiation share similar error bounds that split into factors responsible for the smoothness of the data (e.g. Sobolev norm) and for the geometry of the nodes (Lebesque constant, growth function). Growth function can be efficiently estimated by least squares methods. It may be useful for node generation and selection of sets of influence with prescribed consistency orders of generalized finite difference methods. Sparse sets of influence can be found with the help of 1,µ -minimal polynomial formulas, and more efficiently by least squares thresholding. Oleg Davydov Consistency Estimates for gfd 45