Radial Basis Functions I

Transcription

1 Radial Basis Functions I Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo November 14, 2008

2 Today Reformulation of natural cubic spline interpolation Scattered data interpolation with RBF s Franke s example using Gaussian Non-singularity via nonnegative Fourier Transform RBF interpolation with polynomial precision Positive definite matrices on a subspace.

3 Cubic Splines Given a < b and x = [x 0,..., x N+1 ] with a =: x 0 < x 1 < < x N < x N+1 := b. Cubic Splines: S 2 3(x) := {g C 2 [a, b] : g (xj,x j+1 ) Π 3, for j = 0,..., N Π d := span{x k : 0 k d} polynomials of degree d. Unique representation of g S 2 3 in terms of truncated powers x 3 + := max{x 3, 0}. x 3 + C 2 (R). g(x) = N k=1 c k(x x k ) k=0 a kx k, x [a, b]. S 2 3(x) is a linear space of functions of dimension N + 4.

4 Natural Cubic Splines NS 2 3(x) := {g S 2 3(x) : g (a,x1 ), g (xn,b) Π 1}. Truncated power representation g NS 2 3(x) g(x) = N k=1 c k(x x k ) a 0 + a 1 x, x [a, b], N k=1 c k = 0, N k=1 c kx k = 0. Proof Choosing x (a, x 1 ) shows that a 2 = a 3 = 0 in the truncated power representation. Choosing x (x N, b) shows the orthogonality conditions. remove + s: g(x) = N k=1 c k(x x k ) 3 + a 0 + a 1 x, x (x N, b). Expand (x xk ) 3 = x 3 3x k x 2 + 3xk 2x x k 3 Expanded Sum: k c k(x x k ) 3 = x 3 k c k 3( k c kx k )x 2 + 3( k c kxk 2)x ( k c kxk 3).

5 Radial basis function representation truncated power representation: g NS 2 3(x) g(x) = N k=1 c k(x x k ) a 0 + a 1 x, x [a, b], N k=1 c k = 0, N k=1 c kx k = 0. Replace x+ 3 by 1 2 ( x 3 + x 3 ) in truncated power representation. g(x) = 1 N 2 k=1 c k x x k N 2 k=1 c k(x x k ) 3 + a 0 + a 1 x, x [a, b], g(x) = 1 N 2 k=1 c k x x k 3 + 3( N 2 k=1 c kxk 2)x 1 ( N 2 k=1 c kxk 3) + a 0 + a 1 x, x [a, b], radial basis function representation g(x) = N k=1 c k x x k 3 + d 0 + d 1 x, x [a, b], N k=1 c k = 0, N k=1 c kx k = 0.

6 Natural cubic spline interpolation in a radial basis formulation Find P f NS 2 3(x) such that P f (x j ) = f j, j = 1,..., N. N k=1 c kϕ( x j x k ) + d 0 + d 1 x j = f j, j = 1,..., N k c k = 0, k c kx k = 0. Matrix form [ ] [ ] A B T c = B 0 d N + 2 equations in N + 2 unknowns. [ ] [ ] f 1 1, A = [ϕ( x 0 j x k )], B =. x 1 x N

7 Radial Function in R s Definition Let s N and a norm on R s. A function Φ : R s R is called radial if Φ(x) = ϕ( x ), x R s, for some univariate function ϕ : [0, ) R. The norm is often the Euclidian norm 2. = 2 when nothing else is said. Radial: Φ(x) = ϕ(r) for all x with x = r.

8 Gaussian, ϕ(r) = e ε2 r 2, ε = 2, Φ(x) = ϕ( x ) = e ε2 x 2

9 Distance, ϕ(r) = r Φ(x) = ϕ( x ) = x 2 + y 2

10 Thin Plate,ϕ(r) = r 2 log r Φ(x) = ϕ( x ) = 1 2 (x 2 + y 2 ) log(x 2 + y 2 ). 4 Φ(x) = 0.

11 RBF Interpolation without polynomial precision Given N, s N and distinct points x 1,..., x N R s, ordinate-values f j = f (x j ) representing an unknown function f. A radial function Φ : R s R given by Φ(x) = ϕ( x ) Linear combinations P f (x) := N k=1 c KΦ(x x k ) Find c = [c 1,..., c N ] such that P f (x j ) := N c k Φ(x j x k ) = f j, k=1 j = 1,..., N Matrix Problem Ac = f, where A = [ϕ( x j x k )] R N,N, f = [f 1,..., f N ] T.

12 N = 2 [ ] ϕ( x1 x A = 1 ) ϕ( x 1 x 2 ). ϕ( x 2 x 1 ) ϕ( x 2 x 2 ) ϕ(r) = e ε2 r 2, A = [ ] 1 a, a = e a 1 ε2 x 1 x 2 2 < 1 if ε 0. A is symmetric and positive definite. (Positive eigenvalues by Gerschgorin or strict diagonal dominance). [ ] 0 b ϕ(r) = r, A =, b = x b 0 1 x 2. A is symmetric with one positive and one negative eigenvalue. Nonsingular, but indefinite. Same for thin plate

13 Franke s test function

14 Franke s test function f (x, y) :=0.75 Exp ( ((9 x 2) 2 + (9 y 2) 2 )/4 ) Exp ( ((9 x + 1) 2 /49 + (9 y + 1) 2 /10) ) + Exp ( ((9 x 7) 2 + (9 y 3) 2 )/4 ) 0.2 Exp ( ((9 x 4) 2 + (9 y 7) 2 ) )

15 Halton Points x 1,..., x N uniformly distributed points in (0, 1) s. can be downloaded by searching for haltonseq.m Halton Points in R 2

16 Gaussian Example Φ(x) = e ε2 x 2, ε = (left) and 1089 right) Halton Points f sampled from Franke s test function. Matlab program RBFInterpolation2D.m by Fasshauer RBF interpolant false colored by maximum error. RBF interpolant false colored by maximum error Error Error

17 Discussion Dense linear system Non-singular? Extremely ill-conditioned for ε not large. Polynomials of low degree not reproduced. But.. Can work for scattered data in high space dimension without triangulating the data.

18 Non-singularity via Fourier Transform Definition For a function f L 1 (R s ) we define the (symmetric) Fourier transform of f by ˆf (ω) := 1 (2π) s/2 R s f (x)e iω x dx, ω R s. (1) and the inverse Fourier transform 1 f (x) = ˆf (ω)e iω x dω, (2π) s/2 R s x R s. (2) Φ(x) := e ε2 x 2 ˆΦ(ω) := e ω 2 /(4ε 2) 0.

19 Non-negative Fourier transform Theorem Let Φ(x) = ϕ( x ) be a radial function with nonnegative Fourier transform not identically zero. For any distinct points x 1,..., x N the matrix is positive definite. A := [ϕ( x j x k )] R N,N,

20 Proof c T Ac = N j=1 N c j c k Φ(x j x k ) k=1 1 = c (2π) s/2 j c k ˆΦ(ω)e j,k R iω (x j x k ) dω s 1 ( ) = cj e iω x j c (2π) s/2 k e iω x k ˆΦ(ω)dω = 1 (2π) s/2 R s j,k R s j c j e iω x j 2 ˆΦ(ω)dω 0. Equality j c j e iω x j = 0, ω R s, c = 0.

21 Discussion The Fourier transform can be used for some other examples. But, The distance function and thin plate and other examples are not integrable so do not have a Fourier transform Can use nonnegativity of the generalized Fourier transform. We will instead look at the connection between positive definite matrices and completely monotone functions.

22 Polynomial reproduction Gaussian, distance and thin plate do not reproduce polynomials can add terms to achieve this. Example x = (x, y) R 2, add 1, x, y to reproduce linear polynomials P f (x) := N k=1 c kϕ( x x k ) + d 1 + d 2 x + d 3 y, need 3 extra conditions k c k = 0, k c kx k = 0, k c ky k = 0

23 Linear system N k=1 c kϕ( x j x k ) + d 1 + d 2 x j + d 3 y j = f j, j = 1,..., N k c k = 0, k c kx k = 0, k c ky k = 0 [ A B T B 0 ] [ ] c = d [ ] f, A = [ϕ( x 0 j x k )], B T = S 1 := {c R N : Bc = 0}, is a subspace of R N. 1 x 1 y x N y N

24 Positive definite on a subspace Definition Suppose A R N,N and S a subspace of R N. We say that A is positive definite on S if A T = A and c T Ac > 0 for all nonzero c S. If A is positive definite on S = ker(b) := {c R N : Bc = 0} and B R M,N has linearly independent rows then the block matrix [ ] A B T is nonsingular. B 0

25 Proof [ A B T B 0 ] [ ] c = 0 Ac + BT d = 0 d Bc = 0 ct Ac + c T B T d = 0 c T B T = 0. So c = 0 and then d = 0 since 0 = Ac + B T d = B T d and B has linearly independent rows.

26 Positive and negative eigenvalues Theorem If A R N,N is positive definite on the subspace S := {c R N : k c k = 0} and i a ii 0, then A has N 1 positive eigenvalues and one negative eigenvalue. Proof S is a subspace of R N of dimension N 1. Since A is symmetric it has real eigenvalues. Let λ 1 λ 2 λ N be the eigenvalues of A. Recall Courant-Fischer s characterization In particular λ k = max min c T Ac. dim T =k c T c =1 λ N 1 min c T Ac > 0. c S c =1 But i λ i = i a ii 0 so λ N < 0.

27 Non-singularity without polynomial precision Corollary Let Φ(x) := ϕ( x ) be a radial function. If A = [ϕ( x j x k )] R N,N is positive definite on the subspace S := {c R N : k c k = 0} and ϕ(0) = 0 then A is non-singular with N 1 positive eigenvalues and one negative eigenvalue. Proof. This follows immediately from the previous theorem.

28 Distance matrix ϕ(r) = r, Φ(x) = ϕ( x ) = x. A = [ x j x k ] and a jj = x j x j = 0, j = 1,..., N since ϕ(0) = 0 so j a jj 0. We show next time that A is positive definite on S := {c R N : k c k = 0}. It will then follow that the distance matrix is non-singular with N 1 positive and one negative eigenvalue.