Linear least squares problems and smoothing filters

Transcription

1 Linear least squares problems and smoothing filters Krystyna Ziętak (Wroclaw) Iwona Wróbel (Warsaw) Birmingham, September 14, 2010

2 Outline 1 Savitzky-Golay filters 2 Gram polynomials 3 Persson and Strang filters 4 Experiments 5 Legendre-based filter 6 References

3 Goal of talk is to show that false formula can lead to better method im(z) real(z)

4 Task Given points t 1 <... < t m and measured data corrupted by random noise y 1,...,y m. Compute smoothed value of y i ỹ i = N R j= N L g j y i+j where g j are filter coefficients

5 Concept of Savitzky and Golay (1964) interior point t i : 1+N L i m N R polynomial v(t) approximates data over the set of points {t i NL,...,t i+nr } in the least-squares sense smoothed ỹ i = v(t i )

6 N L = N R = N N ỹ i = v(t i ) = g j y i+j = g T y j= N g = A i (A T i A i ) 1 A T i e A i rectangular Vandermonde, t i N,...,t i+n y = [y i N,...,y i+n ] T e = [0,...,0, 1, 0,...,0] T

7 Uniformly spaced data t j+1 t j = new variable, t h = (t t i )/ new points t j h j = j i t i 0

8 Savitzky-Golay filter coefficients g T = [g N,...g N ] g = A i (A T i A i ) 1 A T i e = = B(B T B) 1 B T e = B(B T B) 1 e 1 B rectangular Vandermonde, N,...,N e 1 = [1, 0,...,0] T, e = [0,...,0, 1, 0,...,0] T

9 1 N ( N) 2 ( N) n 1 N + 1 ( N + 1) 2 ( N + 1) n B = N N 2 N n C = ( N) n ( N) n 1 N 1 ( N + 1) n ( N + 1) n 1 N N n N n 1 N 1

10 How to compute g = B(B T B) 1 e 1? n degree of approx. polynomial C = QR 1 g = Qe n+1 r n+1,n+1 Gander, von Matt

11 Orthogonal (discrete) Gram polynomials orthogonal over point set S N = { N,...,N} q j (h) = j k=0 ( 1) k+j(j +k)(2k) (N +h) (k) (k!) 2 (2N) (k), q 0 (h) = 1, a (0) = 1 a (k) = a(a 1) (a k + 1)

12 discrete Dirac delta δ N = e = [0,...,0, 1, 0,...,0] T R n+1 S N = { N,...,N}, n degree, even polynomial least-squares approximation over S N to δ N q(h) = q n(0)q n+1 (h) ξηh ξ and η constant, depend on n and N filter coefficients g T = [q( N),...,q(N)]

13 orthogonal Legendre polynomials [ 1, 1] recurrence relation P k (s) = 2k 1 k P i,p j = linear change of variable 1 1 P i (s)p j (s)ds. sp k 1 (s) k 1 P k 2 (s), k = 1, 2,... k P 0 (s) = 1. [ 1, 1] [ N,N]

14 Filter coefficients Dirac delta δ(s) Persson-Strang δ(s) = 0 for s 0, δ(s) = for s = 0, least-squares approximation to Dirac delta on interval [ N,N] expressed by Legendre polynomials Savitzky-Golay least-squares approximation to vector e = [0,...,0, 1, 0,...,0] T over the point set { N, N + 1,...,N}

15 Persson and Strang filter (2003) degree of polynomial n even, uniformly spaced data, Persson-Strang polynomial [ N,N] filter coefficients g N,...,g N g j = L(j) L(h) = n+1 2 P n(0) P n+1 ( 2h 2N+1 h ) Optimal Legendre" polynomial g j = K(j) K(h) = n+1 2 P n(0) P ( h n+1 N) h

16 Examples of approximating polynomials n = 2 q(h) = Savitzky-Golay (Gram) 15h 2 (2N 1)(2N + 1)(2N + 3) + 9N 2 + 9N 3 (2N 1)(2N + 1)(2N + 3) Persson-Strang optimal Legendre L(h) = 15h2 (2N + 1) (2N + 1) K(h) = 15h2 8N N

17 0.08 error Opt. Leg error Strang Gram N=16, n= error Opt. Leg error Strang Gram N=100, n= Figure : Savitzky-Golay polynomial q(h) (green); q(h) L(h) and q(h) K(h); N = 16 (on left), right); degree n = 2 differences N = 100 (on

18 Opt Leg error Strang error Savitzky Golya N=16, n= Opt Leg error Strang error Savitzky Golya N=100, n= Figure : Savitzky-Golay polynomial q(h) (green); q(h) L(h) and q(h) K(h); N = 16 (on left), right); degree n = 4 differences N = 100 (on

19 Gander and von Matt test function on [0, 1] F(t) = e 100(t 1 5 )2 +e 500(t 2 5 )2 +e 2500(t 3 5 )2 +e 12500(t 4 5 )2 uniform spaced data; t j = j 1 for j = 1,...,m, where m = 1000 m 1 y j = F(t j )+ε randn, where ε = 0.1 degree of polynomial n = 4 window N = 16 Savitzky-Golay, Persson-Strang, optimal Legendre

20 Figure : F(t) exact and noise (on left); by Savitzky-Golay (on right) F(t) exact and smoothed

21 Figure : F(t) exact and smoothed by optimal Legender (on left); F(t) exact and smoothed by Persson-Strang (on right)

22 Errors: F(t) exact smoothed data (subtraction) Figure : Savitzky-Golay (green) and Persson-Strang (red), (on left); Savitzky-Golay (green), optimal Legendre (red), (on right)

23 Gander-von Matt test function on [0, 0.5] F(t) = e 100(t 1 5 )2 +e 500(t 2 5 )2 +e 2500(t 3 5 )2 +e 12500(t 4 5 )2 uniform spaced data; t j = j 1 for j = 1,...,m, where m = 1000 m 1 y j = F(t j )+ε randn, where ε = 0.1 degree of polynomial n = 4 window N = 32 Savitzky-Golay, Persson-Strang, optimal Legendre

24 Savitzky Golay N=32,n= Figure : F(t) exact and noise (on left); F(t) exact and smoothed by Savitzky-Golay (on right); N = 32, n = 4

25 Errors: F(t) exact smoothed data (subtraction) Error Savitzky Golay error Savitzky Golay Opt Legendre Strang N=32,n=4 N=32,n= Figure : error Savitzky-Golay (blue), optim. Legendre (red), on left; error Savitzky-Golay (blue), Persson-Strang (red), on right, N = 32,n = 4

26 Persson and Strang write that the Legendre-based filters have extra advantages: in the case of irregularly spaced or missing data the polynomials stay the same and it is only the sampling points that change. (...) The simplicity becomes especially valuable when the input no longer consists of uniformly spaced samples. The output from the new filter will be the natural non-uniform generalization of an ordinary convolution". [t i N,t i+n ] [ N,N]

27 window t i NL,...,t i+nr new variable h = t t i t, new points t = t i+n R t i NL N L +N R h (i) j = t j t i t for j = i N L,...i +N R. α i = h (i) i N L, β i = h (i) i+n R approximation of Dirac delta on [α i,β i ]

28 P k (t) Legendre polynomial [ 1, 1] [α i,β i ] Z (i) k (h) = P k ( 2 h+ α ) i +β i β i α i α i β i K (i) (h) = n+1 2 Z(i) n+1 (0)Z(i) n (h) Z n (i) (0)Z (i) n+1 (h) h ỹ i = i+n R g (i) j y j, where g (i) j j=i N L = K (i) (h (i) j )

29 Bad news

30 A. Eisinberg, P. Pugliese, N. Salerno, Numer. Math Vandermonde matrices on integer nodes: the rectangular case their case 1, 2,...,N B = V T V Hankel matrix explicit Cholesky factor of B pseudo-inverse of V combinatorial identities our case N,...,N

31 References Abraham Savitzky ( ) was an Americal analytical chemist. He specialized in the digital processing of infrared spectra. Marcel J.E. Golay ( ) was a Swiss-born mathematician, physicst and information theorist. Golay error-correcting codes Savitzky coauthored with Marcel J.E. Golay an oft-cited paper describing the Savitzky-Golay smoothing filter.

32 THANK YOU FOR YOUR ATTENTION