Estimating Periodic Signals

Transcription

1 Department of Mathematics & Statistics Indian Institute of Technology Kanpur Most of this talk has been taken from the book Statistical Signal Processing, by D. Kundu and S. Nandi. August 26, 2012

2 Outline 1 Introduction 2 3 Least Squares and Approximate Least Squares Estimators

3 Outline 1 Introduction 2 3 Least Squares and Approximate Least Squares Estimators

4 Some Definitions: What is a Signal? A signal is a function that conveys information about the behavior or attributes of some phenomenon (Wikipedia)

5 Some Definitions: Different Examples: 1 Daily Gold price. 2 Monthly expenditure in a family 3 ECG signal of a human being. 4 Satellite images. 5 Textures

6 Some Definitions: What is Signal Processing? Signal Processing may broadly be considered to involve the recovery of information from physical observations. The received signal is usually disturbed by external or internal noises. Due to random nature of the signal, statistical techniques play important roles in analyzing the signals.

7 Examples: ECG Signal

8 Examples: Rumford Data

9 Example: Vowel Sound

10 Example: Variable Star Brightness Signal

11 Example: Airlines Passenger Data

12 Some More Definition What is a Periodic Signal? A signal (function) which repeats after a fixed period of time. f (t) = f (t ); where t = t mod T Example: y(t) = A cos(ωt) + B sin(ωt).

13 Fourier Transform A smooth mean zero periodic function can be written as y(t) = A k cos(kωt) + B k sin(kωt), k=1 and it is known as the Fourier expansion of y(t). Most of the times y(t) is corrupted with noise, hence we use y(t) = A k cos(kωt) + B k sin(kωt) + X (t) k=1

14 Sinusoidal Signal Since it is impossible to estimate infinite number of parameters, the following model has been used y(t) = p A k cos(ω k t) + B k sin(ω k t) + X (t); k=1 where p <. Often the problem boils down to estimate A k s, B k s, ω k s and p based on a sample of size n, namely y(1),..., y(n).

15 Outline 1 Introduction 2 3 Least Squares and Approximate Least Squares Estimators

16 Periodogram Estimators The most used and popular estimation procedure is the periodogram estimators. The periodogram at a particular frequency is defined as or equivalently I (ω) = 1 n 2 y(t)e iωt n t=1 ( n ) 2 ( n I (ω) = 1 y(t) cos(ωt) + 1 y(t) sin(ωt) n n t=1 t=1 ) 2

17 Periodogram Estimator Consider the following sinusoidal signal: Sinusoidal Example 1: y(t) = 3.0(cos(0.2πt)+sin(0.2πt))+3.0(cos(0.5πt)+sin(0.5πt))+X (t) Here X (t) s are i.i.d. N(0,0.5)

18 Examples: Sinusoidal Signal

19 Periodogram Estimator Consider the following sinusoidal signal: Sinusoidal Example 2: y(t) = 3.0(cos(0.2πt)+sin(0.2πt))+0.25(cos(0.5πt)+sin(0.5πt))+X (t) Here X (t) s are i.i.d. N(0,2.0)

20 Examples: Sinusoidal Signal

21 Outline 1 Introduction 2 3 Least Squares and Approximate Least Squares Estimators

22 Least Squares Estimators The model can be seen as a non-linear regression model: where y(t) = f t (θ, p) + X (t) p f t (θ, p) = A k cos(ω k t) + B k sin(ω k t) k=1

23 Least Squares Estimators Assuming p is known, the most natural estimators will be the least squares estimators and they can be obtained as follows: ( [ n p ]) 2 y(t) A k cos(ω k t) + B k sin(ω k t) t=1 k=1

24 Theoretical and Numerical Issues 1 It does not satisfy the standard sufficient condition of Jennrich or Wu. 2 The least squares may not be consistent. 3 The asymptotic distribution of the least squares estimators are not n consistent. 4 Numerically it is a challenging problem.

25 Separable Regression Technique The model can be written as follows Y = A(θ)β + e where cos(ω 1 ) sin(ω 1 )... cos(ω p ) sin(ω p ) A(θ) =..... cos(nω 1 ) sin(nω 1 )... cos(nω p ) sin(nω p ) β T = (A 1, B 1,..., A p, B p ), e T = (X (1),..., X (n)).

26 Separable Regression Technique The least squares estimators can be obtained by minimizing Q(θ, β) = (Y A(θ)β) T (Y A(θ)β) with respect to the unknown parameters. Note that if θ is known then β T (θ) = (A(θ) T A(θ)) 1 A T (θ)y.

27 Separable Regression Technique The least squares estimators of θ can be obtained by minimizing Q(θ, β(θ)) = with respect to θ. It is equivalent in saying minimize ( ) T ( ) Y A(θ) β T (θ) Y A(θ) β(θ) Q(θ) = Y T (I P A )Y, where P A is the projection matrix as P A = A(A T A) 1 A T

28 Approximate Least Squares Estimators Note that minimizing Q(θ) = Y T (I P A )Y, is equivalent to maximizing R(θ) = Y T P A Y. Approximate 1 n (AT A) = I. Therefore, ( n ) 2 ( n Q(θ) = 1 n Y T AA T Y = 1 y(t) cos(ωt) + 1 y(t) sin(ωt) n n t=1 t=1 ) 2

29 Outline 1 Introduction 2 3 Least Squares and Approximate Least Squares Estimators

30 Complex Exponential The sum of sinusoidal model has a very close resemblance with the corresponding model y(t) = p A k e iωkt + X (t) k=1 Here y(t) s are complex valued, A k and B k are complex valued, 0 < ω k < 2π. The problem remains the same, estimate the unknown parameters based on y(t) s.

31 Prony s Equation Prony in 1795 observed the following interesting facts: If µ(t) = p A k e βkt ; t = 1,..., n, k=1 here A k s and β k s are real and β k s are distinct, then there exists g 0,..., g p such that µ(1)... µ(p + 1) g 0 0 µ(2)... µ(p + 2) g µ(n p)... µ(n). g p =. 0

32 Prony s Equation The constants g 0,..., g p do not depend on A k s, they depend only on β k s. β k s can be obtained from g k s as follows: Consider the following polynomial equation g 0 + g 1 x g p x p = 0, then e β 1,..., e βp are the roots of the above polynomial equations. Once β k s are obtained, A k s are obtained using simple linear regression method.

33 Prony s Equations Similar results are true in case of complex exponential also, i.e. if µ(t) = p A k e βkt ; t = 1,..., n, k=1 here A k s and β k s are complex. Similarly, if µ(t) = p A k cos(ω k t) + B k sin(ω k t); t = 1,..., n, k=1 here A k s and B k s are real, and 0 < ω k < 2π.

34 Prony s Equation It is immediate that if there is no error then A k s and β k can be recovered from µ(t) s without any problem.. Now suppose p y(t) = A k e βkt + e(t); t = 1,..., n, k=1 here A k s and β k s are real and β k s are distinct, and e(t) s are small mean zero error. Then it is expected y(1)... y(p + 1) g 0 0 y(2)... y(p + 2) g y(n p)... y(n) g p 0

35 Prony s Equation Therefore, if y(1)... y(p + 1) g 0 y(2)... y(p + 2) A =... g = g 1. y(n p)... y(n) g p then we want to solve Ag = 0 A T Ag = 0 g is an eigen vector corresponds to 0 eigenvalue of A T A.

36 Outline 1 Introduction 2 3 Least Squares and Approximate Least Squares Estimators

37 Numerical Issues 1 It is a highly non-linear problem. The least squares surface has several local minima. 2 Most of the time the standard Newton-Raphson algorithm may not converge. 3 Even if they converge, often it converges to the local minimum rather than the global minimum. 4 If p is large, it becomes a higher dimensional optimization problem, extremely accurate initial guesses are required for any iterative procedure to work well.

38 Sequential Estimation Procedures It is based on the facts that the components are orthogonal and it works like this First minimize n (y(t) A cos(ωt) B sin(ωt)) 2 t=1 with respect to A, B and ω. Take out their effect from y(t), i.e. consider ỹ(t) = y(t) Â cos( ωt) B sin( ωt) Repeat the procedure p times.

39 Advantage It reduces the computational burden significantly. For example if p = 25, instead of solving a 25 dimensional optimization problem, we need to solve 25 one dimensional optimization problems. It does not have any problem about initial guess or convergence. It produces the same accuracy as the least squares estimators.

40 Super Efficient Estimators When p = 1, the Newton-Raphson algorithm will be of the following form: ω (j+1) = ω (j) Q (ω) Q (ω) After few pages of calculations it has been suggested ω (j+1) = ω (j) 1 Q (ω) 4 Q (ω) It not only converges, it produces estimators which are better than the least squares estimators.

41 Outline 1 Introduction 2 3 Least Squares and Approximate Least Squares Estimators

42 Main Asymptotic Results 1 Least squares estimators are consistent under mild assumptions on the errors. 2 Least squares estimators have the convergence rate n 3/2. 3 Sequential estimators have the same convergence rate as the least squares estimators. 4 Asymptotic variances of the super efficient estimators are smaller than the least squares estimators. 5 Prony s estimators are not consistent. 6 Periodogram estimators are consistent, but it has the convergence rate n 1/2.

43 Outline 1 Introduction 2 3 Least Squares and Approximate Least Squares Estimators

44 1 Consider the number of peaks of the periodogram function. 2 It can be very misleading. 3 In the least squares procedure, consider residual sums of squares. 4 It can be very misleading too. 5 Information theoretic criterion. 6 Cross validation technique. 7 Likelihood ratio approach.

45 Information Theoretic Criterion AIC(k) = n ln R k + 2(3k) BIC(k) = n ln R k + 1 ln n(3k) 2 EDC(k) = n ln R k + C n k. Here C n satisfies certain conditions namely C n n C n. ln ln n Choose that model for which AIC(k), BIC(k) or EDC(k) is minimum

46 Information Theoretic Criterion Which C n to choose? Resampling technique can be used to compute PCS for each C n and choose that C n for which the PCS is maximum.

47 Outline 1 Introduction 2 3 Least Squares and Approximate Least Squares Estimators

48 Compartment Model Consider the following real valued model: y(t) = p A k e βkt + e(t); k=1 t = 1,..., n Here A k s and β k s are real numbers. The number of components p may be known or unknown. The problem is to estimate A k s and β k s based on y(t) s.

49 Fundamental Frequency Model Consider the following model: y(t) = p [A k cos(kλt) + B k sin(kλt)] + e(t) k=1 Here λ is the fundamental frequency, and it has p harmonics. The problem remains the same.

50 Chirp Signal Model Consider the following model: p [ y(t) = Ak cos(λ k t + β k t 2 ) + B k sin(λ k t + β k t 2 ) ] + e(t) k=1 The problem is to estimate the frequency and frequency rates.

51 Partially Sum of Sinusoidal Model Consider the following model: p y(t) = a + bt + [A k cos(ω k t) + B k sin(ω k t)] + e(t) k=1 The problem is to estimate the unknown parameters.

52 Thank You