On Two Different Signal Processing Models

Transcription

1 On Two Different Signal Processing Models Department of Mathematics & Statistics Indian Institute of Technology Kanpur January 15, 2015

2 Outline First Model 1 First Model

3 Outline First Model 1 First Model

4 Introduction First Model We observe periodic phenomena everyday in our lives. For example the number of tourists visiting the famous Taj Mahal, the daily temperature of Delhi or the ECG data of a normal human being clearly follow periodic pattern. Sometimes the data may not be exactly periodic but it is nearly periodic. Our aim is to analyze such periodic/ nearly periodic data.

5 Question? First Model 1 What is a periodic data? 2 Why do we care to analyze?

6 What is a periodic data? We do not give the formal definition. But informally speaking 1 It shows a repeated (periodic) pattern in one dimension. 2 It shows a symmetric (periodic) pattern in higher dimension.

7 Why do we want to analyze? 1 Theoretical reason. 2 Prediction purposes. 3 Compression purposes.

8 Example: Airlines Passenger Data Airline passengers data x(t) > t >

9 Example: Brightness of Variable Star Data y(t) t

10 Example: Vowel Sound Data uuu y(t) t

11 Example: ECG Data of a Normal Human 700 Original Signal y(m) m

12 Example: Two Dimension Periodic Data..

13 Example: Three Dimension Periodic Data..

14 Example: Three Dimension Periodic Data..

15 Outline First Model 1 First Model

16 Simplest Periodic Function The simplest periodic function is the sinusoidal function, and it can be written in the following form: y(t) = Acos(ωt)+Bsin(ωt) The period of y(t) is the shortest time taken for y(t) to repeat itself, and it is 2π/ω.

17 Smooth Periodic Function In general a smooth periodic function (mean adjusted) with period 2π/ω, can be written in the form: y(t) = [A k cos(ωkt)+b k sin(ωkt)], k=1 and it is well known as the Fourier expansion of y(t).

18 Extracting Parameters From y(t), A k and B k can be obtained uniquely. and 2π/ω 0 cos(jωt)y(t)dt = 2π/ω 0 πa j ω if j 1 2πA 0 ω if j = 0 sin(jωt)y(t)dt = πb j ω.

19 Noisy Periodic Function Most of the times y(t) is corrupted with noise, so we observe the following: y(t) = [A k cos(ωt)+b k sin(ωt)]+x(t), k=1 where X(t) is the noise component.

20 Practical Model It is impossible to estimate infinite number of parameters. Hence the model is approximated by the following model: y(t) = for some p <. p [A k cos(ω k t)+b k sin(ω k t)]+x(t), k=1

21 Model First Model The model has two components, 1 Deterministic component 2 Random component

22 Aim First Model The aim is to extract (estimate) the deterministic component µ(t), where p µ(t) = [A k cos(ω k t)+b k sin(ω k t)], k=1 in presence of the random error component X(t), based on the available data y(t),t = 1,...,N.

23 Problem Formulation Based on the available data {y(t);t = 1,...,N}, 1 Deterministic Component Determine (estimate) p Determine (estimate) A 1,...,A p, B 1,...,B p Determine (estimate) ω 1,...,ω p. 2 Random Component Estimate X(t)

24 Procedure First Model 1 Assume certain structure on X(t) 2 Estimate the deterministic component µ(t) 3 Estimate the error X(t) 4 Verify the assumption. 5 If the assumption is satisfied then stop the process, otherwise go back to step 1.

25 Outline First Model 1 First Model

26 Periodogram Estimators The most used and popular estimation procedure is the periodogram estimators. The periodogram at a particular frequency is defined as ( 2 ( 1 N 1 N I(θ) = y(t)cos(θt)) + y(t) sin(θt) N N t=1 t=1 t=1 ( 2 ( 1 N 1 N µ(t)cos(θt)) + µ(t) sin(θt) N N t=1 ) 2 ) 2

27 Periodogram Estimator Consider the following sinusoidal signal: Sinusoidal Example 1: y(t) = 3.0(cos(0.2πt)+sin(0.2πt))+3.0(cos(0.5πt)+sin(0.5πt))+X(t) Here X(t) s are i.i.d. N(0,0.5)

28 Examples: Sinusoidal Signal

29 Periodogram Estimator Consider the following sinusoidal signal: Sinusoidal Example 2: y(t) = 3.0(cos(0.2πt)+sin(0.2πt))+0.25(cos(0.5πt)+sin(0.5πt))+X(t) Here X(t) s are i.i.d. N(0,2.0)

30 Examples: Sinusoidal Signal

31 Least Squares Estimators Assuming p is known, the most natural estimators will be the least squares estimators and they can be obtained as follows: ( [ n p ]) 2 y(t) A k cos(ω k t)+b k sin(ω k t) t=1 k=1

32 Numerical Issues First Model 1 It is a highly non-linear problem. The least squares surface has several local minima. 2 Most of the time the standard Newton-Raphson algorithm may not converge. 3 Even if they converge, often it converges to the local minimum rather than the global minimum. 4 If p is large, it becomes a higher dimensional optimization problem, extremely accurate initial guesses are required for any iterative procedure to work well.

33 Theoretical Issues First Model 1 It can be treated as a standard non-linear regression problem as follows: y(t) = f t (θ)+x(t) 2 Unfortunately it does not satisfy the standard sufficient conditions of Wu (1981) or Jennrich (1969) for the consistency of the least squares estimators. 3 It can be shown that the least squares estimators are consistent. 4 Â k and B k have the convergence rate n 1/2, where as ω k has the convergence rate n 3/2

34 Sequential Estimation Procedures It is based on the facts that the components are orthogonal and it works like this First minimize n (y(t) Acos(ωt) Bsin(ωt)) 2 t=1 with respect to A, B and ω. Take out their effect from y(t), i.e. consider ỹ(t) = y(t) Âcos( ωt) Bsin( ωt) Repeat the procedure p times.

35 Advantage First Model It reduces the computational burden significantly. For example if p = 25, instead of solving a 25 dimensional optimization problem, we need to solve 25 one dimensional optimization problems. It does not have any problem about initial guess or convergence. It produces the same accuracy as the least squares estimators.

36 Super Efficient Estimators When p = 1, the Newton-Raphson algorithm will be of the following form: ω (j+1) = ω (j) Q (ω) Q (ω) It has been suggested ω (j+1) = ω (j) 1 Q (ω) 4Q (ω) It not only converges, it produces estimators which are better than the least squares estimators.

37 Main Theoretical Results 1 Least squares estimators are consistent under mild assumptions on the errors. 2 Least squares estimators have the convergence rate N 3/2. 3 Sequential estimators have the same convergence rate as the least squares estimators. 4 Asymptotic variances of the super efficient estimators are smaller than the least squares estimators. 5 Periodogram estimators are consistent, but it has the convergence rate N 1/2.

38 Outline First Model 1 First Model

39 Chirp Signal Model It has the following mathematical form: Simple Chirp Model y(t) = Acos(αt +βt 2 )+B sin(αt +βt 2 )+X(t) General Chirp Model p { y(t) = Ak cos(α k t +β k t 2 )+B k sin(α k t +β k t 2 ) } +X(t) k=1

40 Several Applications 1 Chirps are naturally encountered in many audio signals, ranging from bird songs, music to animal vocalization and speech. 2 Radar and Sonar systems: Chirp signals are also commonly observed in natural sonar systems. 3 Biology and Medicine: Chirp models have been used to analyze EEG signals.

41 Theoretical Consideration 1 Least squares estimators are consistent under finiteness of the fourth order moment conditions on the error random variables. 2 Least squares estimators of the amplitude has the convergence rate N 1/2. 3 Least squares estimators of the frequencies have the convergence rate N 3/2. 4 Least squares estimators of the chirp parameters have the convergence rate N 5/2. 5 Least squares estimators are asymptotically normally distributed.

42 One Number Theory Result: If (θ 1,θ 2 ) (0,π) (0,π), then except for countable number of points the following results are true: lim N lim N 1 N t+1 1 N N cos(θ 1 n+θ 2 n 2 ) = 0 n=1 N n t cos 2 (θ 1 n+θ 2 n 2 ) = n=1 1 2(t +1). lim N 1 N t+1 N n t sin(θ 1 n+θ 2 n 2 )cos(θ 1 n+θ 2 n 2 ) = 0. n=1

43 Sequential Estimator Based on the above number theory results it can be shown that the sequential estimators are consistent.

44 One Number Theory Conjecture If θ 1,θ 2,θ 1,θ 2 (0,π), then except for countable number of points the following results are true: lim N 1 NN t N n t cos(θ 1 n+θ 2 n 2 )sin(θ 1n+θ 2n 2 ) = 0 n=1

45 Asymptotic Distribution of the Sequential Estimators Based on the above conjecture, it can be shown that the sequential estimators and the least squares estimators are equivalent.

46 Two dimensional Chirp Signals y(m,n) = Acos(α 1 m+β 1 m 2 +α 2 n+β 2 n 2 )+ B sin(α 1 m+β 1 m 2 +α 2 n+β 2 n 2 )+X(m,n) Results can be extended to two dimensional chirp signals models.

47 Outline First Model 1 First Model

48 Z.D. Bai Li Bai Swagata Nandi Ananya Lahiri Amit Mitra Anurag Prasad

49 Outline First Model 1 First Model

50 First Model Kundu, D. and Nandi, S. (2008), Parameter estimation on chirp signals in presence of stationary noise, Statistica Sinica. Nandi, S. and Kundu, D. (2004), Asymptotic properties of the least squares estimators of the parameters of the chirp signals, Annals of the Institute Statistical Mathematics. Kundu, D., Bai. Z.D., Nandi, S. and Bai, L. (2011), Super efficient frequency estimation, Journal of Statistical Planning and Inference.

51 First Model Ananya Lahiri, D. Kundu, and A. Mitra (2012) Efficient algorithm for estimating the parameters of chirp signal Journal of Multivariate Analysis. Ananya Lahiri, D. Kundu, and A. Mitra Estimating the parameters of multiple chirp signals Journal of Multivariate Analysis. Ananya Lahiri, D. Kundu, Amit Mitra (2014), On least absolute deviation estimator of one dimensional chirp model, Statistics.

52 First Model B.G. Quinn and E.J. Hannan (2001), The estimation and tracking of frequency, Cambridge University Press. D. Kundu and S. Nandi (2012), Statistical signal processing: Frequency Estimation.

53 THANK YOU