An Efficient and Fast Algorithm for Estimating the Parameters of Two-Dimensional Sinusoidal Signals

Transcription

1 isid/ms/8/ November 6, 8 statmath/eprints An Efficient and Fast Algorithm for Estimating the Parameters of Two-Dimensional Sinusoidal Signals Swagata Nandi Anurag Prasad Debasis Kundu Indian Statistical Institute, Delhi Centre 7, SJSS Marg, New Delhi 6, India

2

3 An Efficient and Fast Algorithm for Estimating the Parameters of Two-Dimensional Sinusoidal Signals Swagata Nandi, Anurag Prasad & Debasis Kundu Abstract In this paper we propose a computationally efficient algorithm to estimate the parameters of a -D sinusoidal model in presence of stationary noise. The estimators obtained by the proposed algorithm are consistent and asymptotically equivalent to the least squares estimators. Monte Carlo simulations are performed for different sample sizes and it is observed that the performances of the proposed method are quite satisfactory and they are equivalent to the least squares estimators. The main advantage of the proposed method is that the estimators can be obtained using only finite number of iterations. In fact it is shown that starting from the average of periodogram estimators, the proposed algorithm converges in three steps only. One synthesized texture data and one original texture data have been analyzed using the proposed algorithm for illustrative purpose. Keywords: Sinusoidal signals; Least squares estimators; Asymptotic distribution; Twodimensional frequency estimation; Bayesian Information Criterion; Efficient algorithm. Theoretical Statistics and Mathematics Unit, Indian Statistical Institute, Delhi Center, Pin 6, INDIA. Department of Mathematics and Statistics, Indian Institute of Technology Kanpur, Kanpur, Pin 86, INDIA. Corresponding Author: Swagata Nandi, nandi@isid.ac.in.

4 Introduction In this paper we consider the problem of estimating the parameters of the following two dimensional (-D) sinusoidal signal; y(m,n) = A cos(λ m + µ n) + B sin(λ m + µ n)] + X(m,n). () Here A and B are unknown real numbers, known as amplitudes, λ and µ are unknown frequencies. It is assumed that A +B >, and λ,µ (,π). The additive error {X(m,n)} is from a stationary random field. The explicit assumptions on {X(m, n)} and also on the model parameters are provided in section. The main problem is to estimate the unknown parameters, namely A, B, λ and µ, given a sample {y(m,n);m =,...,M,n =,...,N}. The first term on the right hand side of () is known as the signal component and the second term as the noise or error component. The detection and estimation of the signal component in presence of additive noise is an important and classical problem in Statistical Signal Processing. Particularly, the -D sinusoidal model has received a considerable attention in the signal processing literature because of its widespread applicability in texture synthesis. Francos et al. ] first observed that the -D sinusoidal model can be used quite effectively to analyze symmetric texture images. For some of the theoretical developments of the -D sinusoidal or related models, the readers are referred to Rao et al. 8], Zhang and Mandrekar ] and Kundu and Nandi 4]. The -D frequency estimation is well known to be a numerically difficult problem. The problem becomes more severe particularly if p is quite large. The most efficient estimators, as expected are the least squares estimators. The order of convergence of the least squares estimators of λ s and µ s are O p (M 3 N ) and Op (M N 3 ) respectively. Here U = Op (M δ N δ ) means M δ N δ U is bounded in probability. Finding the least squares estimator tends to be computationally intensive as the functions to be optimized are highly non-linear in parameters. Even in one dimension, it is known that the least squares surface has several local minima, see

5 Rice and Rosenblatt 9]. Recently Prasad et al. 6] proposed a sequential procedure to estimate the unknown parameters of one dimensional sinusoidal model and which can be easily extended for model (). It has reduced the computational time considerably. At each stage the standard Newton-Raphson or Gauss-Newton method may be used, for optimization purposes, but the proof of convergence of the Newton-Raphson or Gauss-Newton method is not known and it is not very easy to establish also in this case. In this paper we propose a new algorithm to estimate the unknown parameters of -D sinusoidal model when the number of components is known. This is motivated by the one dimensional algorithms proposed by Bai et al. ], Nandi and Kundu 5] and the one dimensional sequential procedure proposed by Prasad et al. 6]. The method uses correction terms based on the data vector and the available frequency estimators, similarly as the Newton-Raphson or Gauss- Newton method. But naturally the correction term is different. It is possible to choose the initial guesses in such a manner that within fixed number of steps, the iterative procedure produces efficient estimators, which have the same rate of convergence as the least squares estimators. In the proposed algorithm, we do not use the fixed sample size available at each step. At first step we use a fraction of it and at the last step we use the whole data set, by gradually increasing the effective sample sizes. The method can be easily extended for the model (8), when more than one component is present, using the sequential estimation procedure, similarly as in Prasad et al. 6]. It is shown that if we start the algorithm with the average of periodogram estimators (the details will be explained later) as initial guesses, then after three steps, it produces estimators, which have the same order of convergence as the least squares estimators. We perform some simulation studies to examine the behavior of the proposed algorithm for different sample sizes, and also to compare their performances with the least squares estimators. It is observed that the performances of the proposed estimators and the least squares estimators are very similar, in terms of biases and mean squared errors. But the main advantage of the proposed 3

6 estimators is that they can be obtained in three steps only and the computational time is much less compared to the least squares computation. Moreover, the proof of convergence of the least squares method is not available in the literature, but our method produces efficient estimators almost surely from the above mentioned starting values in three steps only. For illustrative purposes, we have also analyzed one real texture data and one synthesized data. It is observed that the performances of the estimators obtained by the proposed method, are quite satisfactory. The rest of the paper is organized as follows. In section, we provide the model assumptions and the algorithm. Numerical results are provided in section 3. The data analysis results are provided in section 4 and the conclusions appear in section 5. All the necessary theoretical results are provided in the appendix. Model Assumptions and Proposed Algorithm. Assumptions In this subsection we provide the necessary assumptions on the model parameters and particularly on the errors. It is assumed that the observed data {y(m,n);m =,...,M,n =,...,N} is of the form (). The additive error {X(m,n)} is from a stationary random field and it satisfies the following Assumption ; Assumption : Let us denote the set of positive integers by Z. It is assumed that {X(m,n);m,n Z} can be represented as follows; X(m,n) = a(j,k)e(m j,n k), j= k= where a(j,k)s are real constants such that a(j,k) <, j= k= 4

7 and {e(m,n);m,n Z} is a double array sequence of i.i.d. random variables with mean zero and finite variance σ.. Proposed Algorithm The proposed algorithm requires initial estimators of the frequencies, which are consistent but their order of convergence may be low. The method to obtain initial estimators will be discussed later in this section. The algorithm gradually improves upon the initial estimators in a finite number of steps. The final estimators, which are obtained at the last step, have the same asymptotic distribution as the least squares estimators. We provide the necessary theoretical results in the following theorem. Theorem. Suppose ( λ, µ) are consistent estimators of (λ,µ ) and ( λ, µ) are obtained from ( λ, µ) using the following equations, ] ] λ = λ + P (λ) M Im MN, µ = µ + P (µ) Q MN N Im MN, () Q MN where, P (λ) MN = M t= s= P (µ) MN = M Q MN = t= s= t= s= and Im.] denotes the imaginary part of a complex number. ( t M ) y(t,s)e i(e λt+eµs), (3) ( s N ) y(t,s)e i(e λt+eµs), (4) y(t,s)e i(e λt+eµs), (5) If λ λ = O p (M δ N δ ) and µ µ = O p (M δ N δ ), where δ i (, ], i =,, then, (i) λ λ = O p (M δ N δ ), if δ 4 and δ > 4, µ µ = O p (M δ N δ ), if δ 4 and δ > 4, (ii) λ λ µ µ T D N (,4σ Σ ), if δ > 4,δ > 4, 5

8 where Σ = c ρ c ρ, D = ρ = A + B and c = M 3 N, M N 3 e i(λ j +µ j ). Proof. See the Appendix. Based on the above result, we provide the algorithm to find efficient estimators of λ s and µ s. The main idea in the algorithm is to use Theorem step by step to improve the estimates. Moreover, we will not use the whole sample size at each step, rather a fraction of it judiciously, similarly as in Bai et al. ] or Nandi and Kundu 5]. Therefore, at the r th step if we use the sample size (M r,n r ), then the r th step estimators λ (r) and µ (r) are computed from the (r ) th step estimators λ (r ) and µ (r ) by; λ (r) = λ (r ) + (λ) P M Mr Im rn r Q MrN r µ (r) = µ (r ) + Nr Im P (µ) M rn r Q MrN r ] ], (6), (7) where P (λ) M rn r, P (µ) M rn r and Q MrN r can be obtained from (3), (4) and (5), by replacing M, N, λ and µ with M r, N r, λ (r ) and µ (r ), respectively. For better understanding, let us look at the algorithm when the initial estimators of λ and µ are of the order O p (M N ) and O p (M N ) respectively, i.e. ( λ () λ ) = O p (M N ) and ( µ () µ ) = O p (M N ). Although a similar algorithm can easily be developed when the initial estimators are of the order O p (M δ N δ ) and O p (M δ N δ ) respectively for any δ i (, ]; i =,. Observe that, it is possible to obtain initial estimators λ, µ of λ and µ respectively, from the data {y(m,n);m =,...,M, n =,...,N}. Let us consider the data vector {y(,n),...,y(m,n)} 6

9 for any fixed n {,...,N}, and model it using the -D sinusoidal model. Suppose the periodogram estimate of λ, over Fourier frequencies, obtained from this data stream is denoted by λ n, which is O p (M ), see Rice and Rosenblatt 9]. We find λ n for n =,...,N separately, and take their average to arrive at the initial estimate λ of λ, i.e. λ = N λ n, n= which is O p (M N ). Similarly, considering the data {y(m,),...,y(m,n)} for m {,...,M} it is possible to obtain the initial estimate µ of µ, which is of the order O p (M N ). Thus, we have initial estimators of λ and µ for which, ( λ λ ) = O p (M N ) and ( µ µ ) = O p (M N ). We start our algorithm with these initial guesses. It may be mentioned that the choice of M and M are not fixed. Now we provide the exact algorithm for λ, and for µ it can be obtained similarly. Algorithm for estimating λ : Step : When r =, choosing M = M.8, N = N, and λ () = λ, where λ is an initial estimator such that ( λ λ ) = O p (M N ) = O p (M 4 N ). Applying part (a) of Theorem, we obtain; λ () λ = O p (M N ) = O p (M 5 N ). Step : When r =, let M = M.9, N = N. λ () λ = O p (M 5 N ) = Op (M 3 N ). Now, by part (b) of Theorem, λ () λ = O p (M 3 N ) = O p (M 7 N ). Step 3: When r = 3, let M 3 = M, N 3 = N. λ () k λ k = O p(m 7 N ) = Op (M 7 3 N 3 ). 7

10 Again, by part (b) of Theorem, M 3 N ( λ (3) λ ) d N (,4σ cρ ). Therefore, it is observed that from the initial estimate λ, of the order of convergence O p (M N ), we obtain after Step, an improved estimator of the order of convergence O p (M 6 5N ). At Step, we have not used the full sample. At Step, the improved estimator has the order of convergence O p (M N 7 ). Finally at Step 3, when we use the complete sample, and obtain the efficient estimator of λ, which has the same order of convergence as the least squares estimators, i.e. O p (M 3 N ). As we had mentioned before that the choice of M and M are not fixed. For example another choice can be M = M.83 and M = M.9. Several other choices are also available, which will produce efficient estimator of λ, which has the same order of convergence as above. It is observed in our simulation experiment that the performance does not depend much on different choices of M and M. Similarly, we can obtain an efficient estimator of µ, which has the same order of convergence as the least squares estimator. Now we describe how to extend our method for multiple components signal..3 More than one Components When there are more than one component present in the signal, the model can be written as p y(m,n) = A k cos(λ k m + µ k n) + B k sin(λ k m + µ k n)] + X(m,n). (8) k= Here the number of components p is assumed to be known and X(m, n) is same as before. We have the following additional assumptions for this model. Assumption. The frequency sets {λ i,µ i } are distinct i.e. for i j, (λ i,µ i ) (λ j,µ j ) and (λ i,µ i ) (,π) (,π) for i =,...,p. 8

11 Assumption. The amplitudes satisfy the following restriction, < A p + B p <... < A + B < K <, for some K >. Using the sequential procedure similarly as suggested in Prasad et al. 6], we can obtain estimators of the parameters of the model in (8). Initial estimators are obtained at each step and improved upon using the proposed algorithm. Proceeding in this way, we can obtain estimators of all the parameters. It may be mentioned that since ( λ i, µ i ) and ( λ j, µ j ) are asymptotically independent for i j, the sequential procedure works as the one dimensional method. 3 Numerical Results In this section, we present some numerical results to see the performance of the proposed algorithm for different sample sizes. All the computations were performed at the Indian Institute of Technology Kanpur, using the random number generator RAN of Press et al. 7]. All the programs are written in FORTRAN-77. We consider the following model; Model: y(m,n) = A k cos(mλ k + nµ k ) + B k sin(mλ k + nµ k )] + X(m,n). k= Here A =.5, B =.5, λ =., µ =., A =., B =., λ =., µ =. and X(m,n) = e(m,n) + e(m,n) + e(m,n ), (9) where e(m,n) s are i.i.d. normal random variables with mean and variance σ. We have considered different sample sizes, M = N = 5, 75,, and the error variance, σ =.5. In each case we have obtained initial guess of frequencies by computing the average of the periodogram estimates. We then apply the proposed algorithm to improve upon the estimates. In each case we have repeated the procedure for times and reported the average estimates and the corresponding mean squared errors of the proposed estimates. We have reported the average 9

12 and the corresponding mean squared errors, of the usual least squares estimates obtained using sequential procedure as proposed in Prasad et al. 6]. To compute the least squares estimates we have used the optimization routine available in Press et al. 7]. For comparison purposes, we have also reported the asymptotic variance of the least squares estimators. The results are reported in Tables - 4. Some of the points are easily noticed from the tables. As the sample size increases, biases and MSEs decrease as expected for both least squares estimators and the estimators obtained using the proposed algorithm. This verifies the consistency property of the estimators. Biases of the linear parameters are more than the non-linear parameters. In both the cases the mean squared errors are quite close to the corresponding asymptotic variances. Although the proposed estimators can be obtained after three steps, but the performance of the proposed estimators are almost same with the least squares estimators and the required computational time also is much less. Moreover, the proposed algorithm does not require any stopping criterion like any other standard optimization method and it is going to converge almost surely. 4 Data Analysis In this section we present two data analysis for illustrative purpose. One is an original texture data analysis and the other is a synthesized texture analysis when the two adjacent frequencysets are close to each other. Real Texture Data We obtained the texture in Figure 3 from pbourke. The usual procedure to get an idea of initial guess of frequencies is to plot the periodogram function of the data, see Figure, and look at the peaks on the surface of periodogram. From an observation of the periodogram we see that there are many adjacent peaks on the surface. It is not easy to guess the correct number of components from the periodogram. Here the number

13 of components is chosen by minimizing the Bayesian Information Criterion (BIC) give as; BIC(k) = MN ln σk + (4k + )ln(mn) () where k is the number of components, σ k is the corresponding innovations variance. We plot BIC(k) as a function of k. It is observed that BIC takes its minimum value at k = 6, see Figure therefore we take the estimate of the number of components p as p = 6. We have fitted the model in () with p = 6 to the the texture data in Figure 3. The original and estimated textures are plotted in Figure 3. It matches with the original texture quite well Figure : Periodogram of the Original Texture Synthesized Texture Data Now we analyze a texture signal generated from the following model for m =,..., and n =,...,; y(m,n) = 5.cos(.5m +.n) + 5.sin(.5m +.n) +.cos(.4m +.9n) +.sin(.4m +.9n) + X(m,n). () The noise structure X(m,n) follows (9) and e(m,n)s have mean and variance.. The noisy texture is plotted in Figure 4 and the original texture (without the noise component X(m,n))

14 . x BIC(k) k Figure : BIC as a function of number of components Figure 3: The Original and Estimated Texture

15 is plotted in Figure 5. The problem is to extract the original texture given only the noisy texture. Note that here the two frequency-sets, viz. (.5,.) and (.4,.) are very close to each other. When we plot the periodogram of the above data, see Figure 6, we observe a single peak. This obscures the fact that originally there were two frequency components and thus makes it difficult to provide correct initial guess of frequencies. But by using our algorithm we obtain the following estimates of the unknown parameters; Â = 4.737, B = 4.999, λ =.55, µ =.3 Â =.79, B =.986, λ =.3995, µ =.9. We have plotted the actual and estimated texture in Figure 7. They match quite well. We would like to mention here that the usual least square method may not work well if the two frequency sets are close and error variance is large. But sequential least squares method, similar to Prasad et al. 6] is able to distinguish the two frequencies (plots are not shown) and provides reasonably good estimates. Figure 4: Synthesized Noisy Texture Figure 5: Synthesized Actual Texture 3

16 8 6 I(λ,µ) λ.5 µ Figure 6: Periodogram of the Synthesized Texture Figure 7: Synthesized Actual and Estimated textures 4

17 5 Conclusions In this paper we have given an efficient and fast algorithm towards estimating the unknown parameters of a -D sinusoidal model. The iterative methods for multi-dimensional optimization take long to converge to an optimal solution. The periodogram estimates have larger bias and mean squared error. The proposed algorithm takes the periodogram estimates as the initial estimates about the frequencies and improves upon it in a finite number of iterative steps. We have done extensive simulations for different sample sizes and increasing error variances, though not reported here, and found that as the sample size becomes large, the method performs increasingly well and the performance is quite satisfactory even for fairly large error variances. We have derived the asymptotic distribution of the proposed estimators; it coincides with the least squares estimators. Since only a finite number of steps are required to reach the final estimators, the algorithm produces very fast results and it can be used for online implementation purpose. The algorithm can be extended even for colored texture also. The work is in progress and it will reported later. Appendix In this Appendix we provide the proof of Theorem. The following two lemmas are required to prove Theorem. Lemma. If Q MN = MN (A ib ) P (λ) MN = i M3 N ( A + B i ] + O p (M δ N δ ) + O p (M δ N δ ), () ( e(m j,n j ) m M ) e i(λ m+µ n) ) + O p (M δ N δ ) + O p (M δ N δ )] ( λ λ ) (3) 5

18 and λ is obtained from λ, which is O p (M δ N δ ), using the following equation, ] λ = λ + P (λ) M Im MN, Q MN then, (i) λ λ = O p (M δ N δ ), if δ 4 and δ > ( ) 4, (ii) M 3 N (ˆλ λ d ) N,4σ c, if δ ρ > 4 and δ > 4, where c and ρ are same as defined in Theorem. Proof. ] λ = λ + P (λ) M Im MN Q MN = λ + M Im MN M e(m j,n j ) ( m M ) e i(λ m+µ n) (A ib ) + O p (M δ N δ ) + Op (M δ N δ )] i(a ib ) M3 N 4 + O ] p(m δ N δ ) + O p (M δ N δ )]( λ λ ) MN (A ib ) + O p (M δ N δ ) + Op (M δ N δ )] = λ + O p (M δ N δ ) + O p (M δ N δ )]( λ λ ) + M e(m j,n j ) ( m M ) e i(λ m+µ n) ] j M Im = j = MN (A ib ) = λ O p (M δ N δ ) + O p (M δ N δ )]( λ λ ) + 4 M 3 N Im (A ib ) e(m j,n j ) If δ 4 and δ > 4, then ( m M ) e i(λ m+µ n) ]. (4) ] O p (M δ N δ ) + O p (M δ N δ ) ( λ λ ) = O p (M δ N δ ) + O p (M δ δ N δ δ ) = O p (M δ N δ ). Note that the last equality follows because δ δ, and due to the same reason, ignoring the last term in (4), we have, λ λ = O p (M δ N δ ). 6

19 If δ > 4 and δ >, then the second term in (4) is ignored and we get, 4 where, V = Im (A ib ) = A + B A + B It can be proved that, ( ) 4 lim Var V = M,N M N 3 λ λ = 4 M 3 N V, 4σ (A + B ) ( e(m j,n j ) m M ) ] e i(λ m+µ n) e(m j,n j ) ( e(m j,n j ) m M ) sin(λ m + µ n) ( m M ) ] cos(λ m + µ n). (5) e i(λ j +µ j ). (6) Now, using the Central Limit Theorem of the stochastic processes, (see Fuller 3]), we have the following, ( M 3 N (ˆλ λ d ) N,4σ c ) ρ. Lemma. If Q MN is same as in () and P (µ) MN = i MN3 ( A + B i ( e(m j,n j ) n N ) e i(λ m+µ n) ) + O p (M δ N δ ) + O p (M δ N δ )] ( µ µ ) (7) and µ is obtained from µ, which is O p (M δ N δ ), using the following equation, then, ] µ = µ + P (µ) N Im MN, Q MN (i) µ µ = O p (M δ N δ ), if δ 4 and δ > ( ) 4 (ii) M N 3 (ˆµ µ d ) N,4σ c, if δ ρ > 4 and δ > 4 where c and ρ are same as before. 7

20 Proof. The Proof is similar. Along the same line as before, if δ > 4 and δ > 4 it can be shown in this case also that where, W = Im (A ib ) = A + B A + B µ µ = 4 MN 3W, Moreover, it also can be shown that, ( ) 4 lim Var 4σ W = M,N M N 3 (A + B ) ( e(m j,n j ) n N ) ] e i(λ m+µ n) e(m j,n j ) ( e(m j,n j ) n N ) sin(λ m + µ n) ( n N ) ] cos(λ m + µ n). (8) e i(λ j +µ j ), (9) and ( 4 lim Cov V, M,N M N 3 4 M N 3 ) W =. () Now, using Lemma, Lemma and (), Theorem follows immediately, provided we show that Q MN, P (λ) MN and P (µ) MN as defined in Theorem, can be written as (), (3) and (7) respectively. We will use the following results in the subsequent proofs. Taylor s Theorem: Suppose f(t) is a real valued function on a,b], n is a positive integer, f (n ) (t) is continuous on a,b], f (n) (t) exists for every t (a,b). Let α,β be distinct points of a,b], then we can write, f(β) = n k= f (k) (α) k! where x is a point on the line joining α and β. e(m,n) = O p (M N ). (β α) k + f(n) (x) (β α) n, () n! 8

21 te(m,n) = O p (M N 3 ) and In general, ( ) m M m k e(m,n) = O p (M k+ N ) and e(m,n) = O p (MN). e(m,n) = O p (M 3 N ) and M se(m,n) = O p (M N 3 ). m k e(m,n) = O p (M N k+ ). ( ) n N e(m,n) = O p (M N 3 ). Now we will prove (), (3) and (7). Proof of (). From the definition of Q MN in (5), Here, Q MN = ( A = R = = A cos(λ m + µ n) + B sin(λ m + µ n) + X(m,n) ] e i(e λm+eµn) + B i ) ( A ) R + B R + R 3 (say). () i e i{(λ λ)m+(µ e eµ)n} = M + i(λ λ) m= e i(λ λ)m e m= me i(λ λ )m ] n= N + i(µ µ) e i(µ eµ)n n= ne i(µ µ )n ] ] = M + O p (M δ N δ )O p (M ) N + O p (M δ N δ )O p (N ) ] ] = M + O p (M δ N δ ) N + O p (M δ N δ ) ] = MN + O p (M δ N δ ) + O p (M δ N δ ), (3) ] λ is a point on the line joining λ and λ and µ is a point on the line joining µ and µ. Further, R = e i{(λ + λ)m+(µ e +eµ)n} = O p (), (4) and R 3 = X(m,n)e i(e λm+eµn). (5) 9

22 Now we will evaluate the order of R 3. Choose L large enough, such that L.min{δ,δ } >. We obtain the following of R 3, using the Taylor series approximation similarly as in Bai et al. ] or Nandi and Kundu 5], up to L th order terms. Here λ is a point on the line joining λ and λ. R 3 = = = = e i(λ m) k= e(m j,n j )e i(e λm+eµn) M e ieµn e(m j,n j )e ie λm n= m= M e ieµn e(m j,n j ) n= m= ] ( i( λ λ )) k m k e i(λm) + ( i( λ λ )) L m L e i(λ m), k! L! M e ieµn e(m j,n j )e i(λ m) n= k= m= ( i( λ λ )) k k! ( i( λ λ )) L L! M e ieµn e(m j,n j )m k e i(λ m) n= m= M e ieµn e(m j,n j )m L e i(λ m). n= m= Note that, e(m j,n j )m L e i(λ m) ML m= e(m j,n j ), m= since e i(λ t) =. Hence, we can write, M e(m j,n j )m L e i(λ m) = θ M L e(m j,n j ), m= for θ. Now re-arranging the terms, R 3 becomes R 3 = + + k= m= e(m j,n j )e i(λ m+eµn) ( i( λ λ )) k k! θ (M( λ λ )) L L! e(m j,n j )m k e i(λ m+eµn) e(m j,n j ) e ieµm,

23 = T + T + T 3 (say). (6) We will consider T, T and T 3 one by one, and find out their order. First, T = Expanding using Taylor s theorem, we get, T = + + = O p (M N ) + = O p (M N ). k= e(m j,n j )e i(λm+eµn). e(m j,n j )e i(λ m+µ n) ( i( µ µ ) ) k k! θ (N( µ µ )) L L! k= e(m j,n j )n k e i(λ m+µ n) e(m j,n j ) e iλ m O p (M δk N k kδ ) O p (M N k+ ) + Op (N L M Lδ N L Lδ MN) k! (7) Expanding using Taylor s theorem, we get, T = + + = k= k= k= k= ( i( λ λ )) k k! ( i( λ λ )) k k! ( i( λ λ )) k k! e(m j,n j ) m k e iλ m O p (M (+δ )k N δ k )O p (M k+ N ) s= e(m j,n j )m k e i(λ m+µ n) ( i( µ µ )) s s! θ 3(N( µ µ )) L L! + O p (M (+δ)k N δk )O p (M δs N (+δ)s ) O p (M k+ N s+ ) k= s= + k= O p (M (+δ )k N δ k ) O p (N L M Lδ N L Lδ ) O p (M k+ N) = O p (M kδ N kδ ) + O p (M kδ sδ N kδ sδ ) k= k= s= m k n s e i(λ m+µ n)

24 O p (M kδ Lδ N kδ Lδ ) + k= = O p (M δ N δ ) + O p (M δ δ 3 N δ δ 4 ) + O p (M δ Lδ N δ Lδ ) = O p (M δ N δ ). (8) Expanding using Taylor s theorem for θ i for i =,...,4, we get, T 3 = + + θ (M( λ λ )) L L! θ (M( λ λ )) L L! e(m j,n j ) n s e iµ n θ 4(N( µ µ )) L L! s= θ (M( λ λ )) L L! = O p (M Lδ N Lδ ) O p (MN) e(m j,n j ) e(m j,n j ) e iµ s ( i( µ µ )) s s! + O p (M Lδ N Lδ ) O p (M δs N (+δ)s ) O p (MN s+ ) s= + O p (M Lδ N Lδ ) O p (M Lδ N Lδ ) O p (MN) = O p (M Lδ N Lδ ) + O p (M nδ Lδ N nδ Lδ ) + O p (M Lδ Lδ N Lδ Lδ ) n= = O p (M Lδ N Lδ ). (9) Hence, from (6), using (7), (8) and (9), we have, R 3 = T + T + T 3 = O p (M N ) + Op (M δ N δ ) + O p (M Lδ N Lδ ) = O p (M N ), (3) since Lδ i > for i =,. Therefore, from (), using (3), (4) and (3) we have, ( A Q MN = + B i ( A = + B i ) ( A R + B i ) R + R 3 ) MN + O p (M δ N δ ) + O p (M δ N δ ) ]

25 ( ) A + B O p () + O p (M N ) i = MN (A ib ) ] + O p (M δ N δ ) + O p (M δ N δ ). (3) The expressions for P (λ) MN similarly. and P (µ) MN as given in (3) and (7) respectively can be obtained References ] Bai Z. D. and Rao C. R. and Chow M. and Kundu D. (3), An efficient algorithm for estimating the parameters of superimposed exponential signals, Journal of Statistical Planning and Inference, vol., ] Francos J. M. and Meiri A. Z. and Porat B. (993), A united texture model based on -D Wald like decomposition, IEEE Transaction on Signal Processing, vol. 4, ] Fuller W. A.(976), Introduction to Statistical Time Series, John Wiley and Sons, New York. 4] Kundu D. and Nandi S. (3), Determination of Discrete Spectrum in a Random Field, Statistica Neerlandica, vol. 57, ] Nandi S. and Kundu D. (6), Fast and efficient algorithm for estimating the frequencies of sinusoidal signals, Sankhya, vol. 68, ] Prasad A. and Kundu D. and Mitra A. (8), Sequential Estimation of the Parameters of Sum of Sinusoidal Model, Journal of Statistical Planning and Inference, vol. 38, ] Press W.H. and Teukolsky S.A. and Vellerling W.T. and Flannery B.P., (99), Numerical Recipes in FORTRAN, the art of scientific computing, nd. edition, Cambridge University Press, Cambridge. 3

26 8] Rao C. R. and Zhao L.C. and Zhou B. (994), Maximum likelihood estimation of the -D superimposed exponential, IEEE Transactions on Signal Processing, vol. 4, ] Rice J. A. and Rosenblatt M. (988), On Frequency Estimation, Biometrika, vol. 75, ] Zhang H. and Mandrekar V. (), Estimation of hidden frequencies for D stationary processes, Journal of Time Series Analysis, vol., Table : First Component: Algorithm Estimates. A =.5 B =.5 λ =. µ =. Time AE M= 5 MSE (.346E-) (.35E-) (.95E-6) (.896E-6) :53.8 N= 5 ASYV (.334E-) (.334E-) (.889E-6) (.889E-6) AE M= 75 MSE (.5) (.5) (.83E-6) (.73E-6) 7:3.96 N= 75 ASYV (.5) (.5) (.76E-6) (.76E-6) AE M= MSE (.86E-3) (.8E-3) (.56E-7) (.559E-7) 9:37.8 N= ASY V (.834E-3) (.834E-3) (.556E-7) (.556E-7) 4

27 Table : Second Component: Algorithm Estimates. A =. B =. λ =. µ =. Time AE M= 5 MSE (.7E-) (.75E-) (.43E-5) (.48E-5) :53.8 N= 5 ASYV (.76E-) (.76E-) (.43E-6) (.43E-6) AE M= 75 MSE (.33) (.33) (.86E-6) (.9E-6) 7:3.96 N= 75 ASYV (.3) (.3) (.849E-6) (.849E-6) AE M= MSE (.89E-) (.86E-) (.7E-6) (.59E-6) 9:37.8 N= ASYV (.79E-) (.79E-) (.69E-6) (.69E-6) Table 3: First Component: Least Squares Estimates. A =.5 B =.5 λ =. µ =. Time AE M= 5 MSE (.349E-) (.356E-) (.9E-6) (.97E-6) 5:53.8 N= 5 ASYV (.334E-) (.334E-) (.889E-6) (.889E-6) AE M= 75 MSE (.5) (.5) (.88E-6) (.78E-6) 7:3.96 N= 75 ASYV (.5) (.5) (.76E-6) (.76E-6) AE M= MSE (.9) (.9) (.68E-7) (.63E-7) 45:3. N= ASYV (.8) (.8) (.556E-7) (.556E-7) 5

28 Table 4: Second Component: Least Squares Estimates. A =. B =. λ =. µ =. Time AE M= 5 MSE (.695E-) (.744E-) (.433-5) (.47E-5) 5:53.8 N= 5 ASYV (.76E-) (.76E-) (.43E-6) (.43E-6) AE M= 75 MSE (.33) (.36) (.84E-6) (.98E-6) 7:3.96 N= 75 ASYV (.3) (.3) (.849E-6) (.849E-6) AE M= MSE (.9) (.9) (.73E-6) (.63E-6) 45:3. N= ASYV (.7) (.7) (.69E-7) (.69E-7) The average estimates and the MSEs are reported for each parameter. The first row represents the true parameter values. In each box corresponding to each sample size, the first row represents the average estimates, the corresponding MSEs and the asymptotic variances (ASYV) are reported below within brackets. 6