Comparison of Various Periodograms for Sinusoid Detection and Frequency Estimation

I. INTRODUCTION Comparison of Various Periodograms for Sinusoid Detection and Frequency Estimation H. C. SO, Member, IEEE City University of Hong Kong Y. T. CHAN, Senior Member, IEEE Q. MA Royal Military College of Canada P. C. CHING, Senior Member, IEEE The Chinese University of Hong Kong With the advent of the fast Fourier transform (FFT) algorithm, the periodogram and its variants such as the Bartlett s procedure and Welch method, have become very popular for spectral analysis. However, there has not been a thorough comparison of the detection and estimation performances of these methods. Different forms of the periodogram are studied here for single real tone detection and frequency estimation in the presence of white Gaussian noise. The threshold effect in frequency estimation, that is, when the estimation errors become several orders of magnitude greater than the Cramér-Rao lower bound (CRLB), is also investigated. It is shown that the standard periodogram gives the optimum detection performance for a pure tone while the Welch method is the best detector when there is phase instability in the sinusoid. As expected, since the conventional periodogram is a maximum likelihood estimator of frequency, it generally provides the minimum mean square frequency estimation errors. Manuscript received January 19, 1998; revised November 3, 1998. IEEE Log No. T-AES/35/3/645. Authors current addresses: H. C. So, Dept. of Electronic Engineering, City University of Hong Kong, Tat Chee Avenue, Kowloon, Hong Kong; Y. T. Chan, Dept. of Electrical and Computer Engineering, Royal Military College of Canada, Kingston, Ontario, Canada, K7K 5L; Q. Ma, Wireless Networks, Nortel Networks, Ottawa, Canada; P. C. Ching, Dept. of Electronic Engineering, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong. 18-951/99/$1. c 1999 IEEE Detection and frequency estimation of sinusoidal signals from a finite number of noisy discrete-time measurements have applications in many fields. It has been widely used in sonar and radar for moving target detection. Estimation of Doppler shift in this case is usually required in order to find its position and speed. A recent and interesting application is in the search for extraterrestrial intelligence (SETI) [1]. Other well-known examples include demodulation of frequency-shift keying signals, wind profiling [], geolocation, and identification and tracking of an emergency location transmitter [3]. A direct method for tone detection and frequency estimation is the standard periodogram. The periodogram of an N-point sequence, x(),x(1),:::,x(n 1), is defined as S x (f)= 1 N 1 x(n)exp( j¼nf), N n= k =,1,:::,N 1: (1) Samples of the periodogram at f k = k=n, k =,1,:::,N 1 can be computed efficiently with the use of the fast Fourier transform (FFT) algorithm while values at the other frequencies are evaluated by either zero padding or interpolation. In the Bartlett method which is a variant of the periodogram, the sequence x(n) is divided into K nonoverlapping segments, where each segment has length M. For each segment, the periodogram is computed and the Bartlett power spectral estimate is obtained by averaging the periodograms for the K segments. By so doing, the variance in the periodogram estimate is reduced by a factor K but at the expense of reducing the frequency resolution by K. Welch had modified Bartlett s procedure by allowing the data segments to overlap and at the same time to be multiplied by a window function prior to computing the periodogram. The overlapping is used for further reducing the periodogram variance while the windowing is applied to reduce the spectral leakage [4] associated with finite observation intervals. Although the many forms of periodogram have been derived for more than two decades, there is not a comprehensive study of these approaches in sinusoid detection and frequency estimation. In this paper, the standard periodogram with and without windowing, together with Bartlett s procedure and the Welch method are compared in order to determine the best detector for a sinusoid in the presence of white Gaussian noise. We would also like to examine if these modifications can delay the threshold effect of the conventional periodogram in frequency estimation of a noisy sinusoid, even though the latter provides the maximum likelihood estimate. It is important to know the theoretical signal-to-noise ratio (SNR) IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 35, NO. 3 JULY 1999 945

below which the estimates become unreliable. This will provide a guide to system designers to decide on the necessary data length, for a given SNR, to avoid large estimation errors. Theoretical developments for single tone detection and frequency estimation are given in Section II and Section III, respectively. In particular, the detection probability, the false alarm probability, and the mean square frequency error of the periodogram and the Bartlett method are derived. Simulation results are presented in Section IV to corroborate the analytical derivations and to evaluate the detection and estimation performance of different types of periodograms. II. SINGLE TONE DETECTION The problem of detecting a single sinusoid in the presence of noise is formulated as follows. Given the received sequence x(n), a decision has to be made between the hypotheses: H : x(n)=q(n), n =,1,:::,N 1 () H 1 : x(n)= sin(¼f n + Á(n)) + q(n) where q(n) is a white Gaussian process while, f, and Á(n) are unknown parameters which represent the tone amplitude, frequency, and time-dependent phase, respectively. The first hypothesis H assumes that x(n) consists only of noise while in H 1, the sinusoidal signal is presumed to be present. The aim is to find the receiver operating characteristic (ROC) which is determined by the false alarm probability P FA = P(D 1 j H ) and detection probability P D = P(D 1 j H 1 ), where D 1 represents the decision of hypothesis H 1. When the standard periodogram is used for tone detection, three steps are involved. They are 1) compute the N samples of the periodogram using FFT, ) find the peak value by interpolation, and 3) compare it with a threshold TH. If the peak coefficient is larger than TH, H 1 is accepted, otherwise H is chosen. Notice that in the first step, we only need to compute the spectral coefficients for f = 1=N,=N,:::,1= 1=N, because the power spectrum is symmetric for real signals. To derive the P FA and P D in this method, we first obtain the probability density functions (PDFs) of the discrete Fourier transform (DFT) spectral coefficients. For ease of analysis, it is assumed that Á(n)=Á is a constant uniformly distributed between and ¼ while f (,:5) such that f = k =N where k f1,,:::,n= 1g, implying that the tone frequency corresponds exactly to one of the FFT bins. In this case, only the largest coefficient is selected among the DFT power spectrum and no interpolation is necessary. However, the simulation results will also contain experiments with sinusoids of off-bin frequencies. It is noteworthy that the standard periodogram is a digital realization of the quadrature receiver which is optimum in the sense that it attains the minimum probability of error. Denote the spectral coefficients at f = k=n for the noise only case and the signal present case by S x (k) ands x1 (k), respectively. The value of S x1 (k ) is computed as S x1 (k )=A + B (3) where A = p 1 N 1 ¼nk ( sin(¼f n + Á)+q(n))cos N N n= and B = p 1 N 1 ¼nk ( sin(¼f n + Á)+q(n))sin N N n= are Gaussian random variables. It can be proved [5] that A and B are independent and of identical variances equal to ¾ = ¾q =where¾ q represents the power of q(n). We then follow [6] to derive the PDF of S x1 (k ), denoted by p s (u), as p s (u)= 1 ¾ exp s + u ³ pu s ¾ I ¾ (4) which is a noncentral chi-square random variable. The quantity s equals the square root of the sum of mean square of A and B, thatis,s = p N=, and I r is the modified Bessel function of the first kind of order r. Similarly, it can be shown that the remaining S x1 (k) and all S x (k) are of central chi-square distribution and have the same PDF function p q (u) which is given by p q (u)= 1 ³ ¾ exp u ¾ : (5) Note that expressions similar to (4) and (5) can be found in [7] which derives the PDFs of the Fourier transform coefficients of a complex tone signal under noisy environment. Assuming that the DFT spectral coefficients are independent which is valid when the time-bandwidth product of the signal is sufficiently large, then the false alarm rate P FA is calculated as Z VT (N=) 1 P FA =1 p q (u)du =1 1 exp V (N=) 1 T ¾ (6) and the probability of detection P D is derived as P D =1 =1 Z VT 1 exp p q (u)du (N=) Z VT p s (u)du V (N=) Ã T s Ã1 ¾ Q1 ¾, p VT ¾!! (7) 946 IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 35, NO. 3 JULY 1999

where Q denotes the Marcum Q function [8]. The ROC is then acquired by plotting the P D against P FA. On the other hand, the DFT spectrum of the Bartlett method is defined as where Sx (k)= 1 N K 1 M 1 i= n= x i (n)exp j¼nk M, k =,1,:::,M 1 (8) x i (n)=x(i M + n), i =,1,:::,K 1, n =,1,:::,M 1: The derivation for the PDFs is similar to the standard periodogram case but now the number of the power coefficients are reduced to M because of the averaging of K segments. In this case, the spectral coefficients are expressed as K 1 Sx (k)= (A i (k)+b i (k)) (9) where and i= A i (k) = p 1 M 1 ¼nk x i (n)cos N M n= B i (k) = p 1 M 1 ¼nk x i (n)sin N M n= are independent Gaussian random variables. By further assuming f = k =M where k [1,M= 1] is an integer and following the previous derivation that gives rise to (4), it can be shown that when the signal is present, the PDF of the spectral coefficients at k = k, denoted by p s a (u), is of the form p sa (u)= 1 u (K 1)= ¾a sa exp s a + u pu s ¾a I a K 1 ¾a (1) where ¾a = ¾ q =(K) ands a = p M=. The PDFs of all other coefficients for the hypotheses H and H 1 are equal to p qa (u)= ¾ K a 1 K (K) uk 1 exp u ¾a (11) where (K) is the gamma function. As a result, the false alarm and detection probability of single tone detection using the Bartlett method, P FAa and P Da,are given by Z VT P FAa =1 p qa (u)du (M=) 1 (1) and Z VT (M=) Z VT P Da =1 p qa (u)du p sa (u)du (13) respectively. III. FREQUENCY ESTIMATION When the hypothesis H 1 is chosen, often there is a need to estimate the tone frequency from the received sequence. It is well known that the maximum likelihood frequency estimate of a pure sinusoid is given by the location of the peak of the periodogram [7]. This estimator will attain the Cramér Rao lower bound (CRLB) for frequency [9], 3 var f (N)= ¼ N(N (14) 1)SNR when SNR = =(¾q ) is greater than a threshold T. When SNR <T, however, the mean square frequency error (MSFE) rises very rapidly above the CRLB. This is because the frequency estimation problem is nonlinear and hence will suffer from this threshold phenomenon, sometimes also known as occurrence of outliers. Since the modified periodograms do provide smaller variances, one might expect them to give a lower SNR threshold than the standard periodogram. In this section, we derive expressions for the overall MSFEs of the periodogram and the Bartlett s procedure. When f = k =N, the probability of occurrence of an anomaly in the standard periodogram is derived as q = P(S x1 (k ) at least one of S x1 (1),S x1 (),:::, S x1 (k 1),S x1 (k +1),:::,S x1 (N= 1)) 3 Z 1 (N=) 1 Y = P(S x1 (k )=u) 41 P(S x1 (k) <u) 5du k=1,k6=k 3 Z 1 (N=) 1 Y Z u = p s (u) 41 p q (v)dv5du (15) k=1,k6=k which must be computed numerically. Assuming that the anomaly estimate is uniformly distributed between and.5, the overall MSFE is given by MSFE = (1 q)var f (N)+q Z :5 (u f ) du: (16) Similarly, the mean square error in the frequency estimate using the Bartlett method, denoted by MSFE a,isgivenby MSFE a = 1 q a K var f (M)+q a Z :5 (u f ) du (17) SO ET AL.: COMPARISON OF VARIOUS PERIODOGRAMS 947

Fig.. Comparison of periodogram and Bartlett method with different segment lengths for detecting single sinusoid with frequency f =:5 at SNR = 9 db. Fig. 1. ROCs for detection of single sinusoid with frequency f =:5 at different SNRs. (a) Using periodogram. (b) Using Bartlett method with four segments. where q a represents the probability of the occurrence of an outlier in the averaged periodogram, which can be calculated using p sa (u) andp qa (u). Note that there is a factor of K in (17) because each segment will give an independent frequency estimate. IV. SIMULATION RESULTS Computer experiments were conducted to verify the theoretical calculations derived in Sections II and III. The performances of different variants of the periodogram, including the windowed periodogram, the Bartlett method and the Welch method in single tone detection and frequency estimation were also evaluated. The variance of the white noise was fixed to unity while different SNRs were produced by properly scaling the signal power. The total sample size N was 56 and unless stated otherwise, the segment length M had a value of 64. The results for detection were averages of 1 independent runs while those for frequency estimation were based on 5 independent trials. Fig. 1(a) shows the experimental and theoretical ROCs in detecting a pure sinusoid in the presence of white Gaussian noise using the standard periodogram. The simulation results were obtained by using the method suggested in [1]. Four values of SNR, namely, 6 db, 9 db, 1 db, and 15 db were tried. The frequency f was chosen as.5 which corresponded to one of the FFT bins. It can be seen that in all cases the simulation results agreed very well with theory. The test was repeated for the Bartlett method and the results are shown in Fig. 1(b). Again, we observe that the experimental and theoretical results were very similar. By comparing Figs. 1(a) and 1(b), it can be seen that the periodogram always provides a better detection performance than the Bartlett method. Fig. plots the theoretical ROCs of the periodogram and the Bartlett method for different segment length M at SNR = 9 db. It is observed that the detection accuracy decreased as the number of segment K increased. Fig. 3(a) compares the detection performance of different forms of periodograms for f =:5 and SNR = 9 db. In the windowed periodogram, the Hann window function was used while in the Welch method, rectangular and Hann windows were tried. Again, the standard periodogram gave the best performance. It is because it provided the greatest SNR of N=(¾q )atf =:5, although other modified periodograms had smaller spectral variances. The Welch method with 5% overlap using Hann window was superior to the one without overlap and the windowed periodogram, but was inferior to the Bartlett s procedure and the Welch method with 5% overlap using rectangular window. In order to investigate their performance in the presence of spectral leakage, another similar experiment was conducted for tone frequency which was uniformly distributed within one bin and the results are illustrated in Fig. 3(b). In this test, f was a uniform random variable whose value is between f 1=(N) andf +1=(N) in the standard/windowed periodogram and f (f 1=(M),f +1=(M)) in other methods. It can be seen that the relative detectabilities of the methods were similar to those in Fig. 3(a), except that the Welch method with 5% overlap using Hann window outperformed the Bartlett 948 IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 35, NO. 3 JULY 1999

Fig. 3. Comparison of ROCs for detecting single sinusoid using periodogram, windowed periodogram, Bartlett s procedure, Welch methods at SNR = 9 db.(a)f =:5. (b) f uniformly distributed within one bin. method at approximately P FA > :15. Comparing Figs. 3(a) and 3(b), we observe that the detection performance of the periodogram, the 5% overlapped Welch methods and the Bartlett s procedure degraded when the frequency was not exactly on the bin, while the remaining two methods had almost identical detectablities in both cases. Figs. 4(a) and 4(b) compare the ROCs for a sinusoid that exhibited phase instability with exact-bin frequency and frequency uniformly distributed within one bin, respectively, at SNR = 9 db. This sinusoid was modeled by a narrowband random process with normalized autocorrelation function r(m)= :95 jmj cos(¼f m). The results of Figs. 4(a) and 4(b) were almost identical and the detection performance in descending order was as follows, the Welch method with 5% overlap using Hann window, the 5% overlapped Welch method with rectangular window, the Bartlett method, the % overlapped Welch method, the periodogram and the windowed periodogram. Fig. 5 plots the mean square frequency errors of the periodogram and the Bartlett method together with the CRLB, when the source signal is a pure sinusoid with f =:5. It is observed that the simulation results agreed with the theoretical values of (16) and (17), particularly at the threshold SNRs. Fig. 4. Comparison of ROCs for detecting non-stationary-phase single sinusoid with autocorrelation function r(m)=:95 jmj cos(¼f m) using periodogram, windowed periodogram, Bartlett s procedure, Welch methods at SNR = 9 db.(a)f =:5. (b) f uniformly distributed within one bin. Fig. 5. Theoretical and experimental frequency variances of periodogram and Bartlett method with four segments for single sinusoid with f =:5 at different SNRs. Moreover, at SNR 5 db, the periodogram met the CRLB and had an MSFE which was approximately one-tenth of the Bartlett method. Fig. 6(a) shows the simulation results of all six methods in the same trial. Although the periodogram did not possess the smallest threshold SNR, it gave the optimum MSFEs for all SNRs. The estimation performance in SO ET AL.: COMPARISON OF VARIOUS PERIODOGRAMS 949

the corresponding frequency using conventional periodogram and the Bartlett s procedure are derived and confirmed by computer simulations. The standard periodogram provides the best ROC for a pure tone while the Welch method is the optimum detector for a sinusoid with nonstationary phase. Furthermore, the periodogram is the best frequency estimator for a noisy sinusoid among its variants. ACKNOWLEDGMENT The authors thank Mr. W. K. Ma for performing some of the simulations. REFERENCES Fig. 6. MSFEs for single sinusoid using periodogram, windowed periodogram, Bartlett s procedure, Welch method at different SNRs. (a) f =:5. (b) f =:5 + 1=(N) in periodograms and f =:5 + 1=(M) in Bartlett and Welch methods. descending order is as follows: the basic periodogram, the windowed periodogram, the two 5% overlapped Welch methods, the Bartlett s procedure, and the % overlapped Welch method. Comparison of the methods for frequency located in the middle of bins is depicted in Fig. 6(b). In general, the standard periodogram still provided the minimum mean square error, and the second best was the windowed periodogram, then the two 5% overlapped Welch methods, while Bartlett method and the % overlapped Welch method were the poorest frequency estimators. V. CONCLUSIONS The theoretical performance of detecting a real tone under noisy environment and estimating [1] Zimmerman, G. A., and Gulkis, S. (1991) Polyphase-discrete Fourier transform spectrum analysis for the search for extraterrestrial intelligence sky survey. TDA progress report 4-17, Jet Propulsion Laboratory, Pasadena, CA, July Sept. 1991, 141 154. [] Keeler, R. J., and Griffiths, L. J. (1977) Acoustic Doppler extraction by adaptive linear-prediction filtering. Journal of the Acoustical Society of America, 61, 5(May 1977), 118 17. [3] Report KMV76- (1976) Fourier analysis and identification of ELT signals. Prepared for NASA GSFC, Sept. 1976. [4] Harris, F. J. (1978) On the use of windows for harmonic analysis with the discrete Fourier transform. Proceedings of the IEEE, 66, 1 (Jan. 1978), 51 83. [5] Papoulis, A. (1965) Probability, Random Variables, and Stochastic Processes. New York: McGraw-Hill, 1965. [6] Proakis, J. G. (1989) Digital Communications. New York: McGraw-Hill, 1989. [7] Rife, D. C., and Boorstyn, R. R. (1974) Single-tone parameter estimation from discrete-time observations. IEEE Transactions on Information Theory,, 5(Sept. 1974), 591 598. [8] Helstrom, C. W. (199) Computing the generalized Marcum Q-function. IEEE Transactions Information Theory, 38, 4 (July 199), 14 148. [9] Kay, S. M. (1993) Fundamental of Statistical Signal Processing: Estimation Theory. Englewood Cliffs, NJ: Prentice-Hall, 1993. [1] Hung, E. K. L., and Herring, R. W. (1981) Simulation experiments to compare the signal detection properties of DFT and MEM spectra. IEEE Transactions on Acoustics, Speech, Signal Processing, ASSP-9, 5 (Oct. 1981), 184 189. 95 IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 35, NO. 3 JULY 1999

H. C. So (M 96) was born in Hong Kong, on June 7, 1968. He received the B.Eng. degree in electronic engineering from City Polytechnic of Hong Kong in 199. In 1995, he received the Ph.D. degree in electronic engineering from the Chinese University of Hong Kong. From 199 to 1991, he was an Electronic Engineer at the Research and Development Division of Everex Systems Engineering Ltd. During 1995 1996, he worked as a post-doctoral Fellow at the Chinese University of Hong Kong. He is currently a Research Assistant Professor in the Department of Electronic Engineering of City University of Hong Kong. His research interests include adaptive signal processing, detection and estimation, source localization, and wavelet transform. Y. T. Chan (SM 8) was born in Hong Kong. He received the B.Sc. and M.Sc. degrees from Queen s University, Kingston, Ontario, Canada in 1963 and 1967, and the Ph.D. degree from the University of New Brunswick, Fredericton, Canada in 1973, all in electrical engineering. He has worked with Northern Telecom Ltd. and Bell-Northern Research. Since 1973, he has been at the Royal Military College of Canada, where he is presently professor and Head, Department of Electrical and Computer Engineering. He has also spent two sabbatical terms at the Chinese University of Hong Kong, during 1985 1986 and in 1993. His research interests are in wavelet transform, sonar signal processing and passive localization and tracking techniques and he has served as a consultant on sonar systems. He is author of the book Wavelet Basics. He was an Associate Editor (198 198) of the IEEE Transactions on Signal Processing and was the Technical Program Chairman of the 1984 International Conference on Acoustics, Speech and Signal Processing (ICASSP 84). He directed a NATO Advanced Study Institute on Underwater Acoustic Data Processing in 1988 and was the General Chairman of ICASSP 91 held in Toronto, Canada. Qiang Ma was born in China, on December 9, 1963. He received the Ph.D. degree in 1995 from Loughborough University, UK. From December 1995 to May 1997, he was a Research Associate at Royal Military College of Canada. He is currently a DSP Designer in Wireless Networks, Nortel Networks in Ottawa, where he has been engaged in research and development of digital radio communication system. His research interests include adaptive filtering, detection, and channel estimation. SO ET AL.: COMPARISON OF VARIOUS PERIODOGRAMS 951

P. C. Ching (M 8 SM 9) received the B.Eng. (Hons.) and Ph.D. degrees in electrical engineering and electronics from the University of Liverpool, UK, in 1977 and 1981, respectively. From 1981 to 198 he worked at the School of Electrical Engineering of the University of Bath, UK, as a research officer. During 198 1984, he was a Lecturer in the Department of Electronic Engineering of Hong Kong Polytechnic. Since 1984 he has been with the Chinese University of Hong Kong, where he is presently Dean of Engineering and a professor in the Department of Electronic Engineering. He has taught courses in digital signal processing, stochastic processes, speech processing, and communication systems. His research interests include adaptive filtering, time delay estimation, signal processing for communication, speech coding, synthesis and recognition. Dr. Ching was the Chairman of the IEEE Hong Kong Section in 1993 1994, and is currently a member of the Signal Processing Theory and Methods Technical Committee of the IEEE Signal Processing Society and the IEE Hong Kong Centre Executive Committee. Since 1997, he has served as an Associate Editor for the IEEE Transactions on Signal Processing. He has also been involved in organizing many international conferences including the 1997 IEEE International Symposium on Circuits and Systems where he was the Vice-Chairman. Dr. Ching is a Fellow of IEE and HKIE. 95 IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 35, NO. 3 JULY 1999