Wavelet Thresholding Techniques for Power Spectrum Estimation

Transcription

1 Wavelet Thresholding Techniques for Power Spectrum Estimation Pierre Moulin Bell Communications Research 445 South Street Morristown, NJ tel (201) fax (201) August 1993 to appear in IEEE Trans. on Signal Processing ABSTRACT Estimation of the power spectrum S( f ) of a stationary random process can be viewed as a nonparametric statistical estimation problem. We introduce a nonparametric approach based on a wavelet representation for the logarithm of the unknown S( f ). This approach offers the ability to capture statistically significant components of lns( f ) at different resolution levels and guarantees nonnegativity of the spectrum estimator. The spectrum estimation problem is set up as a problem of inference on the wavelet coefficients of a signal corrupted by additive non-gaussian noise. We propose a wavelet thresholding technique to solve this problem under specified noise/resolution tradeoffs and show that the wavelet coefficients of the additive noise may be treated as independent random variables. The thresholds are computed using a saddle-point approximation to the distribution of the noise coefficients. EDICS: 3.1.1, 2.2.1, Index terms : wavelets, spectrum estimation, noise reduction, non-gaussian signal processing.

2 INTRODUCTION We consider the problem of estimating the power spectrum {S( f ), 1/2 < f 1/2} of a discrete-time, wide-sense stationary, real, Gaussian random process {x(n), n = 0, ±1, ±2,... } from a finite set of observations. Consistent estimates may be obtained by suitable processing of the empirical spectrum estimates (periodogram). This may be done using sieves [1][2], spline smoothing [3-5], or kernel smoothing [6, 5.4][7][8]. These methods all require the choice of a certain resolution parameter, often called bandwidth in the spectrum estimation literature [7][8]. Various techniques for optimal bandwidth selection have been studied in [2][4-8]. They produce estimates that have a good overall bias vs variance tradeoff. However, the bias vs variance tradeoff is generally not optimal locally. Sharp peaks and troughs in the spectrum require a narrow spectral bandwidth, otherwise they are oversmoothed and the bias is large; in contrast, smooth components of the spectrum require a wide bandwidth in order to achieve a significant noise reduction. These concerns have long been recognized [8, 7.2], so various ideas have been introduced to remedy this situation. Jenkins technique of window closing produces a family of estimates using different bandwidths [8]. Unfortunately, this technique does not provide a mechanism for combining the information obtained at different resolutions. Capon s method (originally developed for wavenumber analysis with arrays [9]) produces estimates of the spectrum power in different frequency bands using adaptive filtering techniques, but is not designed to produce an estimate of the spectral density itself. In this paper, we show how wavelet techniques can be used in a spectrum estimation problem for combining information about the spectrum at different resolutions. The concept has been suggested in [2], but was not further developed. The method developed here is based on wavelet smoothing of the log-periodogram and captures significant components of the log-spectrum at different resolution levels. There are two essential motivations for estimating the logarithm of the spectrum instead of the spectrum itself. The first is the need to preserve positivity of the spectrum estimates. The second is that the log-periodogram may be modeled as samples of the logspectrum corrupted by additive non-gaussian white noise, as discussed by Wahba [4][5]. In fact, the log transform is the variance-stabilization technique for the spectrum estimation problem [7, p. 470]. In speech processing, the log-spectrum is routinely estimated for the purpose of analyzing the frequency response of the vocal tract, and extracting the formants and the pitch of the speech signal [10].

3 - 3 - Wavelet smoothing techniques capitalize on the different properties of signal and noise wavelet components. Techniques such as thresholding or shrinkage of the noisy wavelet coefficients, and reconstruction from the local minima of the wavelet transform, have been successfully used in a variety of denoising problems [11-18]. In the spectrum estimation problem, the wavelet coefficients of the additive noise are non-gaussian and mutually dependent. However, a large-sample analysis shows that the distribution of these coefficients converges to an independent, identically distributed (iid) Gaussian distribution at coarse scales. This property suggests that estimation techniques based on an independent-noise assumption may be applied. Our approach is then to detect the presence of components of the log-spectrum by application of a binary hypothesis test on each empirical wavelet coefficient. The hypothesis test is equivalent to the application of a hard threshold to the empirical wavelet coefficients. Log-spectrum estimates are obtained by applying the inverse wavelet transform to the thresholded coefficients. Interestingly, hypothesis testing is often used to detect spectral lines in the spectrum [7][19]. The wavelet thresholding framework can be viewed as an extension of this technique, where all spectral components are systematically tested for, including the highly localized components present at fine scales. The approach above has also been extended to problems of restoration of images corrupted by multiplicative noise [18]. The significance level of the test is a critical design parameter which determines the noise/resolution tradeoff. We study several designs and discuss their merits. Particularly relevant here is the work of Donoho and Johnstone on wavelet smoothing under an additive iid Gaussian noise model [16]. After pointing out that the residual noise typically takes the annoying form of isolated blips, they show that the probability that the largest noise coefficient exceeds the particular threshold σ 2 lnn is asymptotically negligible, where N is the sample size and σ is the standard deviation of the noise. In our context, this probability is viewed as the probability of false alarm [20]. Donoho and Johnstone demonstrate that the resulting "visually noise-free" estimates possess a variety of important theoretical properties, including near-ideal spatial adaptation and near-minimaxity [13-16]. In the spectrum estimation problem, similar benefits may be expected from a suitable choice of the probability of false alarm P F. For a specified significance level of the hypothesis test, P F may be computed from the tail probabilities of the noise coefficients. Recently it has been recognized by Gao that the far-tail probabilities are much heavier than Gaussian probabilities, and that with the thresholds σ 2 lnn, P F tends to one for large N [21]. Larger thresholds are needed at

4 - 4 - fine scales, where the departure from the Gaussian model is greatest. Using a large-deviations upper bound on the tail probabilities, Gao has proposed a threshold design for which P F tends to zero for large N. Here we show that for a specified P F, the appropriate thresholds may be computed using a saddle-point approximation to the tail probabilities [22-24]. In Section 2, we present the statistical model for power spectrum estimation. Section 3 contains the mathematical foundations of the wavelet-based estimation approach. In Section 4, we study thresholding of the wavelet coefficients and illustrate the techniques with examples and computer simulations. Finally, Section 5 contains conclusions. 2. STATISTICAL MODEL The following notation is used throughout this paper. The mathematical expectation of a random variable with respect to its distribution is denoted by E[. ]. The probability density function (pdf) for the random variable X is denoted by p X (. ). The standard normal pdf is denoted by p G (. ); the normal cumulative density function (cdf) is erfc (./ 2 ). The notation f (x) g(x) as x x 0 indicates that lim f (x)/ g(x) = 1. x x 0 Consider a real, wide-sense stationary Gaussian random process {x(n), n = 0, ±1, ±2,... } with zero mean and covariance {r(n) = E[ x(m) x(m + n) ], n = 0, ±1, ±2,... }. The power spectrum of x is defined as S( f ) : = Σ r(n) e j2πn f, 1/2 < f 1/2. (2.1) n = In order to ensure the validity of the technical results presented in this paper, we make the somewhat restrictive assumption that S( f ) is strictly positive over the interval [ 1/2, 1/2 ]. The observations are 2 N samples of x, {x(n), 0 n < 2N}. The periodogram, I N ( f ) : = 2N 1 Σ x(n) e j2πn f n = 0 2, 1/2 < f 1/2, (2.2) is symmetric around f = 0 and has the following well-known properties [6, 5.2]. If S is smooth and N is large enough, then, up to a good approximation, the samples {I N (k/2n), k = 0, 1,..., N} satisfy the following model, I N (k/2n) S(k /2N) u(k), k = 0, 1,..., N, (2.3)

5 - 5 - where the random variables {u(k), k = 1,..., N 1} are independently distributed as one half times a χ 2 random variable with two degrees of freedom, and u( 0 ) and u(n) are χ 2 random variables with one degree of freedom. The periodogram samples in (2.3) are asymptotically the sufficient statistics for the spectrum estimation problem. Using a technique introduced by Wahba [4][5], we transform the multiplicative model (2.3) into an additive one by taking a logarithmic transform, ln I N (k/2n) E[ ln u(k) ] = ln S(k /2N) + ε(k), k = 0, 1,..., N. (2.4) In (2.4), the samples {ε(k) : = ln u(k) E[ ln u(k) ], k = 0,..., N} are zero-mean and mutually independent. It is shown in [4-6] that for 0 < k < N, E[ ln u(k) ] = γ , the Euler-Mascheroni constant, and for k = 0 and k = N, E[ ln u(k) ] = ln 2 + γ 1. The variance of ε(k) is π 2 /6 for 0 < k < N, and π 2 /2 for k = 0 and k = N. As in [25], our analysis shall be conducted as if ε( 0 ) and ε(n) were distributed identically to {ε(k), k = 1,..., N 1}, since the influence of these two terms is negligible for N large enough. Then, the model (2.4) becomes ln I N (k/2n) γ = ln S(k /2N) + ε(k), k = 0, 1,..., N, (2.5) where {ε(k), k = 0,..., N} are iid with zero mean and variance π 2 /6. A straightforward analysis shows that their pdf is p ε (x) = e x γ e x γ. (2.6) The periodogram (2.3) or its scaled logarithm (2.5) are asymptotically unbiased but are not consistent in the mean square. For finite N, the data are often tapered in order to reduce the bias and correlation of the periodogram samples [6][7][8][26]. By Theorem in [6], the statistical model (2.3) is then still approximately valid. Numerical Examples The properties of the scaled log-periodogram are illustrated by the following spectrum estimation examples. (a) Spectrum for an autoregressive process AR(2) with a pair of complex poles at z ± = e ±iπ/4 (Fig. 1a). The time series was generated by feeding the Gaussian pseudo-random number generator in [27, p. 216] to the AR filter and discarding the first In [4][5], the incorrect value E[ ln u( 0 ) ] = E[ ln u(n) ] = ( ln 2 + γ)/π is quoted.

6 - 6 - output samples in order to reduce the influence of the initial conditions. The number of data was 2N = (b) The second example is typical of mobile radio communications spectra. The underlying process x is the superposition of two bandlimited, fading, mobile radio signals with firstorder Bessel covariance functions [28], a white background noise with intensity N 0, and a narrow-band interference term with Gaussian spectrum. The power spectrum for x is S( f ) = 1 ( f / B) 2 rect ( f ) + 1 (( f f c )/ B) 2 rect ( f ) [ 0, B] [ f c B, f c + B] + N 0 + A i e ( f f i ) 2 / 2B i 2, where f c = 0. 3, B = 0. 1, f i = 0. 45, B i = , A i = 0. 2, and N 0 = 10 3 (Fig. 1c). A number 2N = 512 of data were generated for this process by simulation of the generalized Fourier integral for stationary processes (Cramer s spectral representation) [7, 4.11]. The integral was discretized to a fine grid of equally spaced discrete frequencies. The data were tapered using a Hamming window. In this example, the taper was necessary to reduce the leakage effects in the periodogram. The scaled log-periodogram for the tapered data has low bias and inter-sample correlations and fits the model (2.3) reasonably well. Fixed-Bandwidth Smoothing The drawbacks of applying single-resolution (fixed-bandwidth) smoothing techniques to the scaled log-periodogram are illustrated in Fig. 2. In Fig. 2a and b, we show the scaled logperiodogram for Example 1, convolved with Hamming windows with bandwidths 128 ( 0. 5/ N) and 32 ( 0. 5/ N), respectively. For the wide bandwidth, the peak at the pole frequency is smeared out. For the narrow bandwidth, the peak is captured correctly, but the spectral estimates far from the peak are too wiggly. In the following section, we show how wavelet techniques can be used to achieve a local adaptation of the bandwidth to the data.

7 WAVELET THRESHOLDING The results in Sections 3 and 4 are applied to the eight wavelets displayed in Fig. 3. These are the Haar wavelets, Daubechies s D4, D6 and D8 wavelets [29, p. 195] as well as their "most regular" counterparts RD4 and RD6 [30], and the most regular coiflets of orders 2 and 3, RC2 and RC3 [30] Discrete Wavelet Transform We assume that N is a multiple of 2 J and consider an orthonormal, discrete wavelet transform (DWT) for signals on the interval [ 0, N 1 ]. The transform is implemented using an iterated bank of subsampled lowpass and highpass quadrature-mirror, finite-impulse-response filters {g(l)} and {h(l)}, respectively. We denote by ψ(x) the (continuous) wavelet associated with the filter bank [29, 5.6]. In order to keep the exposition of basic concepts clear, the theoretical results which motivate our approach are presented for the simple case of periodic wavelets. This is equivalent to assuming that the signals are periodic beyond the boundaries at l = 0 and l = N 1. A more accurate treatment of the boundaries is proposed in Section 3.5. We denote by j and l the scale and location parameters in the discrete index set Λ = { 0 j < J, 0 l < 2 j 1 N } { J, 0 l < 2 J N }, where j = 0 and j = J are the fine and coarse scales, respectively. An explicit formula for the DWT coefficients {y j l } j,l Λ of a signal {x(k), 0 k < N} is y j l = N 1 Σ x(k) w j, (k 2 j l) modn k = 0, j, l Λ, (3.1) where {w j k } 0 k < N denotes the "discrete wavelet" at scale j, namely, the impulse response of the cascade of j 1 lowpass filters {g(l)} and (if j < J) the highpass filter {h(l)}. The signal {x(k), 0 k < N} is reconstructed from {y j l } j,l Λ using the inverse DWT. Denote by b j l : = N 1 Σ ( ln I N (k/2n) γ) w j, (k 2 j l) modn k = 0, j, l Λ, (3.2) and a j l = e j l : = N 1 Σ ln S(k /2N) w j, (k 2 j l) modn k = 0 N 1 Σ ε(k) w j, (k 2 j l) modn k = 0, j, l Λ, (3.3), j, l Λ (3.4)

8 - 8 - the DWT coefficients for the scaled log-periodogram, the discretized log-spectrum and the additive noise ε in (2.5), respectively. From (2.5), we obtain b j l = a j l + e j l, j, l Λ, (3.5) in which {e j l } j,l Λ is interpreted as an additive noise corrupting the wavelet coefficients of the log-spectrum. The estimation problem consists in estimating {a j l } given the empirical wavelet coefficients {b j l }. The estimates for the log-spectrum are then produced by applying the inverse DWT to the estimated {a j l } Statistics of the Wavelet Coefficients The coefficients {e j l } in (3.4) are uncorrelated, since {ε(k), k = 0, 1,..., N 1} are iid and the DWT is orthonormal. Notice that {e j l } are not independent (as would occur if ε were Gaussian) and are not identically distributed. The distribution of a given e j l is independent of the location index l but depends on the scale j. We denote by p j (.) the pdf for e j l ; its variance σ 2 e = π 2 /6 is independent of j. The number of terms in the linear combination (3.4) of iid random variables increases exponentially with j. By application of the central limit theorem for the case of variable components, p j (.) converges to the Gaussian pdf p G (./σ e ) at coarse scales, provided that the Lindeberg conditions on the coefficients in the linear combination are satisfied [31, 8.4, 15.6]. This condition ensures that asymptotically e j l is the sum of "many individual negligible components" [31, 15.6]. A sufficient condition satisfied for the DWTs commonly used in signal processing is given below. Lemma 1. If j. max ψ(x) = M <, then p j (.) converges to the Gaussian pdf p G (./σ e ) as x Proof. Under the hypothesis, max w j k 2 j /2 M 0 as j. This ensures that the Linde- k berg conditions are satisfied [31, 15.6]. The convergence of p j (.) to a Gaussian distribution is illustrated in Fig. 4 in the case of the D4 wavelet. The densities were computed using a standard Fourier technique, which is reasonably accurate for small to moderate arguments (see Appendix). Lemma 1 also suggests that the wavelet coefficients {e j l } j,l Λ may be treated as mutually independent random variables. It is then legitimate to apply statistical estimation techniques based on the assumption of independent

9 - 9 - {e j l }. The validity of this assumption is verified by Monte-Carlo simulations in 4.3. The tail probabilities P j (x) = p j (y) dy x x p j (y) dy : x 0 : x < 0 (3.6) play an important role in this study. When x is not too large, the following large-deviations result applies [31, 16.7]. Lemma 2. If max ψ(x) < and x = o( 2 j /6 ), then P j (x) may be accurately approximated x with the Gaussian tail probability erfc( 2 σ e erfc( x ) tends to zero at the rate O( 2 j /2 x 3 ). 2 σ e x ) in the sense that the relative error P j (x) / 3.3. Inference on Wavelet Coefficients If ln S is smooth enough, the wavelet coefficients {a j l } decay rapidly at fine scales [32]. This behavior is to be contrasted with that of the wavelet coefficients for the noise {e j l }, which have scale-independent variance σ 2 e. This property is illustrated in Fig. 5, where the DWT coefficients for the log-spectrum and for the scaled log-periodogram in Examples 1 and 2 are displayed at various scales. The DWT is computed using the RD6 filters. It can be seen that the noise corrupts wavelet coefficients at all scales equally, while the signal contributions appear either at coarse scales or in the vicinity of abrupt changes in the spectrum. The different behavior of the signal and noise coefficients can be used to discriminate between signal and noise components of the observations, so that only significant wavelet components of the log-spectrum, regardless of their scale, are retained in the estimates. Inference problems of this type are common in the statistics literature (the coefficients usually are coordinates of a signal in other bases). If a prior distribution for the wavelet coefficients is available, Bayesian estimates of the coefficients can be obtained, but no such assumption is made here. A possible idea is then to shrink small coefficients towards zero, since those are likely to be due to the noise [11]. This may be done in a decision-theoretic framework by testing the null hypothesis H 0 : a j l = 0 against the alternative H a : a j l 0, for each wavelet coefficient j, l Λ. The

10 null hypothesis is selected when the empirical b j l belongs to the acceptance region [λ j, λ j + ] at significance level 0 < α < 1, Pr [ λ j < b j l < λ j + H 0 ] = + λ j p j (x) dx = 1 α. (3.7) λ j Since p j (. ) is asymmetric around 0, λ j is allowed to be different from λ j +. The hypothesis test is equivalent to applying the following thresholding to the empirical wavelet coefficients: â j l = 0 : λ + j < b j l < λ j b j l : else, j,l Λ. From (3.6) and (3.7), we obtain the basic relationship between α, the thresholds, and the tail probabilities, P j (λ j ) + P j (λ + j ) = α. (3.8) Since the thresholds decrease with α, various noise/resolution tradeoffs can be obtained by selecting α appropriately Choice of the Wavelet In signal compression and denoising applications, it is often desired that the wavelet used have properties such as a). b). c). regularity. Produces smooth estimates. vanishing moments. Ensures a fast decay of the wavelet coefficients when the signal is smooth [32]. symmetry. Has a cosmetic effect, and ensures that the estimator is invariant to the transformation f f of the frequency axis. In addition, the odd moments of the wavelet and scaling function are zero. For compactly-supported, smooth, orthonormal wavelets, symmetry may be approximated, but not obtained exactly [29, 8]. For a given filter length, the choice of a wavelet results from a tradeoff between the properties above [30]. Property a). is clearly desirable when the underlying function is smooth. Since the sparsity of the wavelet representation is essential to the success of the thresholding technique, Property b). is also important. The benefits of vanishing moments are of the same nature as those arising from fixed-bandwidth smoothing using high-order kernels [33, 4.6]. On a cautionary note, it should be mentioned that kernels with more than two or three vanishing moments have

11 limited usefulness, as extensive studies have shown. Similar conclusions are conjectured to apply in wavelet smoothing Boundary Conditions The easiest solution to the boundary problem is to construct a periodic extension of the signal. Unfortunately, this introduces a discontinuity at the boundaries. After thresholding of the DWT coefficients, the spectral estimates exhibit typical artifacts due to the Gibbs phenomenon. A possible remedy is to use special wavelets near the boundaries [29, 10.7][34], a solution analogous to the use of special boundary kernels in linear smoothing [33, 4.4]. Another solution, which has the advantage of using standard wavelet software, consists in applying the periodic DWT to a full period of the scaled log-periodogram (2 N points). The spectral estimates are obtained by thresholding the 2N DWT coefficients, applying the inverse DWT, and displaying the N positive-frequency samples. This approach is equivalent to introducing extra coefficients to handle the boundaries. It should be noticed that the assumption of asymptotic mutual independence of the wavelet coefficients breaks down near the boundaries, so the thresholding technique is not expected to perform as well as in the interior of the interval. 4. THRESHOLD DESIGN The wavelet coefficients at scale J play a special role. Since they describe the coarse features of the signal, these coefficients are presumed to contain relevant information, so no thresholding is applied to them. The thresholds at other scales are determined as follows. The probability of false alarm P F is the probability that the largest noise wavelet coefficient is smaller than λ j or larger than λ + j. From (3.8), P F is upper-bounded by Pˆ F = 1 Π ( 1 P j (λ j ) P j (λ + j )) j,l Λ (4.1a) = 1 ( 1 α) N( 1 2 J ), (4.1b) with equality if {e j l } can be treated as mutually independent random variables. For small P F and sufficiently large J, Pˆ F Nα. Our objective is to compute thresholds that achieve, or at least approximate, a determined probability of false alarm P F. This may be done using various approximations to P j (.) in (4.1a). For small P F, the relative error on P F is approximately equal

12 to the relative error on {P j (. )} Gaussian Approximation In their study of wavelet shrinkage for signals corrupted by Gaussian noise, Donoho and Johnstone indicate that for the choice Pˆ F tends to zero for large N. Indeed, λ j = λ + j = σ e 2 lnn, (4.2) Pˆ F = N erfc ( lnn ) (π lnn) 1/2 as N. (4.3) By application of Lemma 2 in Section 3.2, the Gaussian tail probability approximation is valid when 2 lnn = o( 2 j /6 ). Although the approximation is always justified for N and j large enough, its practical value is unfortunately limited due to the slow growth of the function 2 j /6 in the range of interest, e.g., 1 2 j /6 2 for the seven finest scales. At fine and moderate scales, the actual tail probability evaluated at (4.2) is generally substantially larger than that given by the Gaussian approximation [21]. For instance, for N = 512, λ in (4.2) and P F 0. 2 using the Gaussian approximation (4.3). However, a more accurate evaluation of (4.1) using the saddle-point approximation of the next section reveals that, in fact, Pˆ F 0. 7 for the wavelet RD Saddle-Point Approximation A good approximation to P j (.) is needed at fine and moderate scales. The Fourier technique for computing P j (x) performs poorly for large x due to numerical instabilities (see Appendix). Cramer s large deviations upper bound [35, 3] is not tight, in the sense that the relative error on P j (x) is unbounded. A more suitable technique is the saddle-point approximation [22-24]. The saddle-point approximation is the first term in an asymptotic series expansion for P j (.), as described in the Appendix. It is displayed on a log scale in Fig. 6 at different scales, for a variety of wavelets. The size of the next-order correction term is typically no greater than 30% at the finest scale and decreases at coarser scales. The next step is to choose the thresholds so as to satisfy (3.8) with α = 1 ( 1 Pˆ F ) N 1 ( 1 2 J ) 1 Pˆ F / N. There is a continuum of solutions. A simple choice is (in implicit form)

13 P j (λ j ) = P j (λ j + ) = α/2 Pˆ F /2N. (4.4) Of course, other choices are possible, including the Neyman-Pearson design which minimizes λ + j λ j [20] Monte-Carlo Simulations In order to assess the validity of the assumption of independent noise wavelet coefficients, the following Monte-Carlo simulations were conducted. For given Pˆ F, the thresholds were designed from (4.4) using the saddle-point approximation to {P j }. The true P F was evaluated from a succession of M independent Bernouilli trials, each of which consisted in computing the DWT coefficients for a N = 512-point sample of p ε and detecting the occurrence of false alarms. The number of trials was chosen to be M = 400/ Pˆ F so that the standard deviation of the empirical frequency of false alarms would be 0. 05Pˆ F. The MN random samples were generated using the random number generator in [27, p. 207]. P F was evaluated for a range of values of Pˆ F between 0.01 and The plots for the Haar, RD6 and RC3 wavelets are shown in Fig. 7a-c along with the 95%-confidence intervals for P F. The results indicate that Pˆ F is a useful upper bound on P F, so that the assumption of independent noise wavelet coefficients seems reasonable Numerical Examples We have applied the Gaussian design (4.2) and the thresholding assignment (4.4) to Examples 1 and 2 using the following wavelets: RD6 wavelet with support width L = 6 (Fig. 3f). This wavelet is differentiable (the Hȯ. lder coefficient of the derivative is conjectured to be ), markedly asymmetric, and has 3 vanishing moments. RC3 coiflet with support width L = 18 (Fig. 3h). This wavelet is twice differentiable, reasonably symmetric, and has 6 vanishing moments. The scaling function has 5 vanishing moments. The resulting estimates are displayed in Figs. 8 and 9 and illustrate the spatial adaptation of the bandwidth to the data. The left and right columns show RD6 and RC3 wavelet estimates, respectively. The estimates obtained using the Gaussian threshold design (Figs. 8b, 8f, 9b and 9f) were corrupted by occasional noise spikes, as expected given the relatively high probabilities of false alarm (0.7 in Fig. 8). The bottom four plots in Figs. 8 and 9 illustrate how the choice of Pˆ F impacts the noise/resolution tradeoff. In the case of Example 1, using RD6, a signal component

14 at the pole frequency was detected for Pˆ F = (Fig. 8c), but not for Pˆ F = (Fig. 8d). Both estimates were visually noise-free. In Figs. 8g and 8h, the detected signal components were identical for both values of Pˆ F, but a noise component appeared for Pˆ F = In Fig. 9, the choice Pˆ F = was clearly superior to Pˆ F = 0. 05, as several significant features of the logspectrum (interferer and transition regions for bandpass signal) were discarded in the former case. The numerical results are summarized in Table 1. These simulations also clearly show that the choice of the wavelet has a significant impact on both the visual aspect and the residual meansquare error of the estimator. 5. CONCLUSION We have shown how wavelet techniques may be applied to perform a nonlinear smoothing of the log-periodogram and produce an automatic adjustment of the bandwidth to the data. We have proposed a wavelet thresholding scheme for which the wavelet coefficients of the noise may be treated as independent random variables. The presence of log-spectrum components is decided from binary hypothesis tests. The significance level of the test determines the desired noise/resolution tradeoff, and the thresholds are determined from the distribution of the noise coefficients. Central-limit-theorem results apply at coarse scales; however, the non-gaussian character of the distribution should be taken into account at fine and moderate scales. The saddle-point approximation has been shown to be an effective tool for computing thresholds that attain a specified probability of false alarm. The estimation techniques studied in this paper do not assume a priori knowledge about the underlying spectrum, besides the presumption that the signal contains significant coarse-scale coefficients. When a priori information is available, special techniques may be developed to improve performance. Such information may take the following form : (1) discrete components are present in the spectrum; (2) the spectrum is presumed to satisfy a certain Lipschitz condition, so that the rate of decay of wavelet coefficients at increasing scales can be determined [11][17][29, 9]; (3) the log-spectrum can be modeled as a multiresolution stochastic process [36]. Developing spectrum estimation techniques incorporating such information represents in our view a challenging, open problem.

15 Appendix. Computation of p j (x) and P j (x). By application of [37 p.930], the cumulant-generating function for ε in (2.6) is K(s) : = ln e st p ε (t) dt = sγ + lnγ( 1 + s). Using the equality cumulant-generating function for e j l in (3.4), Σ w j k k = 0, we obtain the K j (s) = Σ lnγ( 1 + w j k s). k (A1) 1. Fourier Transform The pdf and cdf of e j l are the Fourier transforms of e K j (iω) and e K j (iω), respectively. Both iω transforms may be discretized and evaluated using the Fast Fourier Transform. Unfortunately, this technique fares poorly in the far tails of the distribution due to numerical instabilities [23]. 2. Saddle-Point Approximation Technique The function lnγ(z) in (A1) has simple poles at x = 0, 1, 2,.... The Laplace inversion formula P j (x) = sgn (c) 2πi c + i e sx + K j (s) c i ds, c 0, s converges for c in the real interval I j : = [ ( max w j k ) 1, ( min w j k ) 1 ]. The number of nonzero terms in the linear combination (3.4) is L j k k = O( 2 j ). An asymptotic expansion for P j (x) in terms of L j may be obtained using a saddle-point approximation technique [22-24], suitably generalized to account for unequal coefficients {w j k }. c is chosen to be the real saddle point s * of h(s) : = sx + K j (s), Re(s) I j. Since h(s) is convex over I j, s * is the unique solution of h (s) = x + Σ w j k Ψ( 1 + w j k s) = 0, where Ψ(z) : = dlnγ(z)/ dz is the Psi function [37]. k The equation h (s) = 0 is solved numerically as follows. The asymptotic series in [37], Ψ( 1 + z) = 0. 5z 1 (π/2) cot (πz) ( 1 z 2 ) γ Σ M β(n) z 2n, z < 2, n = 1 where β( 1 ) = , β( 2 ) = , β( 3 ) = , β( 4 ) = ,..., converges exponentially fast with M. A highly accurate approximation to Ψ( 1 + z) in the range of interest may be obtained by

16 We write P j (x) = sgn (s * ) 2πi s * + i e sx + K j (s) s * i ds. s (A2) The integration path in (A2) is tangent to a steepest descent contour of the exponent, so the main contribution to the integral comes from a small neighborhood of the saddle point s *. Using a Taylor series expansion of the exponent and term-by-term integration, the following asymptotic expansion of (A2) is derived, P j (x) = 1 erfc ( z * / 2 ) ( 1 + 2π p G Σ (z * R k ), ) k = 1 e s * x + K j (s * ) (A3) where z * = s * K j (s * ) = σ e s *, and the high-order asymptotic terms R k are polynomial functions of the standardized cumulants {K j (r) (s * ) [K j (s * ) ] r /2 } 3 r k + 2. Under the asymptotic approximation w j k 2 j /2 ψ( 2 j k) as j, we have K j (r) (s) = Σ w j r k K (r) (w j k s) K (r) ( 0 ) k Σ w j r k = O( 2 j( 1 r /2) ), k where K (r) ( 0 ) is the r-th cumulant of ε. A straightforward calculation then shows that R k = O( 2 j( 1 k /2 ) ). In the special case of Haar wavelets, e j l is a normalized sum of iid random variables, and (A3) is Lugannani and Rice s asymptotic expansion of P j (x) in terms of powers of L 1/2 j [23][24]. The first-order approximation Pˆ j (x) = 1 erfc ( z * / 2 ) 2π p G (z * ) e s * x + K j (s * ) (A4) is known as the saddle-point approximation to P j (x). The accuracy of this approximation is uniform in t, reasonably good even for j = 0, and improves rapidly at coarser scales. These properties are illustrated by the results in Table 2, where max R 1, the worst-case first-order correc- x tion term, is displayed for a variety of wavelets at different scales. It can be seen that at the finest level, this quantity is no larger than 30% for all of the wavelets considered. At coarser scales, the relative error Pˆ j (x)/ P j (x) 1 tends to zero since the saddle-point approximation is exact for the Gaussian distribution [22]. using M = 4 or 5. The solution of h (s) = 0 is obtained after a small number of iterations of a modified Newton-Raphson algorithm such as the one in [27].

17 REFERENCES [1] Y. Chow and U. Grenander, "A Sieve Method for the Spectral Density," Annals of Statistics, Vol. 13, No. 3, pp , [2] P. Moulin, J. A. O Sullivan and D. L. Snyder, "A Method of Sieves for Multiresolution Spectrum Estimation and Radar Imaging", IEEE Trans. on Information Theory, Special Issue on Wavelet Transforms and Multiresolution Analysis, Vol. 38, No. 2, pp , [3] R. Cogburn and H. T. Davis, "Periodic Splines and Spectral Estimation," Annals of Statistics, Vol. 2, No. 6, pp , [4] G. Wahba and S. Wold, "Periodic Splines for Spectral Density Estimation: The Use of Cross Validation for Determining the Degree of Smoothing," Communications in Statistics, 4(2), pp , [5] G. Wahba, "Automatic Smoothing of the Log Periodogram," Journal of the American Statistical Association, Vol. 75, No. 369, pp , [6] D. R. Brillinger, Time Series data Analysis and Theory, Holt, Rinehart and Winston, New York, [7] M. B. Priestley, Spectral Analysis of Time Series, Vol. I, Academic Press, London, [8] G. M. Jenkins and D. G. Watts, Spectral Analysis and Its Applications, Holden-Day, Oakland, CA, [9] J. Capon, "High-Resolution Frequency-Wavenumber Spectrum Analysis," Proceedings IEEE, Vol. 57, pp , [10] L. R. Rabiner and R. W. Schafer, Digital Processing of Speech Signals, Prentice-Hall, New Jersey, [11] S. G. Mallat and W. L. Hwang, "Singularity Detection and Processing with Wavelets," IEEE Trans. on Information Theory, Special Issue on Wavelet Transforms and Multiresolution Analysis, Vol. 38, No. 2, pp , [12] J. Lu et al., "Noise Reduction with a Multiscale Edge Representation and Perceptual Criteria," Proc. IEEE-SP Int l Symp. on Time-Frequency and Time-Scale Analysis, Victoria, B. C., pp , 1992.

18 [13] D. L. Donoho and I. M. Johnstone, "Ideal Spatial Adaptation by Wavelet Shrinkage," Dept of Statistics Technical Report, Stanford University, [14] D. L. Donoho, I. M. Johnstone, G. Kerkyacharian and D. Picard, "Wavelet Shrinkage: Asymptotia?" Dept of Statistics Technical Report, Stanford University, [15] D. L. Donoho, "De-Noising by Soft-Thresholding," Dept of Statistics Technical Report, Stanford University, [16] D. L. Donoho and I. M. Johnstone, "Minimax Estimation via Wavelet Shrinkage," Dept of Statistics Technical Report, Stanford University, [17] I. M. Johnstone, G. Kerkyacharian and D. Picard, Estimation d une densite de probabilite par me thode d ondelettes," Comptes Rendus de l Acade mie des Sciences de Paris (A), Vol. 315, pp , [18] P. Moulin, "A Wavelet Regularization Method for Diffuse Radar-Target Imaging and Speckle-Noise Reduction," Journal of Mathematical Imaging and Vision, Special Issue on Wavelets, Vol. 3, No. 1, pp , [19] D. J. Thompson, "Spectrum Estimation and Harmonic Analysis," Proceedings IEEE, Vol. 70, No. 9, pp , [20] H. L. Van Trees, Detection, Estimation, and Modulation Theory, Part I, Wiley, New York, [21] H.-Y. Gao, "Choice of Thresholds for Wavelet Estimation of the Log Spectrum," preprint, UC Berkeley, Dept of Statistics, [22] N. Reid, "Saddlepoint Methods and Statistical Inference," Statistical Science, Vol. 3, No. 2, pp , [23] R. Lugannani and S.Rice, "Saddle Point Approximation for the Distribution of the Sum of Independent Random Variables," Advances in Applied Probability, Vol. 12, pp , [24] H. E. Daniels, "Tail Probability Approximations," International Statistical Review, Vol. 55, No. 1, pp , [25] H. D. Brunk, "Bayes Least Squares Linear Regression is Asymptotically Full Bayes: Estimation of Spectral Densities," pp in Topics in Non-Gaussian Signal Processing, Eds. E. J. Wegman, S. C. Schwartz and J. B. Thomas, Springer-Verlag, 1991.

19 [26] D. R. Brillinger, "The Key Role of Tapering in Spectrum Estimation," IEEE Trans. on Acoustics, Speech and Signal Processing, Vol. 29, No. 5, pp , [27] W. H. Press et al., Numerical Recipes in C, Cambridge University Press, [28] W. C. Jakes, Jr., "Mobile Radio Propagation," Ch. 1 in Microwave Mobile Communications, Ed. W. C. Jakes, Jr., Wiley, [29] I. Daubechies, Ten Lectures on Wavelets, CBMS-NSF Regional Conference Series in Applied Math., SIAM, Philadelphia, [30] I. Daubechies, "Orthonormal Bases of Compactly Supported Wavelets II. Variations on a Theme," SIAM Journal of Mathematical Analysis, Vol. 24, No. 2, pp , March [31] W. Feller, An Introduction to Probability Theory and Its Applications, Vol. II, Wiley, [32] S. G. Mallat, "Multiresolution Approximations and Wavelet Orthonormal Bases of L 2 (R)," Trans. of the American Mathematical Society, Vol. 315, No. 1, pp , [33] W. Hardle, Applied Nonparametric Regression, Cambridge University Press, U. K., [34] A. Cohen, I. Daubechies and P. Vial, "Wavelets on the Interval and Fast Wavelet Transforms", to appear in Journal of Applied and Computational Harmonic Analysis, [35] S. R. S. Varadhan, Large Deviations and Applications, CBMS-NSF Regional Conference Series in Applied Math., SIAM, Philadelphia, [36] M. Basseville et al., "Modeling and Estimation of Multiresolution Stochastic Processes," IEEE Trans. on Information Theory, Special Issue on Wavelet Transforms and Multiresolution Analysis, Vol. 38, No. 2, pp , [37] M. Abramowitz, I. A. Stegun, Handbook of Mathematical Functions, Dover, New York, 1972.

20 List of Tables Table 1. Numerical results for Examples 1 and 2. Table 2. Size of the first-order correction term for saddle-point approximation to P j (x). List of Figures Fig. 1. Log-spectrum (dashed line) and scaled log-periodogram (dotted line) for (a) Example 1, (b) Example 2. Fig. 2. Linear smoothing of log-periodogram data using a Hamming window: (a) bandwidth = 128 ( 0. 5/ N), (b) bandwidth = 32 ( 0. 5/ N). Fig. 3. Wavelet plots for (a) Haar, (b) D4, (c) D6, (d) D8, (e) RD4, (f) RD6, (g) RC2, (h) RC3. Fig. 4. Pdf for wavelet coefficients of the additive noise at different scales, using D4 wavelets. Also included for comparison is the Gaussian pdf p G (x 6 /π). Fig. 5. DWT coefficients for the log-spectrum (dashed line) and scaled log-periodogram (dotted line) using RD6 wavelets, for (a) Example 1, (b) Example 2. Fig. 6. Logarithm of tail probabilities for wavelet coefficients at different scales using (a) Haar, (b) D4, (c) D6, (d) D8, (e) RD4, (f) RD6, (g) RC2, (h) RC3 wavelets. Also included for comparison is the logarithm of the Gaussian tail probability. Fig. 7. Assessment of the validity of the assumption of independent noise coefficients. The probabilities of false alarms Pˆ F and P F are compared based on Monte-Carlo simulations for (a) Haar, (b) RD6, (c) RC3. Fig. 8. Log-spectrum (dashed line) and wavelet estimate (solid line) for Example 1, obtained for RD6, with (a) scaled log-periodogram, (b) α = 0. 25, (c) α = 0. 05, (d) Gaussian thresholds, and for RC3, with (e) scaled log-periodogram, (f) α = 0. 25, (g) α = 0. 05, (h) Gaussian thresholds. Fig. 9. Log-spectrum (dashed line) and wavelet estimate (solid line) for Example 2, using the same techniques as in Fig. 8.

21 DWT Thresholding Thresholds λ J j Bias Var + λ j Technique j = 0 EXAMPLE 1 (N = 512) 0 N/A periodogram, 0, 0, 0, 0, RD6 Gaussian design (4.2), , , , , , RD6 Pˆ F = (4.4), , , , , , RD6 Pˆ F = (4.4), , , , , , RC3 Gaussian design (4.2), , , , , , RC3 Pˆ F = (4.4), , , , , , RC Pˆ F = (4.4), , , , , , EXAMPLE 2 (N = 256) 0 N/A periodogram, 0, 0, 0, 0, RD6 Gaussian design (4.2), , , , RD6 Pˆ F = (4.4), , , , RD6 Pˆ F = (4.4), , , , RC3 Gaussian design (4.2), , , , RC3 Pˆ F = (4.4), , , , RC3 Pˆ F = (4.4), , , ,

22 Scale j Wavelet type Haar D D D RD RD RC RC