H. Krim and J.-C. Pesquet Following a section of preliminaries and denitions of notations, we derive in Section 3 a procedure for an estimation enhanc

Transcription

1 On the Statistics of Best Bases Criteria? H. Krim and J.-C. Pesquet Stochastic Systems Group, LIDS, Massachusetts Institute of Technology, Cambridge MA 039, USA Laboratoire des Signaux et Systemes, CNRS/UPS and GDR TdSI, ESE, 99 Gif sur Yvette Cedex, France To appear in Springer-Verlag : Wavelets and Statistics Lecture Notes in Statistics Series Abstract Wavelet packets are a useful extension of wavelets providing an adaptive timescale analysis. In using noisy observations of a signal of interest, the criteria for best bases representation are random variables. The search may thus be very sensitive to noise. In this paper, we characterize the asymptotic statistics of the criteria to gain insight which can in turn, be used to improve on the performance of the analysis. By way of a well-known information-theoretic principle, namely the Minimum Description Length, we provide an alternative approach to Minimax methods for deriving various attributes of nonlinear wavelet packet estimates. Introduction Research interest in wavelets and their applications have tremendously grown over the last ve years. Only, more recently, however, have their applications been considered in a stochastic setting [Fl, Wo, BB +, CH]. A number of papers which have addressed the optimal representation of a signal in a wavelet/wavelet packet basis, have for the most part given a deterministic treatment of the problem. In [Wo], a Karhunen-Loeve approximation was obtained for fractional Brownian motion with the assumption that the wavelet coecients remained uncorrelated. In [Un, PC], optimal wavelet representations were derived for the analysis of stationary processes. Similar problems can be investigated with a goal of enhancing the estimation of an underlying signal embedded in noise [DJ, LP +, Mo]. More recently, a statistical approach to a best basis search was undertaken in [KPW, DJ]. In this paper, we study the statistical properties of various bases search criteria which have been proposed in the literature. These can then be used to rigorously proceed to a wavelet packet tree search 3 formulated as a hypotheses test.? The work of the rst author was in part supported by grants from ARO (DAAL03-9-G-05) (Center for Intelligent Control), AFOSR (F ) and NSF (MIP-9058). 3 A search for an adaptive local cosine basis could just as well be carried out.

2 H. Krim and J.-C. Pesquet Following a section of preliminaries and denitions of notations, we derive in Section 3 a procedure for an estimation enhancement of a signal embedded in noise, by using information-theoretic arguments. A Minimum Description Length (MDL) [Ri] analysis which achieves that, results in the shortest coding length for an observed process. An interesting connection between this length and a best basis criterion recently proposed in [DJ] is outlined. In Section 4, statistical properties of this criterion and of an entropy-like or L p criterion are derived. These allow one to assess the variability and the potential eect of noise on these criteria, and aord the possibility of constructing decision algorithms. Finally, we give some concluding remarks in Section 5. Preliminaries and Formulation. Wavelet Packet Decomposition The wavelet packet decomposition [Wi] is an extension of the wavelet representation, and allows a \best" adapted analysis of a signal. To dene wavelet packets, we rst introduce real functions of L (IR), W m (t), m IN, such that Z? W 0 (t) dt = () and, for all (k; j) Z, respectively representing a translation parameter and a resolution index,? Wm ( t? k) =? Wm+ ( t? k) = h l?k W m (t? l) and; () g l?k W m (t? l); (3) where m denotes the frequency bin number and (h k ) kzz, (g k ) kzz are the lowpass and highpass impulse responses of a paraunitary Quadrature Mirror Filters (QMF) [Da]. A convenient choice for g k is g k = (?) k h?k (4) and the QMF property then reduces to h l h l?k = k (5) where ( k ) kzz is the Konecker sequence. To dene compactly supported functions W m (t), we can use nite impulse response lters of (necessarily even) length L such that h k = 0 ; if k?l= or k > L= : (6) If we denote by P a partition of IR + into intervals I = [?j m; : : : ;?j (m+)[, j Z and m f0; : : : ; j? g, then f?j= W m (?j t? k); k Z; (j; m)=i Pg is an orthonormal basis of L (IR). Such a basis is called a wavelet packet [Wi]. The coecients resulting from the decomposition of a signal x(t) in this basis are

3 Best Bases Criteria 3 C k (x) = Z? x(t) W m( t? k) dt : (7) j= j For ease of notation, we will omit the variable \(x)" in C k (x), whenever there is no ambiguity. Note that C k j+;m = h l?k C l ; C k j+;m+ = g l?k C l (8) and, for j 0, where h k j+;m = C k = h l h k?j l ; h k j+;m+ = C l 0;0 h l?j k (9) g l h k?j l (0) and h k 0;0 = k. By varying the partition P, dierent choices of wavelet packets are possible. For instance, a special wavelet packet is the orthonormal wavelet basis dened by the scaling function (t) = W 0 (t) and the mother wavelet (t) = W (t). Another particular case is the equal subband analysis which is dened, at a given resolution level j m Z, by P = fi jm;m; m INg. The basis selection is made to adapt to the underlying signal of interest, and various decision criteria have been proposed in the literature and are discussed in the next section.. Energy Concentration Measures An ecient tree search algorithm was rst proposed by Coifman and Wickerhauser [CW] to determine the partition P which leads to a maximal Energy Concentration Measure (ECM). For the sake of algorithmic eciency, this ECM I() is additive, i.e. for every sequence (a k ) kk, I((a k ) 0k<K ) = K? I(a k ) () with the notational convention I(fa k g) = I(a k ). The ECM of choice should result in the \best" adapted basis. Among the better known ECMs, is an entropy-like criterion, 4 dened by and the L p criterion, with p IN n f0; g, I(a) = a log(a ) () I(a) = a p : (3) 4 We here consider the unormalized form of entropy, with an opposite sign of the usual convention.

4 4 H. Krim and J.-C. Pesquet.3 Model Our focus in this paper is on the multiscale analysis of a continuous time process x(t) observed over a time interval. We assume an additive noise model, x(t) = s(t) + b(t) (4) where b(t) is a real zero mean Gaussian white noise with a known power spectral density (psd). The signal s(t) is assumed unknown. We will assume that this signal is real and belongs to SpanfW 0 (t? k); k Zg, so that we have to just consider the projection of Eq. (4) onto this space to estimate s(t). The latter condition amounts to some weak regularity condition on s(t). Furthermore, s(t) is assumed to have a compact support, so that, 9K IN n f0g ; C k (s) = 0 if k < 0 or k K?j ; (5) where K designates the number of wavelet packet coecients retained at the resolution level j = 0. This means that s(t) must be estimated from fc(x); k 0 k < K?j ; (j; m)=i Pg, where P is a partition of [0; [ in intervals I. The wavelet packet coecients C(x) k are the result of a linear transformation and are therefore also Gaussian with means C(s). k Given that this transform is orthogonal, they have a variance and are furhtermore independent. 3 Nonlinear Estimation of Noisy Signals Using information-theoretic arguments in concert with the statistical properties of the assumed noise, we wish to investigate the potential improvement of a multiscale analysis in enhancing the estimate of s(t). Intuitively, our approach here is to use to advantage the spectral and structural dierences of the underlying signal s(t) and those of the noise b(t) across scales, to separate their corresponding components and subsequently eliminate the noise. We proceed by relabeling the wavelet packet coecients of x(t) with a single indexing subscript, and reformulate the problem as one of estimating signal coecients fc n (s)g nk embedded in an additive N(0; ) white noise, from observations fc n (x)g nk. Since fc n (s)g nk represents the coecients of the signal in an adapted orthonormal basis, it is reasonable to assume that s(t) is adequately represented by a small number P < K of orthogonal directions, in contrast to white noise, which necessarily would be present in all the available directions. 5 In a sense, the noise components add no information to understanding the signal. This notion can also be interpreted as an attempt to code the information in the observed process or evaluate its complexity. For that, we call upon the MDL principle. The rationale for this criterion is that the best code fc n (s)g nk for a data sequence is the one which not only best explains it, but also is the shortest. Recalling that the coecients are assumed to be independent, it follows that the joint probability density function is, 5 This is in particular veried under some regularity conditions on s(t), as most of the signal energy is concentrated in a few dimensions.

5 Best Bases Criteria 5 f(c (x); : : : ; C K (x) j ) = () K= K e? ( P P l= (Cn l (x)?cn l (s)) + P K l=p + Cn l (x) ) (6) = (n ; : : : ; n P ; C n (s); : : : ; C np (s)) : (7) where P is the number of \principal directions" of the sequence fc n (s)g nk, which is assumed to satisfy C nl (s) 6= 0 i l P (8) and is the parameter vector. The unknown parameters are the P coecients fc nl (s)g lp and their respective locations fn l g lp for which one could search the maximum of the likelihood hypersurface. The drawback of this direct approach is that it will always maximize the likelihood function by maximizing P. The solution provided by the MDL criterion, attaches a penalty and prevents such a naive optimization. The code length described by the MDL is given as, L(C (x); : : : ; C K (x); ; P ) =? log f(c (x); : : : ; C K (x) j ) + (P ) log K : (9) Proposition 3. The P coecients C (x); : : : ; C P (x) which, based upon the MDL method, give the optimal coding length of x(t), are determined by the components which satisfy the following inequality: j C n (x) j> (0) where = p log K. Furthermore, the resulting minimal code length is ^L(C (x); : : : ; C K (x)) = K n= min( C n(x) ; ) + K log( p ) : () Proof. For algebraic convenience, we reorder the variables fc n (x)g nk in f( j ) such that j C (x) j j C (x) j j C K (x) j : () Clearly, minimizing L() leads to the maximum likelihood estimates of fn l g lp and fc nl (s)g lp as ^n l = l and ^C nl (s) = C l (x). Ignoring the terms independent of P, we obtain, L 0 (C P + (x); ; C K (x); P ) = K C n (x) + P log K : (3) The nite dierences of L 0 () w.r.t. P are n=p + L 0 (C P + (x); ; C K (x); P )? L 0 (C P (x); ; C K (x); P? ) =? C P (x) + log K : (4) This means that, for any P less than (resp. greater than) ^P, the functional decreases (resp. increases), where the optimal value ^P is the largest P such that j C P (x) j> (5)

6 6 H. Krim and J.-C. Pesquet This is equivalent to thresholding the coecients as expressed by (0) and Expression () straightforwardly follows. ut Note that this result coincides with that previously derived by Donoho and Johnstone [DJ] and achieves a Min-Max error of representation of the process x(t) in a wavelet basis. MDL-based arguments were also used by Moulin [Mo] in a spectral estimation problem and more recently by Saito [Sa] to enhance signals in noise of unknown variance. Interestingly, the minimum coding length in Eq. () was recently proposed as part of a criterion for the search of a best basis of a process [DJ]. 6 This criterion is additive, thus algorithmically ecient for a tree search, and results in a representation of minimal complexity. This tree search criterion will subsequently be referred to as the denoising criterion, 4 Statistical Properties of Criteria 4. Properties of ECMs I(a) =? min( a ; ) : (6) The best basis representation as rst proposed by Wickerhauser [Wi] adopted a deterministic approach. In the presence of noise, the cost function, however, is a random variable, and its deterministic use may result in a highly variable representation. The following proposition describes the asymptotic behavior of I(fC k g 0k<K?j) : Proposition 4. If fc k g 0k<K?j is an i.i.d. sequence, then 7 I(fC k g 0k<K?j)? K?j p K?j= N(0; ) K?j! (7) where = EfI(C)g k ; = VarfI(C)g k : (8) Furthermore, when C k is N(0; ), we have for the entropy criterion, 8 = (? log? + log ); (9) + = 3 4 (?40=9 + = + (8=3? + log + log ) ) (30) for the L p criterion, = (p)! p p! p (3) (4p)! + = p (p)! 4p (3) 6 The authors however use a value of higher than p log K to guarantee good estimation performances. 7 The symbol \" stands here for the convergence in law. 8 The Euler's constant is denoted by by = lim n! P n k= k? log n ' 0:577.

7 Best Bases Criteria 7 and, for the denoising criterion, = (f() + (? )F () + =? ) (33) + = ((3? 4 )F ()? ( + 3)f() + 4? 3=) (34) where f() = e? = = p ; F () = Z? f() d : (35) Proof. Invoking the central limit theorem allows one to show the asymptotic normality of I(fCg k 0k<K?j). When C k is Gaussian, Appendix A provides the expressions of and, for the entropy criterion. Eqs. (3) and (3) are well-known expressions of moments of Gaussian random variables. Eqs. (33) and (34) may be easily checked by carrying out an integration by parts of the corresponding mathematical expressions. ut Note that the data length K must be suciently large and the resolution of analysis suciently high (j small enough), for the asymptotic behavior of the criteria to hold. 4. Properties of Criteria The usefulness of the ECM in a stochastic framework hinges upon the fact that its statistical properties at each node (j; m) and its corresponding osprings are determined. Specically, the properties of the following algebraic sum of the ECMs (actual criteria) is key to a best basis search (i.e comparison of costs of a parent node and osprings nodes)[wi]: I P = I(fC k g 0k<K?j)? (j 0 ;m 0 )=I j 0 ;m 0P I(fC k j 0 ;m 0g 0k<K?j0) (36) where P is a partition of I in intervals I j 0 ;m0. More specically, we will determine the asymptotic distribution of I P, when K?jm!, with j m = supfj 0 IN; 9m 0 IN=I j 0 ;m 0 P g. We will rst consider the problem for two successive resolution levels j and j +. Proposition 4. If P = fi j+;m ; I j+;m+ g, the coecients resulting from a compactly supported wavelet packet decomposition of a white Gaussian noise with zero mean and psd are such that I P p K?j= N(0; ) K?j! (37) = (; )? L= k=?l=+ ((; h k ) + (; g k )) (38) where (; r) is the covariance of I() and I(Y), when and Y are jointly zero-mean Gaussian random variables of variance and correlation coecient r = EfYg=.

8 8 H. Krim and J.-C. Pesquet Proof. First recall that fcg k 0k<K?j are i.i.d. N(0; ) and the coecients fcj+;mg k 0k<K?j? and fcj+;m+g k 0k<K?j? are i.i.d. N(0; ) and mutually independent. Furthermore, we have EfC k C l j+;mg = h k?l : (39) It can simply be checked that the rst order moment of I P vanishes as I(Cj+;m) k and I(Cj+;m+) k follow the same probability distribution as I(C). k By using the independence of fcj+;mg k 0k<K?j? and fcj+;m+g k 0k<K?j?, the second order moment of I P reduces to VarfI P g = K?j VarfI(C)g k + K?j? (VarfI(Cj+;m)g k + VarfI(Cj+;m+)g) k? K?j? K?j?? l=0 p=0 (CovfI(C l ); I(C p j+;m)g + CovfI(C l ); I(C p j+;m+)g) : (40) As shown by Eq. (39), C k and C l j+;m are independent, when k?l= + l or k L= + l + and, by using Eq. (39), K?j? l=0 K?j?? p=0 CovfI(C l ); I(C p j+;m)g = K?j?? p=0 = K?j? L= p+l= l=p?l=+ l=?l=+ CovfI(C l ); I(C p j+;m)g (; h l ) : (4) An identical approach is applied to evaluate the covariance of I(fC k g 0k<K?j) and I(fC k j+;m+g 0k<K?j?). By further noting that VarfI(C k )g = VarfI(C k j+;m)g = VarfI(C k j+;m+)g = (; ); (4) Eqs. (40) and (4) lead to lim K?j! VarfIP g = lim K?j! K?j ((; )? L= k=?l=+ ((; h k ) + (; g k )) : (43) It remains to establish that I P is asymptotically normal, when appropriately normalized. This result relies on the fact that I P may be rewritten as I P K?j?? = k (44) k = I(C k ) + I(C k+ )? I(C k j+;m)? I(C k j+;m+) : (45) The random variables k are identically distributed and k is only a function of C k?l=+ ; : : : ; C k+l=. This implies that k and l are independent for j k? l j> (L? )=. This property together with central limit theorems in [Ib] nish the proof. ut

9 Best Bases Criteria 9 Expression (38) for the variance can be further simplied, subject to some conditions, as shown below: Corollary 4.3 If I(a) is an even function of (; 0) exists, we can write under the assumptions of Proposition 4., = ((; )? L= k=?l=+ (; h k )) (46) where (; r) = (; r)? (; (; 0)r is an even function of r. Furthermore, have, for the entropy criterion, (; r) = 4 ((? r ) 5= S k = r k (k + ) (k)! k k! S k + 4( + 3r ) log(? r )?( + r )(log(? r ))? 4? 8r ) (47) k l=0 for the L p criterion, l + = (; r) = 4p (p!) and, for the denoising criterion, (; r) = 6 k= p k= (48) (p?k) (k)!(p? k)! rk (49) k?3 k! ( p Q k? ( p ) + Q k?3 ( p )) f() r k (50) where Q k () is the Hermite polynomial of degree k. 9 Proof. It is straightforward to check that (; r) and thus (; r) are even functions of r as I() is also an even function. Formula (46) is derived from Eq. (38) by using Relations (4) and (5). The expressions of (; r) for the considered ECMs are nally established in Appendices A and B. ut For the entropy criterion, we nd that is a linear function of, unlike which was proved to be also dependent upon log. The corresponding expression of (; r) is rather intricate w.r.t. r. It may however be approximated (for r [?; ]) by a Taylor expansion whose convergence is relatively fast : 9 Recall that Hermite polynomials can be computed recursively through the relation Qk+(a) = aqk(a)? kqk?(a), with Q 0 (a) = and Q (a) = a.

10 0 H. Krim and J.-C. Pesquet (; r) = 4 ( 3 r r r r r + (5) r r r8 + O(r 0 )) : (5) It is also worth noting that the result obtained for the L p criterion takes a very simple form when p = as it reduces to (; r) = 4 8 r 4. Proposition 4. may be extended to an arbitrary choice of the partition P. Proposition 4.4 The coecients resulting from a compactly supported wavelet packet decomposition of a white Gaussian noise with zero mean and psd are such that I P p K?j= N(0; ) K?jm! (53) = (; )? (j 0 ;m 0 )=I j 0 ;m 0 P?j0 +j+ k (; h k j 0? 0? j0?j m ) : (54) This result may be proved by proceeding in a way similar to the proof of Proposition 4.. The above propositions are useful in characterizing the sequences of coecients fc k g 0k<K?j resulting from noise. They allow to build statistical tests to determine whether the values of I P caused by noise. 5 Conclusion are statistically signicant in relation to the variations We have outlined some of the connections between information theoretic concepts and statistics. As demonstrated, the Minimum Description Length approach provides an alternative and comprehensive view of the nonlinear wavelet/wavelet packet estimation methods introduced by Donoho and Johnstone. We have also established some asymptotic results on the probability distribution of the additive criteria which are used to adapt wavelet packet representations. As noted, these results allow one to build statistical tests for improving the robustness of the search for the best basis to noise. We are currently numerically evaluating the ecacy of these tests, in various signal/noise scenarios. Appendices A Statistical Properties of Entropy We will need the following result:

11 Best Bases Criteria Lemma A. If Z is a (a; b) random variable, a > 0, b > 0, we have Eflog Zg = (a)? log b (55) Varflog Zg = 0 (a) (56) where () denotes the logarithmic derivative of the Euler's? function. The proof follows from straightforward calculations. We will then calculate the rst and second order moments of log( ), when is N(0; ). We nd after a change of variables that Ef log( )g = Eflog U g (57) Ef 4 (log( )) g = 3 4 Ef(log(U )) g (58) where U and U are respectively (3=; ( )? ) and (5=; ( )? ). Lemma A then yields Ef log( )g = ( ( 3 ) + log + log ) (59) Ef 4 (log( )) g = 3 4 ( 0 ( 5 ) + ( (5 ) + log + log ) ) : (60) Using the properties of the? function results in Eqs. (9) and (30). We now proceed to calculate the crosscorrelation EfI()I(Y)g when I() = log( ) and and Y are jointly Gaussian, zero-mean random variables with variances and Y and correlation coecient r. The crosscorrelation is then dened as EfI()I(Y)g = EfI() EfI(Y) j gg : (6) The expression EfI(Y) j g can be obtained using the conditional distribution p Yj (y), which for a N(0; ) will have the following mean and variance, Yj = r Y (6) Yj = p? r Y (63) where r = EfYg Y is the correlation coecient. We can now evaluate EfI(Y) j g, given that Y is now N( Yj ; ). For a given, if we let Z = Yj Y, it is simple to conclude that Z= is Yj (; ) (i.e. noncentral ) with = Yj Yj = r (? r ) : (64) as By using the expression of the noncentral density, the density of Z may be written

12 H. Krim and J.-C. Pesquet p Z (z) = Yj e? k k! We now want to evaluate EfZ log Zg, EfZ log Zg = or, after some calculations, Z 0 0 e?z= (k + =) e? k k! EfZ log Zg = Yj z Yj 0 e?z= (k + =) k?= A z Yj k+= A ; z > 0 : (65) log z dz (66) e? k (k + =) Eflog Z k g (67) k! where Z k (k + 3=; ( Yj )? ). In other words, according to (55), we can write down, EfZ log Zg = Yj e? k (k + =) ( (k + 3=) + log( k! Yj)) : (68) In the above equation, one can recognize the rst order moment of a Poisson distribution and we thus have EfZ log Zg = Yj( e? k (k + =) k! Replacing by its expression in (64) leads to EfI()I(Y)g = Yj( (k + 3=) + ( + =) log( Yj)) : (69) r k (k + =) k k! k jy (k + 3=)Ef k+ log( )e? r jy g + ( r Ef 4 log( )g + Ef log( )g) log( (? r ) Yj)) (70) with jy = p? r. Furthermore, it can be readily shown that s Ef k+ log( )e? r jy? r g = where U (k + 3=; ( jy )? ). Invoking Lemma A yields EfI()I(Y)g = Y((? r ) 5= k+ k+ jy? (k + 3=)Eflog Ug (7) r k (k + ) (k)! k k! (k + 3=)( (k + 3=) + log( jy)) + (3r ( (5=) + log( )) + (? r )( (3=) + log( ))) log( Yj)) : (7) We therefore nd that EfI()I(Y)g = Y(a (r) log( ) log( Y)+a (r)(log( )+log( Y))+a 3 (r)) (73)

13 Best Bases Criteria 3 where a (r) = + r ; (74) a (r) = (3=) + (3 (5=)? (3=))r =?? log() + (6?? 4 log())r (75) r k (k + ) (k)! ( (k + 3=)) k k! a 3 (r) = (? r ) 5= +a (r) log(? r )? a (r)(log(? r )) : (76) To derive the above expressions, we have used the fact that and Y play symmetric roles, which results in the following relation: (? r ) 5= r k (k + ) (k)! k k! (k + 3=) + a (r) log(? r ) = a (r) : (77) The expression of a 3 (r) can be further simplied by noting that (k + 3=)? (=) = S k = k l=0 l + = (78) which combined with (77), leads to a 3 (r) = (? r ) 5= ( By checking that we nd that r k (k + ) (k)! k k! S k? (=) r k (k + ) (k)! k k! ) +4( + 3r ) log(? r )? a (r)(log(? r )) + (=)a (r) : (79) a 3 (r) = (? r ) 5= r k (k + ) (k)! k k! = (? r )?5= a (r) (80) r k (k + ) (k)! k k! S k + 4( + 3r ) log(? r )?a (r)(log(? r )) + (=)( (=) ( (=) + 6)r ) : (8) By setting = Y and substracting the rst two terms of the Taylor expansion of a 3 (r), we obtain Eq. (47). B Nonlinear Functionals by Price's Theorem Price's theorem [Pr] can be convenient for evaluating the nonlinear function (; r). According to this theorem, we have, subject to the existence of the involved k (; k = EfI (k) ()I (k) (Y)g (8)

14 4 H. Krim and J.-C. Pesquet where and Y are jointly Gaussian zero-mean normalized random variables with correlation coecient r and I (k) () denotes the k th order derivative of I(). When this theorem is applicable 0 and an innite Taylor expansion of (; r) exists at r = 0, we can write (; r) = k= EfI (k) ()g r k : (83) k! Let us rst consider the case of the L p criterion. We readily have (; r) = 4p (; r) (84) and which leads to Eq. (49). EfI (k) ()g = (p)! (p? k)! Efp?k g = (p)! p?k (p? k)! (85) We will now investigate the denoising criterion. The derivative I (k) () is, for k 3, the distribution I (k) () = ( (k?3) ()? (k?3) ) + (?) ((k?) () + (k?) ) (86) (?) where (k) () is the kth order derivative of the Dirac distribution localized at. This in turn leads to EfI (k) ()g = (?) k?3 (f (k?3) ()? f (k?3) (?)) + (?) k? (f (k?) () + f (k?) (?)) : (87) Since f() is an even function, we nd that EfI (k) ()g = 4(f (k?) ()? f (k?3) ()) ; k : (88) Since the Hermite polynomial Q k () of degree k satises Eq. (88) may be rewritten as EfI (k) ()g = Eq. (50) results from the above expression. dk Q k () = (?) k e d k (e? ) (89) 4 (k?3)= ( p Q k? ( p ) + Q k?3 ( p )) f() ; k : (90) Acknowledgements. The authors would like to thank Dr. I. Schick from MIT, and Prof. B. Picinbono and Dr. P. Bondon from LSS for stimulating and helpful discussions. 0 Note that this theorem cannot be used for the entropy criterion since EfI (k) ()g does not exist, as soon as k 3.

15 References 5 References [BB + ] Basseville, M., Benveniste, A., Chou, K. C., Golden, S. A., Nikoukhah, R., Willsky, A. S.: Modeling and estimation of multiresolution stochastic processes. IEEE Trans. Informat. Theory. 38 (99) 766{784 [CH] Cambanis, S., Houdre., C.: On the continuous wavelet transform of second order random processes. preprint (993) [CW] Coifman, R. R., Wickerhauser, M. V.: Entropy-based algorithms for best basis selection. IEEE Trans. Informat. Theory. 38 (99) 73{78 [Da] Daubechies I.: Ten Lectures on Wavelets. CBMS-NSF, SIAM, Philadelphia, USA (99) [DJ] Donoho, D. L., Johnstone, I. M.: Ideal spatial adaptation by wavelet shrinkage. Biometrika (to appear) [DJ] Donoho, D. L., Johnstone, I. M.: Ideal denoising in an orthogonal basis chosen from a library of bases. Note CRAS Paris (to appear) [Fl] Flandrin, P.: On the spectrum of fractional Brownian motion. IEEE Trans. Informat. [Ib] Theory. 35 (989) 97{99 Ibragimov, I. A.: Some limit theorems for stationary processes. Th. Prob. Appl.VII (96) 349{38 [KPW] Krim, H., Pesquet, J.-C., Willsky, A. S.: Robust multiscale representation of processes and optimal signal reconstruction. Proceedings of IEEE Symposium on Time-Frequency and Time-Scale Analysis, Philadelphia, USA. (994) {4 [LP + ] Lumeau, B., Pesquet, J.-C., Bercher, J.-F., Louveau, L.: Optimisation of bias-variance tradeo in non parametric spectral analysis by decomposition into wavelet packets. Progress in wavelet analysis and applications, Editions Frontieres. (993) 85{90 [Mo] Moulin, P.: A wavelet regularization method for diuse radar target imaging and speckle noise reduction. Journ. Math. Imaging and Vision. 3 (993) 3{34 [PC] Pesquet, J.-C., Combettes, P. L.: Wavelet synthesis by alternating projections. submitted to IEEE Trans. on S.P. [Pr] Price, R.: A useful theorem for nonlinear devices having gaussian inputs. IRE Trans. Informat. Theory. 4 (958) 69{7 [Ri] Rissanen, J.: Modeling by shortest data description. Automatica. 4 (978) 465{47 [Sa] [Un] [Wi] [Wo] Saito, N.: Local feature extraction and its applications using a library of bases. PhD thesis, Yale University (994) Unser, M.: On the optimality of ideal lters for pyramid and wavelet signal approximation. IEEE Trans. Signal Processing. 4 (993) 359{3596 Wickerhauser, M. V.: INRIA lectures on wavelet packet algorithms. Ondelettes et paquets d'ondelettes, Roquencourt, France. (99) 3{99 Wornell., G. W.: A Karhunen-Loeve-like expansion for =f processes via wavelets. IEEE Trans. Informat. Theory. 36 (990) 859{86