Moments and Wavelets in Signal Estimation

Transcription

1 Moments and Wavelets in Signal Estimation Edward J. Wegman 1 Center for Computational Statistis George Mason University Hung T. Le 2 International usiness Mahines Abstrat: The problem of generalized nonparametri funtion estimation has reeived onsiderable attention over the last two deades. Most of the approahes have assumed smoothness of the funtion to be estimated generally in the form of ontinuity of higher order derivatives and/or bounded variation and have used onvolution kernels or splines as the estimation devies. Generally fous has been on density estimation or nonparametri regression. The spline and kernel-based methods may be inappropriate if either smoothness assumptions are violated or if additional side onditions are present. Wegman (1984) introdued a general framework for optimal nonparametri funtion estimation whih applies to a muh wider lass of problems than simply density estimation or nonparametri regression. In this framework, a lass of admissible estimators is regarded as a ompat, onvex subset of a anah funtion spae and a onvex objetive funtional is to be optimized over this set. Reent work on wavelets suggests a powerful method for onstruting orthonormal bases to span the set of admissible estimators. Moreover, older work on frames has re-emerged to some level of prominene beause of the work on wavelets. The optimal estimates an be omputed as weighted linear ombinations of the orthonormal bases. The weight oeffiients are omputed as moments of the basis funtions. We illustrate these methods with some numerial examples. 1 This researh was supported by the Offie of Naval Researh under Grant N J-1303, the Army Researh Offie under Contrat DAAL03-91-G-0039, and the National Siene Foundation under Grant DMS This paper was presented as an invited talk at the ONR-sponsored workshop on Moments and Signal Proessing held in Monterey, CA on Marh 30 and 31, Wegman used to be an important Navy employee as Division Diretor of the Mathematial Sienes Division at ONR, but now he is only a quasiimportant as Theory and Methods Editor of JASA. After Deember 31, 1993, he will resume being unimportant. 2 Dr. Le is an Adjunt Assistant Professor of Operations Researh and Applied Statistis at George Mason University. This work is performed in part while he was on eduational leave of absene from IM.

2 Moments and Wavelets in Signal Estimation 1. Introdution. The method of moments is a time-honored traditional tehnique in statistial inferene while wavelet analysis has reently burst upon the mathematial sene to apture the enthusiasm and imagination of many applied mathematiians and engineers both beause of their important appliations in signal and image proessing and other engineering appliations and also beause of the inherent elegane of the tehniques. In this paper we bring these tools together to illustrate their appliation to transient signal proessing. Wavelets are desribed in detail in a number of loations. Muh of the fundamental work was done by Daubehies and is reported in Daubehies, Grossmann and Meyer (1986) and Daubehies (1988). Heil and Walnut (1989) provide a survey from a mathematial perspetive while Rioul and Vetterli (1991) provide a survey from a more engineering perspetive. The new book by Chui (1992) is an exellent integrated treatment whih I believe is more mathematially sophistiated than the author supposes. In spite of its title as an introdution, it requires somewhat more mathematial depth and maturity and is best regarded as more of a monograph. This present paper desribes the basi wavelet theory in the ontext of the general statistial problem of nonparametri funtion estimation. It will be show that traditional moment based tehniques have an interesting and useful onnetion to modern nonparametri funtional inferene for signal proessing via wavelets. Wegman (1984) desribes a basi framework for optimal nonparametri funtion estimation. This framework aptures the optimal estimation of a wide variety of pratial funtion estimation problems in a ommon theoretial onstrut. Wegman (1984), however, only disusses the existene of suh optimal estimators. In the present paper, we are interested in ombining this optimality framework with more general wavelet algorithms as omputational devies for general optimal nonparametri funtion estimation. A new appliation of optimal nonparametri funtion estimation is found in Le and Wegman (1991). A seond appliation will be disussed in this paper. In setion 2, we disuss the optimal nonparametri funtion estimation framework. In setion 3, we turn to a disussion of the general funtion analyti framework whih leads to bases and frames. Setion 4 introdues the notion of a wavelet basis and demonstrates the onnetion with Fourier series and Parseval's Theorem. In setion 5 we turn to transient signal estimation, develop an optimization riterion and illustrate the omputation of a transient signal estimator. 2. Optimal Nonparametri Funtion Estimation. Consider a general funtion, f(x), to be estimated based on some sampled data, say x, x, Ã,x. This is, in fat, the most elementary estimation problem in statistial inferene. Often the funtion, f, in question is the probability distribution funtion or the probability density funtion and most frequently the approah taken is to plae the funtion within a

3 parametri family indexed by some parameter, say. Rather than estimate f diretly, the parameter is estimated with f then being estimated by V f f V. Under a variety of irumstanes, it is muh more desirable to take a nonparametri approah so as to avoid problems assoiated with misspeifiation of parametri family. This is partiularly the ase when data is relatively plentiful and the information aptured by the parametri model is not needed for statistial effiieny. Probability density estimation and nonparametri, nonlinear regression are probably the two most widely studied nonparametri funtion estimation problems. However, other problems of interest whih immediately ome to mind are spetral density estimation, transfer funtion estimation, impulse response funtion estimation, all in the time series setting, and failure rate funtion estimation and survival funtion estimation in the reliability/biometry setting. While it may be the ase that we simply may want an unonstrained estimate of the funtion, it is more often the ase that we wish to impose one or more onstraints, for example, positivity, smoothness, isotoniity, onvexity, transiene and fixed disontinuities to name a few appropriate onstraints. y far, the most ommon assumption is smoothness and frequently the estimation is via a kernel or onvolution smoother. We would like to formulate an optimal nonparametri framework. We formulate the optimization problem as follows. Let > be a Hilbert spae of funtions over l, the real numbers (or ], the omplex numbers). For purposes of the present paper, we assume l rather than ] unless otherwise speified. The tehniques we outline here are not limited to a disussion of L ( l ) although quite often we do take > to be L. In this ase, we take f, g f(x) g(x) d (x), where is Lebesgue measure. We emphasize that this is not absolutely required. As usual f l f, f. A funtional > : Slis linear if is onvex on S > if ( f b g) (f) b (g), for every f, g > and, l. (tf b(1 t)g) t (f) b(1 t) (g), for every f, g S with 0 t 1. is onave if the inequality is reversed. is stritly onvex (onave) on S if the inequality is strit. is uniformly onvex on S if t (f) b(1 t) (g) (tf b(1 t)g) t(1 t) f g for every f, g S and 0 t 1. We wish to use as the general objetive funtional in our optimization framework. For example, if we are onerned with likelihood, we may onsider the log likelihood, (f) log f(x ), x are a random sample from f.

4 If we have ensored samples we may wish to onsider (g) log g(x ) b (1 ) log G (x ), x again a random sample, a ensoring random variable, G 1 G, and G(x) g(u) du. This is the ensored log likelihood. Another example is the penalized least squares. In this ase (g) (y g(x )) b (Lg(u)) du. Here L is a differential operator and the solution of this optimization problem over appropriate spaes is alled a penalized smoothing L-spline. If L D, then the solution is the familiar ubi spline. The basi idea is to onstrut S > where S is the olletion of funtions, g, whih satisfy our desired onstraints suh as smoothness or isotoniity. We wish to optimize (g) over S. The optimized estimator will be an element of S and hene will inherit whatever properties we hoose for S. The estimator will optimize (g) and hene will be hosen aording to whatever optimization riterion appeals to the investigator. In this sense we an onstrut designer estimators, i.e. estimators that are designed by the investigator to suit the speifis of the problem at hand. Of ourse, in a wide variety of rather disparate ontexts, many of these estimators are already known. However, they may be proven to exist in a general framework aording to the following theorem. Theorem 2.1: Consider the following optimization problem: Minimize (maximize) (f) subjet to f S >. Then a. If > is finite dimensional, is ontinuous and onvex (onave) and S is losed and bounded, then there exists at least one solution. b. If > is infinite dimensional, is ontinuous and onvex (onave) and S is losed, bounded and onvex, then there exists at least one solution.. If in a. or b. is stritly onvex (onave), the solution is unique. d. If > is infinite dimensional, is ontinuous and uniformly onvex (onave) and S is losed and onvex, then there exists a unique solution. Proof: A full proof is given in Wegman (1984). For ompleteness, we outline the basi elements here. a. For the finite dimensional ase, S losed and bounded implies that S is %

5 ompat. Choose f S suh that (f ) onverges to inf{ (f): f S}. eause of ompatness, there is a onvergent subsequene f having a limit, say f i. y ontinuity of (f i) lim (f ) inf{ (f): f S}. S f i is the required optimizer. For part b., we have the same basi idea exept that S losed, bounded and onvex implies that S is weakly ompat. We use the weak ontinuity of. Uniqueness follows by supposing both f i and f ii are both minimizers. Then (tf i b(1 t)f ii) t (f i) b(1 t) (f ii) inf{ (f): f S}. This implies that neither f i nor f ii is a minimizer whih is a ontradition. This theorem gives us unified framework for the onstrution of optimal nonparametri funtion estimators. It does not, however, give us a definitive method for onstrution of nonparametri funtion estimators. We give a onstrutive framework in the next several setions. In losing this setion we refer the reader to Wegman (1984) for the omplete proof of Theorem 2.1 and many more examples of the use of this result. 3. ases and Subspaes. In this setion, we disuss the basi theory of spanning bases and their appliation to funtion estimation. Consider f, g >. f is said to be orthogonal to g written f g if f, g 0. An element f is normal if f 1. A family of elements, say {e : $ } is orthonormal if eah element is normal and if for any pair e, e in the family, e e. A family {e : $ } is omplete in S > if the only element in S whih is orthogonal to every e, $ is 0. A basis or base of S is a omplete orthonormal family in S. A Hilbert spae has a ountable basis if and only if it is separable, i.e. if and only if it has a ountable dense subset. Ordinary L spaes are separable. We are now in a position to state the basi result haraterizing bases of Hilbert spaes or subspaes. We write span({e }) to be the minimal subspae ontaining {e }. This is the spae generated by the elements {e }. Theorem 3.1: Let > be a separable Hilbert spae. If {e } is an orthonormal family in >, then the following are equivalent. a. {e } is a basis for >. b. If f and f e for every k, then f 0. >. If f >, then f f, e e. (orthogonal series expansion) d. If f, g >, then f, g f, e g, e. e. If f >, f f, e. (Parseval's Theorem) Proof: a => b: Trivial by definition. b => : We laim > span({e }). If not there is f 0, f > suh that f span({e }). This implies that f e for every k. ut f e for every k and f 0 is a ontradition to the

6 {e } being a basis. Let > span(e ). Then > span( r > ) >. This implies that for f >, (3.1) f e. Substituting (3.1) in the expression for the inner produt yields f, e e, e e, e. y the orthonormal property, <e, e 1, if k j and 0, otherwise. It follows that f, e. Thus (3.2) f f, e e. => d: f, g f, g, e e g, e f, e. d => e: Let f g in part d. e => a: If f > and f e for every k implies f, e 0 for every k. This in turn implies that f 0. Thus f 0. This finally implies {e } is a basis. Thus given any basis {e }, we an exatly write f e and we an estimate f 5 V by e. Thus a omputational algorithm for the optimal nonparametri funtion estimator an be based on this result from Theorem However, this does not yet take into aount the design" set, S. In order to more arefully study the struture of S we onsider the following result. In the following disussion let S >. Then define S {f > : f S}. Theorem 3.2: If S > is a subset of >, then a. S is a subspae of > and S q S {0} b. S S span(s). S is a subspae if and only if S S. Proof: S is a linear manifold. To see this if f, f S, then for every g S, af b af, g a f, g ba f, g a h0 ba h 0 0. Thus af baf S. This implies S is a linear manifold whih is suffiient to show that S is a subspae provided we an show S is losed. To see this if f losure (S ), then there exists {f } S suh that f lim f and for every g S, f, g 0. ut f, g

7 lim f, g lim 0 0. This implies f S whih in turn implies f S. Part b S S follows from part a by replaing S by S. Part is straightforward appliation of the two previous parts. Suppose now that we have a basis for >, all it {e }. This basis obviously also spans subset S of > and hene any of our designer" funtions in S an be written in terms of the basis, {e }. The unneessary basis elements will simply have oeffiients of 0. In a sense, however, this basis is too rih and in a noisy estimation setting superfluous basis elements will only ontribute to estimating noise. As part of our designer" set, S, philosophy, we would like to have a minimal basis set for S. Theorem 3.2 gives us a test for this ondition. Consider a basis {e } for >. Form : whih is to be a basis for S. We define : by the following routine. If there is a g S suh that g, e 0, then let e :. If on the other hand there is a g S suh that g, e 0, then let e :. Unfortunately, it may not be that : q : J. ut this algorithm yields {e } : r :. Moreover S span( :). Thus we may be able to eliminate unneessary basis elements. We may also be able to re-normalize the basis elements using a Gram-Shmidt orthogonalization proedure to make : :. Usually if we know the properties of the set, S, we desire and the nature of the basis set {e }, it will be straightforward to onstrut a test funtion, g, with whih to onstrut the basis set, :. If S is a subspae, then S span( :). In any ase we an arry out our estimation by (3.3) V f V e. ) In a ompletely noiseless setting (3.1) is really an equality in norm, i.e. f e 0. If > is L ( ), with Lebesgue measure, then (3.1) is really (3.4) f e, almost everywhere with f, e. This hoie of is a minimum norm hoie. However, in a noisy setting, i.e. where we do not know f exatly, we annot ompute diretly. However, we may be able to estimate by standard inferene tehniques. Example 3.1. Norm Estimate. The minimum norm estimate of is the hoie whih minimizes f e, i.e. f, e. In the L ontext, f, e f(x) e (x) d (x). l If f is a probability density funtion, then f, e E[e ] whih an simply be estimated by n e (x ), where x, 1, Ã, is the sample of observations. We note that the major approah to estimating the weighting oeffiients is via a traditional method of moments.

8 Example 3.2. General Form of Estimate. In the general ontext with optimization funtional we have (3.5) (f) : e ; ({ }). ) Sine (3.5) is a funtion of a ountable number of variables, { }, we an find the normal equations and with the appropriate hoie of basis, find a solution. For this we will typially assume is twie differentiable with respet to all. A wide variety of bases have been studied. These inlude Laguerre polynomials, Hermite polynomials and other orthonormal systems. Perhaps the most well-known orthonormal system is the system of fundamental sinusoids whih span L (0, 2 ). One might reasonable guess that wavelets form another orthogonal system. We disuss the onnetion in the next setion. 4. Fourier Analysis and Wavelets. 4.1 ases for L (0, 2 ). Let us onsider the set of square-integrable funtions on (0, 2 ) whih we denote by L (0, 2 ). L (0, 2 ) is a Hilbert spae and a traditional hoie of an orthonormal basis for this % spae has been e (x) e, the omplex sinusoids. Thus any f in L (0,2 ) has the Fourier representation by Theorem 3.1. f(x) e where the onstants are the Fourier oeffiients defined by % % 1 f(x)e dx. 2 This pair of equations represent the disrete Fourier transform and the inverse Fourier transform and is the foundation of harmoni analysis. An interesting feature of this omplex % sinusoids as a base for L (0, 2 ) is that e (x) e an be generated from the superpositions of dilations of a single funtion, e(x) e %. y this we mean that e (x) e( kx), k Ä, 1, 0, 1, Ä These are integral dilations in the sense that k J, the integers. The onept of dilations of a fixed generating funtion is entral to the formation of wavelet bases as we shall see shortly. A well known onsequene of Theorem 3.1.e for the omplex sinusoid basis is the Parseval Theorem. For this base, we have Theorem 4.1: (Parseval's Theorem):

9 . (4.1) f f(x) dx Equation (4.1) is known as Parseval's Theorem in harmoni analysis and states that the square norm in the frequeny domain is equal to the square norm in the time domain. While the spae L (0, 2 ) is an extremely useful one, for general problems in nonparametri funtion estimation we are muh more interested in L ( l). We an think of L (0, 2 ) as with funtions on the finite support (0, 2 ) or as periodi funtions on l. In the latter ase it is lear that the infinitely periodi funtions of L (0, 2 ) and the square integrable funtions of L ( l) are very different. In the latter ase the funtion, f(x) L ( l), must onverge to 0 as x Sf. The generating funtion e(x) e % learly does not have that behavior and is inappropriate as a basis generating funtion for L ( l). What is needed is a generating funtion, e(x), whih also has the property that e(x) S0 as x Sf. Thus we want to generate a basis from a funtion whih will deay to 0 relatively rapidly, i.e. we want little waves or wavelets. 4.2 Wavelet ases. Let us begin by onsidering a generating funtion whih we will think of as our mother wavelet or basi wavelet. The idea is that, just as with the sinusoids, we wish to onsider a superposition of dilations of the basi waveform. For tehnial onvergene reasons whih we shall explain later we wish to onsider dyadi dilations rather than simply integral translations. Thus for the first pass, we are inlined to onsider (x) 2 (2 x). Unfortunately, beause of the deay of to 0 as x Sf, the elements { } are not suffiient to be a basis for L ( l). We aommodate this by adding translates to get the doubly indexed funtions (x) 2 (2 x k). We hoose suh that Á V( ) l d exists. Here V is the Fourier transform of. Under ertain hoies of, Á forms a doubly indexed orthonormal basis for L (atually also for Sobolev spaes of higher order as well). As we shall see in the next setion, a wavelet basis due to the dilation-translation nature of its basis elements admits an interpretation of a simultaneous time-frequeny deomposition of f. Moreover using wavelets, fewer basis elements are required for fitting sharp hanges or disontinuities. This implies faster onvergene in non-smooth" situations by the introdution of loalized" basis elements. Example 3.1 Continued: Notie that Á f, Á 2 62 x k 7f(x) dx. In the density estimation ase Á E : 2 62 xk 7;.

10 Thus a natural estimator is 2 V 6 2 x k 7, Á where x, 1, Ã, is the set of observations. Again we are simply using a method of moments estimator. Notie that we an onstrut a Parseval's Theorem for Wavelets. Theorem 4.2: (Parseval's Theorem for Wavelets) Á Á. (4.2) f f(x) dx At this stage we are left with the problem of onstruting an appropriate mother wavelet,, suitable for onstruting the basis. To do this we turn to the devie of multiresolution analysis. 4.3 Multiresolution Analysis. To understand multiresolution analysis let us first onsider the onstrution of spae W span{ Á: k J}. That is we fix the dilation and onsider the spae generated by all possible translates. We may write L ( l) as a diret sum of the W, L ( l ) W so that any funtion f L ( l) may be written as f(x) Äbd (x) bd (x) bd (x) bä where d W. If is an orthogonal wavelet, then W W, k j. We shall assume the unknown to be an orthogonal wavelet in what follows. Notie that as j inreases, the basi wavelet form (2 x k) ontrats representing higher frequenies." For eah j we may onsider the diret sum V given by: V ÄbW b W W. The V are losed subspaes and represent spaes of funtions with all frequenies" at or below a given level of resolution. The set of spaes FV G has the following properties: 1) They are nested in the sense that V V b, j J. 2) Closure 6 r V ) L ( l). 3) q V {0}. 4) Vb V bw. 5) f(x) V if and only if f(2x) V, J. b j 1), 4) and 5) follow diretly from the definition of V. 2) is a straightforward onsequene of the fat that r W L ( ). 3) follows beause of the orthogonality property. l

11 Any f L ( l) an be projeted into V. As we have seen with j inreasing the the frequeny" of the wavelet inreases whih an be interpreted as higher resolution. Thus the projetion, Pf, of f into V is an inreasingly higher resolution approximation to f as js. Conversely, as js, P f is an inreasingly blurred (smoothed) approximation to f. We shall take V as the referene subspae. Suppose now that we an find a funtion and that we an define (x) 2 (2 x k) suh that Á V span{ : k J}. Then by property 5), V span{ Á: k J}. While we began our disussion with the notion of wavelets and have seen some of the onsequenes, we ould have atually begun a disussion with the funtion. Definition. A funtion generates a multiresolution analysis if it generates a nested sequene of spaes having properties 1), 2), 3) and 5) suh that { Á, k J} forms a basis for V. If so, then is alled the saling funtion. For the final disussion of this setion, let us onsider a multiresolution analysis in whih {V } are generated by a saling funtion L ( l) and {W } are generated by a mother wavelet funtion L ( l). Any funtion f L ( l) an be approximated as losely as desired by f for some suffiiently large m J. Notie f f bd where f V and d W. This proess an be reursively applied say l times until we have f f d bd bäbd bf. Notie that f is a highly smoothed version of the funtion. Indeed, this suggests that a statistial proedure might be to form a highly smoothed (even overly smoothed) approximation to a funtion to be estimated. The sequene d through d form the higher resolution wavelet approximations. Many of the wavelet oeffiients Á used for onstruting d, i 1, Ã, l are likely to be 0 and hene an ontribute to a very parsimonious representation of the funtion f. Indeed, a wavelet deomposition is a natural suggestion for a tehnology for high definition television (HDTV). If f represents the lower resolution onventional NTSC TV signal, then to reonstrut a high resolution image all that is needed is the differene signal whih ould be parsimoniously represented by the wavelet oeffiients Á, i 1, Ã,l and k J, most of whih would be 0. Most importantly, however, is the observation that the saling funtion V and the mother wavelet W implies that both are in V. Sine V is generated by (x) 2 (2x k), there are sequenes {g( k)} and {h( k)} suh that Á (4.3) (x) g( k) (2x k) and (x) h( k) (2x k). This remarkable result gives us a onstrution for the mother wavelet in terms of the saling funtion. These equations are alled the two-sale differene equations. We an give a time series interpretation to these equations. Lets onsider an original disrete time funtion, f( n), to whih we apply the filter Á

12 y( n) g( k)f(2 n k). First of all we note that there is a sale hange due to subsampling by two, i.e. a shift by two in f( n) results in a shift of one in y( n). The sale of y is only half that of f. Otherwise this is a low pass filter with impulse response funtion g. Let us onsider iterating this equation so that (4.4) y ( n) g( k)y (2 n k). ²³ Notie that if this proedure onverges, it onverges to a fixed point whih will be. This iterative proedure with repeated down sampling by two is suggestive of a method for onstruting wavelets. If g is a finite impulse response (FIR) filter of length l, the onstrution of a omplementary high-pass filter is aomplished with a FIR filter, h, whose impulse response is given by h( l1 n) ( 1) g( n). This sheme is alled sub-band oding in the eletrial engineering literature. The low-pass band is given by ²³ (4.5) y ( n) g( k)f(2 n k) while the high-pass band is given by (4.6) y ( n) h( k)f(2 n k). The filter impulses as defined form an orthonormal set so that the f may be reonstruted by (4.7) f( n) [y ( k)g(2 kn) by ( k)h(2 kn)]. The sub-band oding sheme may be repeatedly applied to form the nested sequene pf V. The nested sequene of {V } is then essentially obtained by reursively downsampling and filtering a funtion with a low-pass filter whose impulse response funtion is g( h ). 4.4 Constrution of Saling Funtions and Mother Wavelets. We have already hinted that the saling funtion may be onstruted as the fixed point of the down-sampled, low-passed filter equation (4.4). This an be formalized by onsidering what statistiians would all the generating funtion of g( n) and what eletrial engineers all the z-transform of g( h ). (4.8) G(z) 1 2 g( j) z. Notie if z e, then (4.8) is essentially the Fourier transform of the impulse response funtion g( h ). In this ase, the first equation in (4.3) may be written as

13 (4.9) V( ) G(z) V 6 7, with z e. 2 This, of ourse, follows beause the Fourier transform of a onvolution is the orresponding produt of the Fourier transforms. This reursive equation may be iterated to obtain (4.10) V( ) G6e 7 V(0). We may take V to be ontinuous and V (0) 1. ased on (4.10) we may reover ( h) and based on this result, the equation h( l1 n) ( 1) g( n) and the seond equation of (4.3) we may reover the mother wavelet, ( h ). Thus Daubehies' original onstrution shows that wavelets with ompat support an be based on finite impulse response filters whih was originally motivated by multiresolution analysis. Theorem 4.3 below summarizes the general form of Daubehies' result. Theorem 4.3: (Daubehies' Wavelet Constrution): Let g( n) be a sequene suh that a. g( n) n for some 0, b. g( n2 j) g( n 2 k),. g( n ) 1. Suppose that V g( ) G(e ) 2 g( n) e an be written as where d. f( n) n for some 0 5 e. sup l f(n) e 2. Define h( n) ( 1) g( nb1), V ( ) G6e 7, (x) h( k) (2x k). 5 V g( ) [ (1 b ) ] h[ f( n) e ] Then the orthonormal wavelet basis is determined by the mother wavelet. Moreover, if g( n) 0 for n n, then the wavelets so determined have ompat support. We state this result without proof whih may be found n Daubehies (1988). We note that Daubehies also shows that the mother wavelet,, annot be an even funtion and also

14 have a ompat support. The exeption to this is the trivial onstant funtion whih gives rise to the so-alled Haar basis. Daubehies illustrates this omputation with the example of g given by g(0) (1 b l3)/8, g(1) (3 b l3)/8, g(2) (3 l3)/8 and, finally, g(3) (1 l 3)/8. This wavelet is illustrated in Figure 4.1. Figure 4.1a. Daubehies' Saling Funtion using 4-term FIR filter. Figure 4.1b. Daubehies' Mother Wavelet using 4-term FIR filter.

15 5. Transient Signal Funtion Estimation. Now with the basi onstrution of wavelets in hand, we an turn to the transient signal proessing appliation. Wavelets have as one of their prime appliations transient signal proessing. In partiular, sine the most effetive wavelets are those with ompat support, they are a natural basis for transient signal estimation. However, if we are to exploit them in the ontext of optimal nonparametri funtion estimation, we must onstrut an optimality riterion for transient signals. The disussion below outlines an approah to transient signal estimation set in the ontext of optimal nonparametri funtion estimation. A fuller treatment an be found in Le and Wegman (1992). We first onsider signals. It is well-known that there is no non-zero funtion in L ( l) whih is both band-limited and time-limited. This being the ase, we will assume the signal to be hard band-limited, i.e. with no energy outside a fixed interval, say [, ], but soft time-limited, i.e. with minimal energy in the tails. This partiular example demonstrates an elegant appliation of moments to signal proessing. 5.1 Measuring of Out-of-and Energy Let L ( l) be the set of square-integrable, real-valued funtions and let h(t) L ( l). Denote by Vf( ) the Fourier transform of f(t) suh that Vf L ( ). We assume V l f is frequeny band-limited so that V f( ) 0, for. We propose approximating the lass of bandlimited time-transient funtions by onsidering funtions whose energy time spread is onfined to some small level s. As a measure of the energy time-spread, we will use analogies to onepts from probability theory to define various moments of f(t), whih plays the role of the energy distribution funtion. Assuming that t f(t) dt <, j 1, 2, Ã, k,! the k moment of the energy distribution will now be defined as follows M t f(t) dt. For k 2, we have the 2nd moment of the energy distribution funtion as a measure of the energy time spread, given as M t f(t) dt. Remark: The fator t serves as a weight on the energy funtion whih is used to ontrol the degree of spreading in f(t). A larger k value implies that more weight is applied at the tailk end of the energy distribution funtion and, therefore, the proess of minimizing M requires that more energy be entrally onentrated. 5.2 Optimal Estimation of and-limited Proesses

16 For and real numbers, and m and p integers, where, and, m 0 and p 1, the Sobolev spae M [, ] of omplex-valued funtions Vf on [, ] is given by: ²³, M [, ] FVf( ): Vf ( ), k 0, 1, Ã, m 1, are absolutely ontinuous and, V ²³ f ( ) d < G. We onsider observing an atual proess, r(t), and we let V r( ) be the Fourier transform of the observed proess, r(t). The Fourier transform of the observed proess, r(t), will then be modeled as Vr( ) Vg( ) b ( ) where, ( ) is the spetrum of a stationary noise proess, Vg( ), [, ]. The fat that V Á M f belongs to the lass M [, ] of band-limited signals implies that the support of f(t) is not bounded. The objetive is, then, to find a funtion Vf( ) M Á [, ] whih best fits the Fourier transform Vr( ) of the observed proess r(t) with minimum time-energy spread; speifially we would like to minimize the following funtional with k m (5.1) min [ (f( V ) Vr( )) ] subjet to t f(t) dt s, V Á f M, µ where f(t) is the inverse Fourier transform orresponding to V Á f( ) in M [, ]. 5.3 Moment Connetion via Parseval's Theorem A rather elegant extension of Parseval's Theorem an be onstruted under appropriate regularity onditions. The Parseval's Theorem for ontinuous Fourier transform pairs is 1 Vf( ) d 2 f(t) dt. ut we know V f( ) 1! f(t) e dt. 2! Take k derivatives with respet to so that ²³ C Vf( ) 1 C 2! ( t) f(t) e dt V f( ) f ( ) is the Fourier transform of ( t) f(t). C V C We an apply Parseval's Theorem to this Fourier transform pair to obtain Theorem 5.1: ²³ 1 Vf ( ) d t f(t) dt. 2 Thus, our optimization problem (5.1) an now be reformulated as

17 (5.2) min [ (f( V ) r( )) ] subjet to V V * f ( ) d s. V Á f M, µ ²³ Using standard Lagrange multiplier tehniques, this in turn may be reformulated as (5.3) min [ (f( V ) r( )) V V b f ( ) d ]. V Á f M, µ Indeed expression (5.3) is the form of optimization problem whih results in a solution whih is a generalized polynomial spline of degree 2k 1. This result may be substantially generalized by the theorem given below whih is developed in Le and Wegman (1992). Theorem 5.2: Let V g( ) be a band-limited spetral proess with transient inverse Fourier transform and V r( ) be the observed spetral proess defined over some finite band. We model this spetral proess as Vr( ) Vg( ) b ( ) where ( ) is some stationary white noise proess. Let $ be the time spread measure, defined as follows: $ (f) V a $ (f) V ba $ (f) V where, b 1 $ 2 (f) V t f(t) dt, ²³ and where a and a are the appropriately hosen weights. Here f is the inverse Fourier transform of Vf belonging to L ( l). Then, the optimal band-limited representation in the Sobolev spae M Á [, ] is Vf ( ) where Vf ( ) is the solution to the problem: minimize [f( V ) r( )] + (f) V V $. V Á f M, µ Vf is a generalized L-spline, and is known as the smoothing parameter. For a general disussion of L-splines, see Wegman and Wright (1983). Notie that if $ (f) V $ (f) V for some large, then we are onstruting a band-limited transient signal estimator with little energy in the tail of the signal estimate, f, where f is the inverse Fourier transform of Vf. If 2, then $ b ²³ (f) V 1 t f(t) dt Vf ( ) d 2

18 and our solution is the well-known ubi spline. However, muh more interesting and physially meaningful solutions may be found. If $ (f) V a $ (f) V ba $ (f), V then for odd b $(f) V 1 a f(t) dt b a t f(t) dt. 2 J K Thus, we may also want to impose a total energy restrition on the estimated signal spae. This imposed restrition may, for example, have resulted from a requirement to minimize hannel bandwidth utilization from data transmission systems. Suh modifiation, thus, yields the following optimization problem for odd b n min [ (f( V ) r( )) + Vf( ) d + V V f ( ) d ]. V Á f M ², ³ i ²³ Hene, by our theorem the optimal solution is again an L-spline. 5.4 Computing and-limited Transient Estimators and Example The rather elegant result that our band-limited transient estimators are generalized L- splines makes the numerial omputation of the estimators rather more routine sine algorithms already exist for omputing L-splines. The fat that we an impose total energy limits as well as tail-energy limits is an unexpeted bonus. Our interpretation of Theorem 5.2 is as follows. We reommend doing an initial spetral estimation to establish the bandwidth,, over whih we want to estimate Vg( ) (or more preisely the signal, g(t), its inverse Fourier transform). This initial spetral estimate will also allow us to selet the sampling frequenies,. We reommend seleting these as the frequenies with the largest spetral mass. Notie that we may regard a transient signal, g(t), as the produt of a signal of infinite support with an indiator funtion of a losed interval. It is well-known that Fourier transform of an indiator funtion is the so-alled Dirihlet kernel whih has a large entral lobe and dereasing side lobes. y hoosing sampling frequenies at the loation of the entral and side lobes, our tehnique allows us to to reover the indiator to an exellent approximation. Thus not only do we estimate the transient signal beause of the penalty term for out-of-band energy, but beause of the hoie of sampling frequenies as well. Figure 5.1 graphially illustrates the results of our tehnique. Referenes Chui, C. K. (1992), An Introdution to Wavelets, Aademi Press: oston Daubehies, I. (1988), Orthonormal bases of ompatly supported wavelets," Comm. on Pure and Appl. Math., 41,

19 Daubehies, I., Grossmann, A. and Meyer, Y. (1986), Painless nonorthogonal expansions," J. Math. Phys., 27, Heil, C. and Walnut, D. (1989), Continuous and disrete wavelet transforms," SIAM Review, 31, Le, H. T. and Wegman, E. J. (1991), Generalized funtion estimation of underwater transient signals," J. Aoust. So. Ameria, 89, Le, H. T. and Wegman, E. J. (1992), A spetral representation for the lass of band-limited funtions," to appear Signal Proessing Rioul, O. and Vetterli, M. (1991), Wavelets and signal proessing," IEEE Sign. Pro. Mag., 8, Wegman, E. J. and Wright, I. W. (1983), Splines in statistis," J. Am. Statist. Asso.,78, Wegman, E. J. (1984), Optimal nonparametri funtion estimation," J. Statist. Planning and Infer., 9, Figure 5.1a. A two-yle transient signal.

20 Figure 5.1b. Same two yle signal buried in Gaussian noise. Figure 5.1. Reovery of two-yle signal waveform by optimal band-limited tehniques.