Wavelet Methods for Time Series Analysis. Wavelet-Based Signal Estimation: I. Part VIII: Wavelet-Based Signal Extraction and Denoising

Transcription

1 Waveet Methods for Time Series Anaysis Part VIII: Waveet-Based Signa Extraction and Denoising overview of key ideas behind waveet-based approach description of four basic modes for signa estimation discussion of why waveets can hep estimate certain signas simpe threshoding & shrinkage schemes for signa estimation waveet-based threshoding and shrinkage case study: denoising ECG time series brief comments on second generation denoising Waveet-Based Signa Estimation: I DWT anaysis of X yieds W = WX DWT synthesis X = W T W yieds mutiresoution anaysis by spitting W T W into pieces associated with different scaes DWT synthesis can aso estimate signa hidden in X if we can modify W to get rid of noise in the waveet domain if W is a noise reduced version of W, can form signa estimate via W T W VIII 1 VIII Waveet-Based Signa Estimation: II key ideas behind simpe waveet-based signa estimation certain signas can be efficienty described by the DWT using a of the scaing coefficients a sma number of arge waveet coefficients noise is manifested in a arge number of sma waveet coefficients can either threshod or shrink waveet coefficients to eiminate noise in the waveet domain key ideas ed to waveet threshoding and shrinkage proposed by Donoho, Johnstone and coworkers in 199s VIII 3 Modes for Signa Estimation: I wi consider two types of signas: 1. D, ann dimensiona deterministic signa. C, ann dimensiona stochastic signa; i.e., a vector of random variabes (RVs) with covariance matrix Σ C wi consider two types of noise: 1. ɛ, ann dimensiona vector of independent and identicay distributed (IID) RVs with mean and covariance matrix Σɛ = σɛi N. η,ann dimensiona vector of non-iid RVs with mean and covariance matrix Ση one form: RVs independent, but have different variances another form of non-iid: RVs are correated VIII 4

2 Modes for Signa Estimation: II eads to four basic signa + noise modes for X 1. X = D + ɛ. X = D + η 3. X = C + ɛ 4. X = C + η in the atter two cases, the stochastic signa C is assumed to be independent of the associated noise Signa Representation via Waveets: I consider deterministic signas D first signa estimation probem is simpified if we can assume that the important part of D is in its arge vaues assumption is not usuay viabe in the origina (i.e., time domain) representation D, but might be true in another domain an orthonorma transform O might be usefu because O = OD is equivaent to D (since D = O T O) we might be abe to find O such that the signa is isoated in M N arge transform coefficients Q: how can we judge whether a particuar O might be usefu for representing D? VIII 5 VIII 6 Signa Representation via Waveets: II et O j be the jth transform coefficient in O = OD et O (),O (1),...,O (N 1) be the O j s reordered by magnitude: O(1) O(N 1) O () exampe: if O =[ 3, 1, 4, 7,, 1] T, then O () = O 3 = 7, O (1) = O = 4, O () = O = 3 etc. define a normaized partia energy sequence (NPES): M 1 j= C M 1 O (j) energy in argest M terms N 1 j= O = (j) tota energy in signa et I M be N N diagona matrix whose jth diagona term is 1if O j is one of the M argest magnitudes and is otherwise Signa Representation via Waveets: III form D M O T I M O, an approximation to D = O T O when O =[ 3, 1, 4, 7,, 1] T and M =3,wehave 1 3 I 3 = 1 1 and thus D M = O T 4 7 can argue that C M 1 =1 D D M D =1 reative approximation error VIII 7 VIII 8

3 Signa Representation via Waveets: IV Signa Representation via Waveets: V 1 D 1 D D t t t consider three signas potted above D 1 is a sinusoid, which can be represented succincty by the discrete Fourier transform (DFT) D is a bump (ony a few nonzero vaues in the time domain) D 3 is a inear combination of D 1 and D three different orthogona transforms identity transform I (time) the orthogona DFT F (frequency), where F has (k, t)th eement exp( iπtk/n)/ N for k, t N 1 the DWT W (waveet) # of terms M needed to achieve reative error < 1%: D 1 D D 3 DFT 9 8 identity LA(8) waveet 14 1 VIII 9 VIII 1 Signa Representation via Waveets: VI Signa Representation via Waveets: VII D 1 D D M M M use NPESs to see how we these three signas are represented in the time, frequency (DFT) and waveet (LA(8)) domains time (soid curves), frequency (dotted) and waveet (dashed) et us consider the vertica ocean shear time series as a signa wi ook at pots of the signa D itsef its approximation D 1 from 1 LA(8) DWT coefficients D. 3 from 3 LA(8) DWT coefficients, giving C 99 =.9983 D. 3 from 3 DFT coefficients, giving C 99 =.9973 note that 3 coefficients is ess than 5% of N = 6784! VIII 11 VIII 1

4 Signa Representation via Waveets: VIII 7 D Signa Representation via Waveets: IX 4 DFT LA(8) DWT X LA(8) D D 5 D meters LA(8) D 3 DFT D 3 need 13 additiona ODFT coefficients to match C 99 for DWT VIII t t D D 4 nd exampe: DFT D M (eft-hand coumn) & J = 6 LA(8) DWT D M (right) for NMR series X (A. Maudsey, UCSF) VIII 14 Signa Estimation via Threshoding: I assume mode of deterministic signa pus IID noise: X = D + ɛ et O be an N N orthonorma matrix form O = OX = OD + Oɛ d + e component-wise, have O = d + e define signa to noise ratio (SNR): D E{ ɛ } = d E{ e } = N 1 = d N 1 = E{e } assume that SNR is arge assume that d has just a few arge coefficients; i.e., arge signa coefficients dominate O VIII 15 Signa Estimation via Threshoding: II reca simpe estimator D M O T I M O and previous exampe: 1 O O O 1 D M = O T 1 O 1 O 3 = O T O O 3 O 4 O 5 et J m be a set of m indices corresponding to paces where jth diagona eement of I m is 1 in exampe above, we have J 3 = {,, 3} strategy in forming D M is to keep a coefficient O j if j J m but to repace it with if j J m ( ki or keep strategy) VIII 16

5 Signa Estimation via Threshoding: III can pose a simpe optimization probem whose soution 1. is a ki or keep strategy (and hence justifies this strategy). dictates that we use coefficients with the argest magnitudes 3. tes us what M shoud be (once we set a certain parameter) optimization probem: find D M such that γ m X D m + mδ is minimized over a possibe I m, m =,...,N in the above δ is a fixed parameter (set a priori) Signa Estimation via Threshoding: IV X D m is a measure of fideity rationae for this term: under our assumption of a high SNR, D m shoudn t stray too far from X fideity increases (the measure decreases) as m increases in minimizing γ m, consideration of this term aone suggests that m shoud be arge mδ is a penaty for too many terms rationae: heuristic says d has ony a few arge coefficients penaty increases as m increases in minimizing γ m, consideration of this term aone suggests that m shoud be sma optimization probem: baance off fideity & parsimony VIII 17 VIII 18 Signa Estimation via Threshoding: V caim: γ m = X D m + mδ is minimized when m is set to the number of coefficients O j such that O j >δ proof of caim: since X = O T O & D m O T I m O, have γ m = X D m + mδ = O T O O T I m O + mδ = (I N I m )O + mδ = O j + δ j J m j J m consider jth coefficient: if j J m, we woud contribute O j to first sum if j J m, we woud contribute δ to second sum to minimize γ m, we need to put j in J m if O j >δ,thus estabishing the caim VIII 19 Threshoding Functions: I more generay, threshoding schemes invove 1. computing O OX. defining O (t) as vector with th eement O (t) = {, if O δ; some nonzero vaue, where nonzero vaues are yet to be defined 3. estimating D via D (t) O T O (t) otherwise, simpest scheme is hard threshoding ( ki/keep strategy): { O (ht), if O = δ; O, otherwise. VIII

6 Threshoding Functions: II pot shows mapping from O to O (ht) O (ht) 3δ δ δ δ δ Threshoding Functions: III aternative scheme is soft threshoding: O (st) = sign {O } ( O δ) +, where +1, if O > ; sign {O }, if O =; and (x) + 1, if O <. { x, if x ;, if x<. one rationae for soft threshoding is that it fits into Stein s cass of estimators (wi discuss this ater) 3δ 3δ δ δ δ δ 3δ O VIII 1 VIII Threshoding Functions: IV here is the mapping from O to O (st) O (st) 3δ δ δ δ Threshoding Functions: V third scheme is mid threshoding: O (mt) = sign {O } ( O δ) ++, where { ( O ( O δ) ++ δ) +, if O < δ; O, otherwise provides compromise between hard and soft threshoding δ 3δ 3δ δ δ δ δ 3δ O VIII 3 VIII 4

7 Threshoding Functions: VI here is the mapping from O to O (mt) 3δ Threshoding Functions: VII exampe of mid threshoding with δ =1 3 δ 3 X t O (mt) δ δ O O (mt) δ 3 D (mt) t 3δ 3δ δ δ δ δ 3δ O 3 64 t or VIII 5 VIII 6 Universa Threshod: I Universa Threshod: II Q: how do we go about setting δ? speciaize to IID Gaussian noise ɛ with covariance σɛi N can argue e Oɛ is aso IID Gaussian with covariance σ ɛi N Donoho & Johnstone (1995) proposed δ (u) [σɛ og(n)] ( og here is og base e ) rationae for δ (u) : because of Gaussianity, can argue that P [ max{ e } >δ (u)] and hence P wi exceed threshod in the imit 1 [4π og (N)] asn [ max e δ (u)] 1asN, so no noise suppose D is a vector of zeros so that O = e impies that O (ht) = with high probabiity as N hence wi estimate correct D with high probabiity critique of δ (u) : consider ots of IID Gaussian series, N = 18: ony 13% wi have any vaues exceeding δ (u) δ (u) is santed toward eiminating vast majority of noise, but, if we use, e.g., hard threshoding, any nonzero signa transform coefficient of a fixed magnitude wi eventuay get set toasn nonetheess: δ (u) works remarkaby we VIII 7 VIII 8

8 Minimum Unbiased Risk: I second approach for setting δ is data-adaptive, but ony works for seected threshoding functions assume mode of deterministic signa pus non-iid noise: X = D + η so that O OX = OD + Oη d + n component-wise, have O = d + n further assume that n is an N (,σn ) RV, where σn is assumed to be known, but we aow the possibiity that n s are correated et O (δ) be estimator of d based on a (yet to be determined) threshod δ put O (δ) s into vector O (δ) Minimum Unbiased Risk: II define D (δ) O T O (δ) and associated risk R( D (δ), D) E{ D (δ) D } = E{ O( D (δ) D) )} = E{ O (δ) d )} { N 1 = E (O (δ) d ) } can minimize risk by making E{(O (δ) d ) } as sma as possibe for each Stein (1981) considered estimators restricted to be of the form O (δ) = O + A (δ) (O ), where A (δ) ( ) must be weaky differentiabe (basicay, piecewise continuous pus a bit more) = VIII 9 VIII 3 Minimum Unbiased Risk: III using O (δ) = O + A (δ) (O ) with O = d + n yieds O (δ) d = n + A (δ) (O ) and hence E{(O (δ) d ) } = σn +E{n A (δ) (O )} + E{[A (δ) (O )] } because of Gaussianity, can reduce midde term (book, p43): { } E{n A (δ) (O )} = σn d E dx A(δ) (x) x=o can now write E{(O (δ) d ) } = E{R(σ n,o,δ)}, where R(σ n,x,δ) σn +σn d dx A(δ) (x)+[a (δ) (x)] Minimum Unbiased Risk: IV risk in using D (δ) given by R( D N 1 (δ), D) =E (O (δ) N 1 d ) = E R(σ n,o,δ) = practica scheme: given reaizations o of O, find δ minimizing N 1 = R(σ n,o,δ) for a given δ, above is Stein s unbiased risk estimator (SURE) = VIII 31 VIII 3

9 exampe: if we set Minimum Unbiased Risk: V A (δ) (O )= { O, δ sign{o }, if O <δ; if O δ, we obtain O (δ) = O + A (δ) (O )=O (st), i.e., soft threshoding for this case, can argue that R(σ n,o,δ)=o σ n +(σn O + δ )1 [δ, ) (O ) ony the ast term depends on δ, and, as a function of δ, SURE is minimized when ast term is minimized Minimum Unbiased Risk: VI data-adaptive scheme is to repace O with its reaization, say o, and to set δ equa to the vaue, say δ (S), minimizing N 1 (σn o + δ )1 [δ, ) (o ), = must have δ (S) = o for some, so minimization is easy if n have a common variance, i.e., σn = σ for a, need to find minimizer of the foowing function of δ: N 1 (σ o + δ )1 [δ, ) (o ), = (in practice, σ is usuay unknown, so ater on we wi consider how to estimate this aso) VIII 33 VIII 34 Signa Estimation via Shrinkage so far, we have ony considered signa estimation via threshoding rues, which wi map some O to zeros wi now consider shrinkage rues, which differ from threshoding ony in that nonzero coefficients are mapped to nonzero vaues rather than exacty zero (but vaues can be very cose to zero!) there are three approaches that ead us to shrinkage rues 1. inear mean square estimation. conditiona mean and median 3. Bayesian approach wi ony consider 1 and, but one form of Bayesian approach turns out to be identica to Linear Mean Square Estimation: I assume mode of stochastic signa pus non-iid noise: X = C + η so that O = OX = OC + Oη R + n component-wise, have O = R + n assume C and η are mutivariate Gaussian with covariance matrices Σ C and Ση impies R and n are aso Gaussian RVs, but now with covariance matrices OΣ C O T and OΣηO T assume that E{R } = for any component of interest and that R & n are uncorreated suppose we estimate R via a simpe scaing of O : R a O, where a is a constant to be determined VIII 35 VIII 36

10 Linear Mean Square Estimation: II et us seect a by making E{(R R ) } as sma as possibe, which can be shown to occur when we set a = E{R O } E{O } because R and n are uncorreated with means and because O = R + n,wehave E{R O } = E{R } and E{O } = E{R } + E{n }, yieding R = E{R } E{R } + E{n }O = σ R σ R + σ n O note: optimum a shrinks O toward zero, with shrinkage increasing as the noise variance increases VIII 37 Background on Conditiona PDFs: I et X and Y be RVs with probabiity density functions (PDFs) f X ( ) and f Y ( ) et f X,Y (x, y) be their joint PDF at the point (x, y) f X ( ) and f Y ( ) are caed margina PDFs and can be obtained from the joint PDF via integration: f X (x) = f X,Y (x, y) dy the conditiona PDF of Y given X = x is defined as f Y X=x (y) = f X,Y (x, y) f X (x) (read as given or conditiona on ) VIII 38 Background on Conditiona PDFs: II by definition RVs X and Y are said to be independent if f X,Y (x, y) =f X (x)f Y (y), in which case f Y X=x (y) = f X,Y (x, y) = f X(x)f Y (y) = f f X (x) f X (x) Y (y) thus X and Y are independent if knowing X doesn t aow us to ater our probabiistic description of Y f Y X=x ( ) is a PDF, so its mean vaue is E{Y X = x} = yf Y X=x (y) dy; the above is caed the conditiona mean of Y, given X Background on Conditiona PDFs: III suppose RVs X and Y are reated, but we can ony observe X suppose we want to approximate the unobservabe Y based on some function of the observabe X exampe: we observe part of a time series containing a signa buried in noise, and we want to approximate the unobservabe signa component based upon a function of what we observed suppose we want our approximation to be the function of X, say U (X), such that the mean square difference between Y and U (X) is as sma as possibe; i.e., we want E{(Y U (X)) } to be as sma as possibe VIII 39 VIII 4

11 Background on Conditiona PDFs: IV soution is to use U (X) =E{Y X}; i.e., the conditiona mean of Y given X is our best guess at Y in the sense of minimizing the mean square error (reated to fact that E{(Y a) } is smaest when a = E{Y }) on the other hand, suppose we want the function U 1 (X) such that the mean absoute error E{ Y U 1 (X) } is as sma as possibe the soution now is to et U 1 (X) be the conditiona median; i.e., we must sove U1 (x) f Y X=x (y) dy =.5 to figure out what U 1 (x) shoud be when X = x Conditiona Mean and Median Approach: I assume mode of stochastic signa pus non-iid noise: X = C + η so that O = OX = OC + Oη R + n component-wise, have O = R + n because C and η are independent, R and n must be aso suppose we approximate R via R U (O ), where U (O )is seected to minimize E{(R U (O )) } soution is to set U (O ) equa to the conditiona mean E{R O }, so et s work out what form the conditiona mean takes to get E{R O }, need the PDF of R given O, which is f R O =o (r )= f R,O (r,o ) f O (o ) VIII 41 VIII 4 Conditiona Mean and Median Approach: II can show that the joint PDF of R and O is reated to the joint PDF f R,n (, ) ofr and n via f R,O (r,o )=f R,n (r,o r )=f R (r )f n (o r ), with the nd equaity foowing since R & n are independent the margina PDF for O can be obtained from the joint PDF f R,O (, ) by integrating out the first argument: f O (o )= f R,O (r,o ) dr = f R (r )f n (o r ) dr putting a these pieces together yieds the conditiona PDF f R O =o (r )= f R,O (r,o ) f R (r )f n (o r ) = f O (o ) f R (r )f n (o r ) dr Conditiona Mean and Median Approach: III mean vaue of f R O =o ( ) yieds estimator R = E{R O }: E{R O = o } = = r f R O =o (r ) dr r f R (r )f n (o r )dr f R (r )f n (o r ) dr to make further progress, we need a mode for the waveetdomain representation R of the signa heuristic that signa in the waveet domain has a few arge vaues and ots of sma vaues suggests a Gaussian mixture mode VIII 43 VIII 44

12 Conditiona Mean and Median Approach: IV et I be an RV such that P [I =1]=p & P [I =]=1 p under Gaussian mixture mode, R has same distribution as I N (,γ σ G )+(1 I )N (,σg ) where N (,σ ) is a Gaussian RV with mean and variance σ nd component modes sma # of arge signa coefficients 1st component modes arge # of sma coefficients (γ 1) exampe: PDFs for case σ G = 1, γ σ G = 1 and p = VIII 45 Conditiona Mean and Median Approach: V to compete mode, et n obey a Gaussian distribution with mean and variance σn conditiona mean estimator of the signa RV R is given by E{R O = o } = a A (o )+b B (o ) o A (o )+B (o ), (book, Ex [1.5]) where γ σ G σ G a γ σ G + σ and b n σg + σ n p A (o ) (π[γ σg + σ n ]) e o /[(γ σ G +σ n )] B (o ) 1 p (π[σ G + σ n ]) e o /[(σ G +σ n )] VIII 46 Conditiona Mean and Median Approach: VI Conditiona Mean and Median Approach: VII et s simpify to a sparse signa mode by setting γ = ; i.e., arge # of sma coefficients are a zero distribution for R same as (1 I )N (,σg ) E{R O = o } 6 3 conditiona mean estimator becomes E{R O = o } = where c = p (σ G + σn ) e o b /(σn ) (1 p )σ n b 1+c o, o conditiona mean shrinkage rue for p =.95 (i.e., 95% of signa coefficients are ); σn = 1; and σg = 5 (curve furthest from dotted diagona), 1 and 5 (curve nearest to diagona) as σg gets arge (i.e., arge signa coefficients increase in size), shrinkage rue starts to resembe mid threshoding rue VIII 47 VIII 48

13 Conditiona Mean and Median Approach: VIII now suppose we estimate R via R = U 1 (O ), where U 1 (O )is seected to minimize E{ R U 1 (O ) } soution is to set U 1 (o )tothemedian of the PDF for R given O = o to find U 1 (o ), need to sove for it in the equation U1 (o ) f R O =o (r ) dr = U1 (o ) f R (r )f n (o r ) dr = 1 f R (r )f n (o r ) dr Conditiona Mean and Median Approach: IX simpifying to the sparse signa mode, Godfrey & Rocca (1981) show that {, if O U 1 (O ) δ; b O, otherwise, where ( )] p σ 1/ G σg δ = σ n [ og and b (1 p )σ = n σg + σ n above approximation vaid if p /(1 p ) σ n /(σ G δ) and σ G σ n note that U 1 ( ) is approximatey a hard threshoding rue VIII 49 VIII 5 Waveet-Based Threshoding assume mode of deterministic signa pus IID Gaussian noise with mean and variance σɛ: X = D + ɛ using a DWT matrix W, form W = WX = WD + Wɛ d + e; because ɛ is IID Gaussian, it foows that e is aso Donoho & Johnstone (1994) advocate the foowing: form partia DWT of eve J : W 1,...,W J and V J threshod W j s but eave V J aone (i.e., administrativey, a N/ J scaing coefficients assumed to be part of d) use universa threshod δ (u) = [σ ɛ og(n)] use threshoding rue to form W (t) j (hard, etc.) estimate D by inverse transforming W (t) 1,...,W(t) J and V J MAD Scae Estimator: I procedure assumes σ ɛ is know, which is not usuay the case if unknown, use median absoute deviation (MAD) scae estimator to estimate σ ɛ using W 1 median { W 1,, W 1,1,..., W 1, N ˆσ (mad) 1 }.6745 heuristic: buk of W 1,t s shoud be due to noise.6745 yieds estimator such that E{ˆσ (mad) } = σ ɛ when W 1,t s are IID Gaussian with mean and variance σɛ designed to be robust against arge W 1,t s due to signa VIII 51 VIII 5

14 MAD Scae Estimator: II Exampes of DWT-Based Threshoding: I exampe: suppose W 1 has 7 sma noise coefficients & arge signa coefficients (say, a & b, with b > a ): W 1 =[1.3, 1.7,.8,.1,a,.3,.67,b, 1.33] T ordering these by their magnitudes yieds.1,.3,.67,.8, 1.3, 1.33, 1.7, a, b median of these absoute deviations is 1.3, so ˆσ (mad) =1.3/.6745 =1.8. ˆσ (mad) not infuenced adversey by a and b; i.e., scae estimate depends argey on the many sma coefficients due to noise t X D (ht), LA(8) D (ht), D(4) top pot: NMR spectrum X midde: signa estimate using J = 6 partia LA(8) DWT with hard threshoding and universa threshod eve estimated by ˆδ (u) = [ˆσ (mad) og (N) bottom: same, but now using D(4) DWT VIII 53 VIII 54 Exampes of DWT-Based Threshoding: II Exampes of MODWT-Based Threshoding 4 D (ht), LA(8) 4 D (ht), LA(8) 4 D (st), LA(8) 4 D (st), LA(8) 4 D (mt), LA(8) 4 D (mt), LA(8) t top: signa estimate using J = 6 partia LA(8) DWT with hard threshoding (repeat of midde pot of previous overhead) midde: same, but now with soft threshoding bottom: same, but now with mid threshoding VIII t as in previous overhead, but using MODWT rather than DWT because of MODWT fiters are normaized differenty, universa threshod must be adjusted for each eve: δ (u) j [ σ (mad) og (N)/j. ] = / j/ resuts are identica to what cyce spinning woud yied VIII 56

15 VisuShrink: I recipe with soft threshoding is known as VisuShrink (Donoho & Johnstone, 1994) but is reay threshoding, not shrinkage one theoretica justification for VisuShrink consider the risk for a possibe signas D using VisuShrink: R( D (st), D) E{ D (st) D } consider idea risk R( D (i), D) formed with the hep of an orace that tes us which W j,t s are dominated by noise Donoho & Johnstone (1994), Theorem 1: R( D (st), D) [ og(n) + 1][σɛ + R( D (i), D)] two risks differ by ony a ogarithmic factor do poorer when compared to the idea risk VisuShrink: II rather than using the universa threshod, can aso determine δ for VisuShrink by finding vaue ˆδ (S) that minimizes SURE, i.e., J N j 1 (ˆσ (mad) W j,t + δ )1 [δ, ) (W j,t ), j=1 t= as a function of δ, with σ ɛ estimated via MAD VIII 57 VIII 58 Exampes of DWT-Based Threshoding: III Waveet-Based Shrinkage: I 4 ˆδ (S). = assume mode of stochastic signa pus Gaussian IID noise: X = C + ɛ so that W = WX = WC + Wɛ R + e t ˆδ (S). = top: VisuShrink estimate based upon eve J = 6 partia LA(8) DWT and SURE with MAD estimate based upon W 1 ony bottom: same, but now with MAD estimate based upon W 1, W,...,W 6 (the common variance in SURE is assumed common to a waveet coefficients) resuting signa estimate of bottom pot is ess noisy than for top pot component-wise, have W j,t = R j,t + e j,t form partia DWT of eve J, shrink W j s, but eave V J aone assume E{R j,t } = (reasonabe for W j, but not for V J ) use a conditiona mean approach with the sparse signa mode R j,t has distribution dictated by (1 I j,t )N (,σg ), where P [ I j,t =1 ] = p and P [ I j,t = ] =1 p R j,t s are assumed to be IID mode for e j,t is Gaussian with mean and variance σɛ note: parameters do not vary with j or t VIII 59 VIII 6

16 Waveet-Based Shrinkage: II Exampes of Waveet-Based Shrinkage mode has three parameters σ G, p and σ ɛ, which need to be set et σ R and σ W be variances of RVs R j,t and W j,t have reationships σ R =(1 p)σ G and σ W = σ R + σ ɛ set ˆσ ɛ =ˆσ (mad) using W 1 et ˆσ W be sampe mean of a W j,t given p, et ˆσ G =(ˆσ W ˆσ ɛ)/(1 p) p usuay chosen subjectivey, keeping in mind that p is proportion of negigibe signa coefficients t p =.9 p =.95 p =.99 shrinkage signa estimates of the NMR spectrum based upon the eve J = 6 partia LA(8) DWT and the conditiona mean with p =.9 (top pot),.95 (midde) and.99 (bottom) as p 1, we decare there are proportionatey fewer nonzero signa coefficients, impying need for heavier shrinkage VIII 61 VIII 6 Case Study Denoising ECG Time Series: I Case Study Denoising ECG Time Series: II X X. D (ht). R (ht).. D (st). R (st).. D (mt). R (mt) t (seconds) hard/soft/mid threshod estimates with J = 6 partia LA(8) DWT, MAD & scaing coefficients to (zaps baseine drift) VIII t (seconds) residuas from signa estimates, i.e., R (t) = X D (t) (assumption of constant noise variance is questionabe) VIII 64

17 Comments on nd Generation Denoising: I 1.5 R t (seconds) T V 6 T 3 W 6 T 3 W 5 T 3 W 4 T 3 W 3 T W T W 1 1st generation denoising ooks at each W j,t aone; for rea word signas, coefficients often custer within a given eve and persist across adjacent eves (ECG series offers an exampe) VIII 65 X Comments on nd Generation Denoising: II here are some nd generation approaches that expoit these rea word properties: Crouse et a. (1998) use hidden Markov modes for stochastic signa DWT coefficients to hande custering, persistence and non-gaussianity Huang and Cressie () consider scae-dependent mutiscae graphica modes to hande custering and persistence Cai and Siverman (1) consider bock theshoding in which coefficients are threshoded in bocks rather than individuay (handes custering) Dragotti and Vetteri (3) introduce the notion of waveet footprints to track discontinuities in a signa across different scaes (handes persistence) VIII 66 Comments on nd Generation Denoising: III Additiona References 1st generation denoising aso suffers from probem of overa significance of mutipe hypothesis tests nd generation work integrates idea of fase discovery rate (Benjamini and Hochberg, 1995) into denoising (see Wink and Roerdink, 4, for a recent appications-oriented discussion) Y. Benjamini and Y. Hochberg (1995), Controing the Fase Discovery Rate: A Practica and Powerfu Approach to Mutipe Testing, Journa of the Roya Statistica Society, Series B, 57, pp. 893 T. Cai and B. W. Siverman (1), Incorporating Information on Neighboring Coefficients into Waveet Estimation, Sankhya Series B, 63, pp P. L. Dragotti and M. Vetteri (3), Waveet Footprints: Theory, Agorithms, and Appications, IEEE Transactions on Signa Processing, 51, pp H. C. Huang and N. Cressie (), Deterministic/Stochastic Waveet Decomposition for Recovery of Signa from Noisy Data, Technometrics, 4, pp A. M. Wink and J. B. T. M. Roerdink (4), Denoising Functiona MR Images: A Comparison of Waveet Denoising and Gaussian Smoothing, IEEE Transactions on Medica Imaging, 3(3), pp VIII 67 VIII 68