Wavelet Methods for Time Series Analysis. What is a Wavelet? Part I: Introduction to Wavelets and Wavelet Transforms. sines & cosines are big waves

Transcription

1 Wavele Mehods for Time Series Analysis Par I: Inroducion o Waveles and Wavele Transforms waveles are analysis ools for ime series and images as a subjec, waveles are relaively new (983 o presen) a synhesis of old/new ideas keyword in 9, 86+ aricles and books since 989 (43 more since 5: an inundaion of maerial!!!) broadly speaking, here have been wo waves of waveles coninuous wavele ransform (983 and on) discree wavele ransform (988 and on) will inroduce subjec via CWT & hen concenrae on DWT Wha is a Wavele? sines & cosines are big waves u waveles are small waves (lef-hand is Haar wavele ψ (H) ( )) u u u I I Technical Definiion of a Wavele: I real-valued funcion ψ( ) defined over real axis is a wavele if inegral of ψ ( ) is uniy: ψ (u) du = (called uni energy propery, wih apologies o physiciss) inegral of ψ( ) is zero: ψ(u) du = (echnically, need an admissibiliy condiion, bu his is almos equivalen o inegraion o zero) u u u Technical Definiion of a Wavele: II ψ (u) du =& ψ(u) du = give a wavele because: by propery, for every small ɛ>, have T ψ (u) du + T ψ (u) du<ɛ for some finie T business par of ψ( ) is over inerval [ T,T widh T of [ T,T migh be huge, bu will be insignifican compared o (, ) by propery, ψ( ) is balanced above/below horizonal axis maches inuiive noion of a small wave I 3 I 4

2 Two Non-Waveles and Three Waveles wo failures: f(u) = cos(u) & same limied o [ 3π/, 3π/: u u Haar wavele ψ (H) ( ) and wo of is friends: Wha is Wavele Analysis? waveles ell us abou variaions in local averages o quanify his descripion, le x( ) be a signal real-valued funcion of defined over real axis will refer o as ime (bu i need no be such) consider average value of x( ) over [a, b: b a b a x() d u u u I 5 I 6 Example of Average Value of a Signal le x( ) be sep funcion aking on values x,x,,x 5 over 6 equal subinervals of [a, b: a x x here we have b a b a 5 x 5 x() d = x 6 j = heigh of dashed line j= b Average Values a Differen Scales and Times define he following funcion of λ and + λ A(λ, ) x(u) du λ λ λ is widh of inerval refered o as scale is midpoin of inerval A(λ, ) is average value of x( ) over scale λ cenered a average values of signals have wide-spread ineres one second average emperaures over fores en minue rainfall rae during severe sorm yearly average emperaures over cenral England I 7 I 8

3 Defining a Wavele Coefficien W muliply Haar wavele & ime series x( ) ogeher: ψ (H) () x() ψ (H) ()x() inegrae resuling funcion o ge wavele coefficien W (, ): ψ (H) ()x() d = W (, ) o see wha W (, ) is elling us abou x( ), noe ha W (, ) x() d x() d = A(, ) A(, ) Defining Wavele Coefficiens for Oher Scales W (, ) proporional o difference beween averages of x( ) over [, &[,, ie, wo uni scale averages before/afer = in W (, ) denoes scale (widh of each inerval) in W (, ) denoes ime (cener of combined inervals) srech or shrink wavele o define W (,) for oher scales : yields W (, ) I 9 I Defining Wavele Coefficiens for Oher Locaions relocae o define W (,) for oher imes : yields W (, ) yields W (, ) Haar Coninuous Wavele Transform (CWT) for all >and all <<, can wrie W (,)= ( ) u x(u)ψ (H) du u does he sreching/shrinking and relocaing needed so ψ, (H) (u) ( ψ u ) (H) has uni energy since i also inegraes o zero, ψ, (H) ( ) is a wavele W (,) over all >and all is Haar CWT for x( ) analyzes/breaks up/decomposes x( ) ino componens associaed wih a scale and a ime physically relaed o a difference of averages I I

4 Oher Coninuous Wavele Transforms: I can do he same for waveles oher han he Haar sar wih basic wavele ψ( ) use ψ, (u) = ψ ( u ) o srech/shrink & relocae define CWT via W (,)= x(u)ψ, (u) du = ( ) u x(u)ψ analyzes/breaks up/decomposes x( ) ino componens associaed wih a scale and a ime physically relaed o a difference of weighed averages du Oher Coninuous Wavele Transforms: II consider wo friends of Haar wavele ψ (H) (u) ψ (fdg) (u) ψ (Mh) (u) u u u ψ (fdg) ( ) proporional o s derivaive of Gaussian PDF Mexican ha wavele ψ (Mh) ( ) proporional o nd derivaive ψ (fdg) ( ) looks a difference of adjacen weighed averages ψ (Mh) ( ) looks a difference beween weighed average and sum of weighed averages occurring before & afer I 3 I 4 Firs Scary-Looking Equaion Second Scary-Looking Equaion CWT equivalen o x( ) because we can wrie [ x() = C W (,u) ( ) u ψ du d, where C is a consan depending on specific wavele ψ( ) can synhesize (pu back ogeher) x( ) given is CWT; ie, nohing is los in reexpressing signal x( ) via is CWT regard suff in brackes as defining scale signal a ime says we can reexpress x( ) as inegral (sum) of new signals, each associaed wih a paricular scale similar addiive decomposiions will be one cenral heme energy in x( ) is reexpressed in CWT because [ energy = x () d = C W (,) d d can regard x () versus as breaking up he energy across ime (ie, an energy densiy funcion) regard suff in brackes as breaking up he energy across scales says we can reexpress energy as inegral (sum) of componens, each associaed wih a paricular scale funcion defined by W (,)/C is an energy densiy across boh ime and scale similar energy decomposiions will be a second cenral heme I 5 I 6

5 Example: Aomic Clock Daa example: average daily frequency variaions in clock 57 Mexican Ha CWT of Clock Daa: I (days) is measured in days (one measurmen per day) plo shows versus ineger = for all would say ha clock 57 keeps ime perfecly < implies ha clock is losing ime sysemaically can easily adjus clock if were consan inheren qualiy of clock relaed o changes in averages of I (days) I 8 Mexican Ha CWT of Clock Daa: II Mexican Ha CWT of Clock Daa: III (days) (days) I 9 I

6 Mexican Ha CWT of Clock Daa: IV Beyond he CWT: he DWT can ofen ge by wih subsamples of W (,) leads o noion of discree wavele ransform (DWT) (can regard as discreized slices hrough CWT) (days) I I Raionale for he DWT DWT has appeal in is own righ mos ime series are sampled as discree values (can be ricky o implemen CWT) can formulae as orhonormal ransform (makes meaningful saisical analysis possible) ends o decorrelae cerain ime series sandardizaion o dyadic scales ofen adequae generalizes o noion of wavele packes can be faser han he fas Fourier ransform will concenrae primarily on DWT for remainder of course Qualiaive Descripion of DWT will give precise definiion of DWT in Par II le X =[X,X,,X N T be a vecor of N ime series values (noe: T denoes ranspose; ie, X is a column vecor) need o assume N = J for some posiive ineger J (resricive!) DWT is a linear ransform of X yielding N DWT coefficiens noaion: W = WX W is vecor of DWT coefficiens (jh componen is W j ) W is N N orhonormal ransform marix; ie, W T W = I N, where I N is N N ideniy marix inverse of W is jus is ranspose, so WW T = I N also I 3 I 4

7 Implicaions of Orhonormaliy: I le Wj T denoe he jh row of W, where j =,,,N noe ha W j iself is a column vecor le W j,l denoe elemen of W in row j and column l noe ha W j,l is also lh elemen of W j le s consider wo vecors, say, W j and W k orhonormaliy says W j, W k N l= W j,l W k,l = {, when j = k,, when j k W j, W k is inner produc of jh & kh rows Implicaions of Orhonormaliy: II example from W of dimension 6 6 we ll see laer on inner produc of row 8 wih iself (ie, squared norm): W 8, W 8, W 8, sum = row 8 said o have uni energy since squared norm is W j W j, W j is squared norm (energy) for W j I 5 I 6 Implicaions of Orhonormaliy: III anoher example from same W inner produc of rows 8 and : W 8, W, W 8, W, sum = rows8&said o be orhogonal since inner produc is The Haar DWT: I like CWT, DWT ell us abou variaions in local averages o see his, le s look inside W for he Haar DWT for N = J row j =: [,,,, }{{} W T N zeros noe: W = + = & hence has required uni energy row j =: [,,,,,, }{{} W T N 4 zeros W and W are orhogonal W, W, W, W, sum = I 7 I 8

8 The Haar DWT: II keep shifing by wo o form rows unil we come o row j = N : [,, }{{},, W T N N zeros firs N/ rows form orhonormal se of N/ vecors N = 6 example The Haar DWT: III o form nex row, srech [,,,, ou by a facor of wo and renormalize o preserve uni energy j = N : [,,,,,, }{{} N 4 zeros noe: W N = W and W N are orhogonal (N W, W T N =, as required = 8 in example) W 8, W, W 8, sum = I 9 I 3 The Haar DWT: IV W and W N are orhogonal W, W 8, W, W 8, sum = W and W N are orhogonal W, W 8, W, W 8, sum = I 3 The Haar DWT: V form nex row by shifing W N o righ by 4 unis W T N + j = N +: [,,,,,,,, },, {{} N 8 zeros W N + orhogonal o firs N/ rows and also o W N W 8, W 9, W 8, W 9, sum = coninue shifing by 4 unis o form more rows, ending wih row j = 3N 4 : [ },, {{},,,, W T 3N 4 N 4 zeros I 3

9 The Haar DWT: VI o form nex row, srech [,,,,,, ou by a facor of wo and renormalize o preserve uni energy j = 3N 4 : [ 8,, }{{} 8, 8,, 8,,, }{{}}{{} W T 3N 4 N 8 zeros 4 of hese 4 of hese noe: W 3N4 =8 8 =, as required j = 3N 4 + : shif row 3N 4 o righ by 8 unis coninue shifing and sreching unil finally we come o j = N : [ N,, }{{ N, } N,, N W T N }{{} N of hese N of hese j = N : [ N,, }{{ N W T N } N of hese I 33 The Haar DWT: VII N = 6 example of Haar DWT marix W I Haar DWT Coefficiens: I obain Haar DWT coefficiens W by premuliplying X by W: W = WX jh coefficien W j is inner produc of jh row W j and X: W j = W j, X can inerpre coefficiens as difference of averages o see his, le (λ) λ λ l= l = scale λ average noe: () = = scale average noe: X N (N) =X = sample average Haar DWT Coefficiens: II consider form W = W, X akes in N = 6 example: W, W, sum X () X () similar inerpreaion for W,,W N = W 7 = W 7, X : W 7, W 7, sum X 5 () X 4 () I 35 I 36

10 Haar DWT Coefficiens: III now consider form of W N = W 8 = W 8, X : W 3N4 Haar DWT Coefficiens: IV = W = W 8, X akes he following form: W 8, W 8, sum X 3 () X () similar inerpreaion for W N +,,W 3N 4 W 8, W 8, sum X 7 (4) X 3 (4) coninuing in his manner, come o W N = W 4, X : W 4, W 4, sum X 5 (8) X 7 (8) I 37 I 38 Haar DWT Coefficiens: V final coefficien W N = W 5 has a differen inerpreaion: W 5, W 5, sum X 5 (6) srucure of rows in W firs N rows yield W j s changes on scale nex N 4 rows yield W j s changes on scale nex N 8 rows yield W j s changes on scale 4 nex o las row yields W j change on scale N las row yields W j average on scale N Srucure of DWT Marices N j wavele coefficiens for scale j j, j =,,J j j is sandardized scale j is physical scale, where is sampling inerval each W j localized in ime: as scale, localizaion rows of W for given scale j : circularly shifed wih respec o each oher shif beween adjacen rows is j = j similar srucure for DWTs oher han he Haar differences of averages common heme for DWTs simple differencing replaced by higher order differences simple averages replaced by weighed averages I 39 I 4

11 Two Basic Decomposiions Derivable from DWT addiive decomposiion reexpresses X as he sum of J + new ime series, each of which is associaed wih a paricular scale j called muliresoluion analysis (MRA) relaed o firs scary-looking CWT equaion energy decomposiion yields analysis of variance across J scales called wavele specrum or wavele variance relaed o second scary-looking CWT equaion Pariioning of DWT Coefficien Vecor W decomposiions are based on pariioning of W and W pariion W ino subvecors associaed wih scale: W W W = W j W J V J W j has N/ j elemens (scale j = j changes) noe: J j= N j = N + N =J =N V J has elemen, which is equal o N X (scale N average) I 4 I 4 Example of Pariioning of W consider ime series X of lengh N = 6 & is Haar DWT W W X W W W 3 W 4 V Pariioning of DWT Marix W pariion W commensurae wih pariioning of W: W W W = W j W J V J W j is N j N marix (relaed o scale j = j changes) V J is N row vecor (each elemen is ) N I 43 I 44

12 Example of Pariioning of W N = 6 example of Haar DWT marix W W wo properies: (a) W j = W j X and (b) W j W T j W W 3 W 4 V 4 = I N j DWT Analysis and Synhesis Equaions recall he DWT analysis equaion W = WX W T W = I N because W is an orhonormal ransform implies ha W T W = W T WX = X yields DWT synhesis equaion: W [ X = W T W = W T, WT,,WT J, W VT J W J V J J = Wj T W j + VJ T V J j= I 45 I 46 Muliresoluion Analysis: I synhesis equaion leads o addiive decomposiion: J J X = Wj T W j + VJ T V J D j + S J j= j= D j W T j W j is porion of synhesis due o scale j D j is vecor of lengh N and is called jh deail S J VJ T V J = X, where is a vecor conaining N ones (laer on we will call his he smooh of Jh order) addiive decomposiion called muliresoluion analysis (MRA) Muliresoluion Analysis: II example of MRA for ime series of lengh N =6 X 5 5 adding values for, eg, =4inD,,D 4 & S 4 yields X 4 S 4 D 4 D 3 D D I 47 I 48

13 Energy Preservaion Propery of DWT Coefficiens define energy in X as is squared norm: X = X, X = X T X = N = (usually no really energy, bu will use erm as shorhand) energy of X is preserved in is DWT coefficiens W because W = W T W =(WX) T WX = X T W T WX = X T I N X = X T X = X X Wavele Specrum (Variance Decomposiion): I le X denoe sample mean of s: X N le ˆσ X denoe sample variance of s: ˆσ X N N = ( X X ) N = N = N = X X = N X X = N W X since W = J j= W j + V J and N V J = X, ˆσ X = J W N j j= I 49 I 5 Wavele Specrum (Variance Decomposiion): II define discree wavele power specrum: P X ( j ) N W j, where j = j gives us a scale-based decomposiion of he sample variance: J ˆσ X = P X ( j ) j= in addiion, each W j, in W j associaed wih a porion of X; ie, W j, offers scale- & ime-based decomposiion of ˆσ X Wavele Specrum (Variance Decomposiion): III wavele specra for ime series X and Y of lengh N = 6, each wih zero sample mean and same sample variance X Y j P X ( j ) P Y ( j ) I 5 I 5

14 Summary of Qualiaive Descripion of DWT DWT is expressed by an N N orhonormal marix W ransforms ime series X ino DWT coefficiens W = WX each coefficien in W associaed wih a scale and locaion W j is subvecor of W wih coefficiens for scale j = j coefficiens in W j relaed o differences of averages over j las coefficien in W relaed o average over scale N orhonormaliy leads o basic scale-based decomposiions muliresoluion analysis (addiive decomposiion) discree wavele power specrum (analysis of variance) sayed uned for precise definiion of DWT! I 53