4. April 2, 27 -order sequences Measurements produce sequences of numbers Measurement purpose: characterize a stochastic process. Example: Process: water surface elevation as a function of time Parameters: mean water level, wave variance (RMS height), power spectrum etc... Commonly obtainable data are: not one realization; not a nite segment of one realization; but a discrete nite segment of one realization. Measurements = nite sequence of numbers. Goal : Apply the theory (e.g. spectral analysis) to sequences.
Denition & properties -order sequences -order sequence is: a set of numbers: {g k }, g k C, k =,,...,. 2 an -dimensional vector: g = (g, g 2,..., g ) C 3 a complex function g of an integer argument g(k). Term-by-term operations op set function vector + {a k } + {b k } = {a k + b k } a + b a + b {a k } {b k } = {a k b k } ab? α α {a k } = {αa k } αa αa / {a k } / {b k } = {a k /b k } a/b?? means no commonly used corresponding op. α is a complex constant Periodicity and symmetry -order sequences As a function, g is dened for k =,,...,. Extend k by gluing together copies of g. Then Extended g is -periodic g(k + ) = g(k). 2 Can dene symmetries for g continuous g( t) = ±g(t) discrete g( k) = g( k) = ±g(k) Using sequence notation g k = g k = ±g k
Example: Periodicity -order sequences k = 3; = ; k + = 3; k = 7. Example: Even/Odd -order sequences k = 3; = ; k + = 3; k = 7; k = 7
-order sequences Problem: Fourier representation for sequences Measured data is discrete: function g of integer argument, g(k), k =,,...,. Is it possible to build a Fourier representation for g? (DFT = Discrete Fourier Transform) What would be it's properties? What properties of the continuous version transfer to DFT? What new properties would DFT have? Procedure for building DFT -order sequences eeded Continuous Discrete time/freq. domain Fourier pair axis g(t) G(f ) = ĝ(f ) grids g n = g(n) G k = G(k) = ĝ(k) g ĝ Remarks: G(f ) = g(t)e 2πift dt g(ft) = G(t)e2πift df G = D [g] g = D [G] The goal is to dene and study DFT operator D; 2 Grids do not enter DFT process: Djust maps sequence sequence; 3 Grids important for physical intrerpretation; 4 Will prefer G(k) instead of ĝ(k);
Step : Discretize FT -order sequences Discretize Fourier Integrals: Direct: Time grid:, t n = n t, n =,,..., t ; G(f ) = g(t) e 2πift dt t n= Time grid dened. Frequency grid OT dened. g(t n )e 2πift n t 2 Inverse: Freq. grid: f k = k f, k =,,..., f ; g(t) = G(f ) e 2πift df f k= Frequency grid dened. Time grid OT dened. o visible relation between time-freq. grids. Grids important for physical intrerpretation. G(f k )e 2πif k t f Step 2: Dene grids -order sequences Time/freq grids have to satisfy Invertibility: t = f =. DFT = linear transformation of vectors. Physical constraints: Time grid: dened by measurement. points t n = n t, n =,,..., ; T = ( ) t; 2 Frequency grid dened by physics: largest period = T f = T ; highest frequency on grid: f max = ( ) f = T ; points; f k = k f, k =,,..., ; f = /T ;
Step 3: Put it all together -order sequences Time grid: t n = n t, n =,,..., ; T = ( ) t; Frequency grid: f k = k f, k =,,..., f ; f = /T ; f k t n = kn f t = kn ote: G(f k ) g(t n ) T T k= n= g(t n )e 2πif k t n = T G(f k )e 2πi kn n= g n e 2πi in G(f k ), rst & last terms: g & g (no exponential); If g = g (periodic sequence), rename =. kn Recount for a periodic sequence -order sequences If the sequence is periodic, we only need terms. Renaming : t = T t = T 2 n= n= 3 {g n } has terms 4 but g = g (periodic) Renaming
Denition and grids for DFT -order sequences The DFT form below is from J Weaver (Mathematica). Denition Grids: t n = n t, t = T /; f k = k f ; f = /T ; 2 Grid reciprocity: t f = /, and Tf max =. 3 DFT and IDFT are dened as: G k = n= g n e 2πif k t n = n= g n e 2πi kn = n= g n ( Ω kn ), g n = j= G k e 2πif k t n = k= G k e 2πi kn = k= G k Ω kn. 4 Ω = e 2πi is called DFT kernel. Other DFT forms -order sequences See for example Briggs & Henson The DFT, an owner's manual Mathematica Matlab IMSL G k = P n= g kn 2πi P ne g n = k= G kn 2πi ke G k = P n= g (k )(n ) 2πi ne g n = P k= G (k )(n ) 2πi ke G k+ = P n= g kn 2πi P n+e g n+ = k= G kn 2πi k+e etc...
Matricial form of DFT -order sequences Use the vector notation g = {g n } and G = {G k }; 2 introduce the matrix Ω = ( ) Ω kn, k, n =,... ; Sequence Matrix G k = g k = P n G k P n g `Ωkn n `Ωkn G = Ω g g = Ω G where Ω = exp(2πi/). For example: B @ G G G 2... G C A = B @... Ω Ω 2... Ω Ω 2 Ω 4... Ω 2( )............... Ω Ω 2( )... Ω ( )2 C A B @ g g g 2... g C A -order sequences DFT Kernel matrix is orthogonal (unitary) Fourier basis functions are orthonormal: e2πift dt = δ(f ). What about the DFT? Rows u k = {( ) } { } u k = Ω kn of kernel matrix Ω : Compare with Theorem u k = ( u k = ( n e 2πik e 2πik 2... e 2πik e 2πif k t e 2πif k t 2... e 2πif k t U(t n ) = e 2πift n. For any integer k, with n and n, n= ( ) u k = n n= Ω kn = 2πi kn e =. n= ) )
DFT Kernel orthonormal -order sequences Proof. Form the sequence V n = V n+ V n = Ωk(n+) Ω k Ωkn Ω k, Ωkn Ω k = Ωkn ( Ω k ) Ω k V n primitive of Ω kn ; V n is periodic, V n = V n+. = Ω kn. n= Ω kn = n= V n = V V =. DFT as operator Convolution, Correlation, Parseval Relation {G k } = D [{g n }] ; {g n } = D [{G k }] D [{g n }]] = n= g n Ω kn. otations f, g, {f j },{g k } are used for sequences the DFT are capitalized F, G A term in the sequence is denoted by the name of the sequence with an index f j, g k {G k } = k= g n (Ω k ) n G = D [g]
DFT: The usual properties () Convolution, Correlation, Parseval Relation Linearity: D [af + bg] = af + bg. Shift : FT F[X (t a)] = ˆX (f ) e 2πifa DFT n D [{g n m }] = {G k } Ω kn o Shift 2: F[X (t) e 2πif t ] = ˆX (f f ) D [{g n } {Ω mn }] = {G k m } Derivative : Derivative 2: For example, d ˆX (f ) df» dx (t) F dt h n = 2πiF [tx (t)] {G j } = D {g k } Ω k oi o = 2πif ˆX (f ) D [ {gk }] = {G j } nω j D [{f k } {cos(2πkn/)}] = 2 [{F j+n} + {F j n }] D [{f k } {sin(2πkn/)}] = i 2 [{F j+n} {F j n }] DFT: The usual properties (3) Convolution, Correlation, Parseval Relation Symmetry relations g real, even real, odd imaginary, even imaginary, odd G real, even imaginary, odd imaginary, even real, odd
Plots Convolution, Correlation, Parseval Relation.8 (a) (b).8.8 (a) (b).8.6.6.6.6.4.4.4.4.2.2.2.2.2.2.2.2.5 (c) (d).5.5 (c) (d).5.5.5.5.5 4 2 2 4 4 2 2 4 t f 4 2 2 4 4 2 2 4 t f Examples Convolution, Correlation, Parseval Relation th-order Delta sequence: with k =,,...,, {δ k } = {,,..., }, Unit sequence, with k =,,...,,{u k } = {,,..., }, Delta and Unit sequences are a DFT pair: {u j} = D [{δ k }]. {δ j } = D [{u k }]. 2 The DFT of the cosine is the sum of 2 symmetric deltas: [{ ( )}] 2πkn D cos = 2 [{δ j+n} + {δ j n }].
Convolution Convolution, Correlation, Parseval Relation Let f and g be two -order sequences, Denition Theorem {(f g) n } = m= {f m } {g n m }. (i) D [fg] = D [f ] D [g], (ii) D [f g] = D [f ] D [g], Correlation Convolution, Correlation, Parseval Relation Denition Theorem {(f g) n } = m= {f m } { g n+m}. (i) D [fg ] = F G, (ii) D [f g] = F G,
Parseval Relation Convolution, Correlation, Parseval Relation Theorem If Let f and F form a DFT pair of th-order sequences, k= f n 2 = k= F k 2 DFT-specic properties Zero padding Aliasing What happens when a sequence is padded with zeros? Use symmetric DFT {G j } = D [{g k }] ; D[ ] = + /2 k= /2 [ ] Ω jk. g old -order sequence; g new M-order sequence zero-padded g new k = M/2 k /2 g old /2 k /2 k /2 k M/2
Zero padding Zero padding Aliasing If say M + = p( + ), DFT of g new : G old j = pg new pj, j = /2,..., /2. Reciprocity relations : f old = p f new. Zero-padding is a technique which renes the frequency grid. Zero padding Zero padding Aliasing (a) (a).5.5 4 3 2 2 3 4 k 4 3 2 2 3 4 k (b) (b).5.5 2 5 5 5 5 2 j 8 6 4 2 2 4 6 8 j a) sequence g; b) its Fourier pair G. Blue line: true shape of g and G. Red circles: zero-padded. Blue stars: no zero-padding.
Aliasing Zero padding Aliasing Aliasing: misrepresenting an oscillation which cannot be resolved as a dierent (lower) frequency. 2 4 6 8 2 4 6 8 2 k Denition The frequency f y = f max = is called the yquist frequency. 2 2 t 2 f y is the highest frequency that can be resolved un-aliased by spectral analysis from a sequence samplet at t. Shannon Sampling Theorem Zero padding Aliasing Theorem (Shannon sampling theorem) ĝ(f ) = for f > f y. If t is chosen so that t fmax, then g(t) may be reconstructed exactly from its samples g n = g(n t) by g(t) = n= g n sinc [ π (t tn ) t ].
Continuous Discrete X (t) = one realization; 2 DFT pair ˆX = F [X ] ; 3 Sample spectrum T ˆX T (f ) 2 ; { 4 h(f ) = E lim T ˆX T (f ) T 2 } {g k } = one realization; 2 DFT pair {G n } = D [{g k }] ; { 3 Periodogram G n 2} ; 4 Average, let. Step 3: Periodogram Parseval, continuous: Parseval, discrete: X (t) 2 dt = ˆX (f ) 2 df k= g n 2 = k= G k 2 Denition Sample spectrum (periodogram) of sequence{g n } is the sequence{p k } : P k = G x 2 Periodogram = natural estimate of power spectrum: {P k } = DFT of autocorrelation {c n } = n= {g m} { } gn+m 2 Direct discretization of continuous version of denition.
Step 4: Let... {P k } is a consistent estimate of the power spectrum if a) lim E [P k ] = h(f k ); and b) lim σp 2 = Example: White noise process. Left: {P k } for =2, 5,, 5. Right: variance of P k versus. 5 5 4 (a) (b) 4 3 3 I k I k 2 2 5 5 2 4 (c) (d) 4.5 I k 3 2 3 I k 2 variance.5..2.3.4.5 k..2.3.4.5 k 2 4 6 8 It does not work! The raw periodogram is extremely poor as estimate h(f k ): P k is not a consistent estimate: σp 2 2 P k = P(f k ) is wildly uctuating. as. ote: we did not do any average yet!
Periodogram statistics Theorem If g is a discrete 2-order stationary process, g = (, σg), 2 then G = D [g] = (, σ2 g) (µ G = ; σ 2 = G σ2 g ; 2 P k and P k+m are independent for m, and σ 2 = P 4σ4 g, for n. Increasing ncreases frequency resolution but does not reduce oscillations: P k are independent does not reduce errors: σp 2 independent of. Heuristical explanation Problem: Error (variance) of auto-correlation{c n } decays like / as. 2 {P k } = D [{c n }] = linear combination of auto-correlations 3 cumulative eects of /-like terms is order. Fix: Since the c n as n, end points contribute only error! Truncate the auto-correlation (maximum lag) Window the time series! Smooth the periodogram!
Discarding large lags Windowing Typical autocorrelation (blue line) curve and associated variance (read: error bars and light blue band). Excluding large lags windowing the time series by a window of T length. Bartlett procedure: average over time blocks Bartlett: goal is to reduce variance Split {g n }, n =,,..., into M sub-blocks of{g n } m of /M points. 2 Calculate autocovariance {c n } m for each sub-block m =,... M. 3 Calculate periodogram {P k } m = D [{c n } m ] for each sub-block 4 Average periodogram over sub-blocks: {H k } = M m {P k} m. The procedure is equivalent to windowing by triangular (Bartlett) window.
Welch's direct method Compute DFT of the time series rather than DFT of the autocovariance: Divide{g n } into M sub-blocks of{g n } m of K = /M points; sub-blocks allowed to overlap (typically by 5 %). 2 Window each sub-block: g n W = {gn } {w n }, {w n }= window. Redundancy due to overlapping is reduced by window tapering. 3 Tapering reduces the variance (see next slide). Adjust for variance change due to windowing: {g w n } m {g w n } m q P ˆ{gn } m 2/q P ˆ{g w n } m 2. 4 DFT each tapered sub-block, calculate the periodogram : {P k } m = K G w k 2. 5 Average over sub-blocks: P {H k } = M m {P k} m. Tapering reduces variance Windowed signal (red) has smaler variance than the original signal (blue).
Spectrum 6 power density 5 4 3 2 (a) 25 2 (b) power density 5 5.5..5.2.25.3.35.4 frequency (Hz) a) {P k } = G k 2, calculated using the entire time series; b) Spectrum estimate by Welch's direct method.