Review of Concepts from Fourier & Filtering Theory. Fourier theory for finite sequences. convolution/filtering of infinite sequences filter cascades

Transcription

1 Review of Concepts from Fourier & Filtering Theory precise definition of DWT requires a few basic concepts from Fourier analysis and theory of linear filters will start with discussion/review of: basic ideas behind Fourier analysis of time series Fourier theory for infinite sequences convolution/filtering of infinite sequences filter cascades Fourier theory for finite sequences circular convolution/filtering of finite sequences periodization of a filter WMTSA: Chapter 2 III 1

2 What is Fourier Analysis?: I one of the most widely used methods for data analysis in geophysics oceanography atmospheric science astronomy engineering (all types) etc. used to analyze time series (observations collected over time) let X t denote value of time series at time indexed by t example: X 89 = 65 = temperature in Loew Hall 105 at 1PM on day 89 of 2018 (30th March) III 2

3 What is Fourier Analysis?: II four examples of time series X 0, X 1,..., X 127 X t X t t t Q: how would you describe these 4 series? Fourier analysis does so by comparing X t s to sines & cosines (note: will collectively refer to sines & cosines as sinusoids ) Q: what do sines and cosines have to do with time series? III 3

4 What is Fourier Analysis?: III let s plot sin(u) and cos(u) versus u as u goes from 0 to 4π: π 4π 0 2π 4π u u let u = 2π128 2 t for t = 0, 1,..., 127 now let s plot sin(2π t) and cos(2π 2 128t) versus t: t t artificial time series exhibiting 2 cycles over time span of 128 (meaning of called frequency f of the sinusoid) III 4

5 Adding Sines & Cosines with Different Frequencies t t t f = 1/128 f = 3/128 f = 5/128 f = 7/128 f = 9/128 f = 11/128 f = 13/128 f = 15/128 f = 17/128 f = 19/128 sinusoid amplitudes fixed in column 1, but random in 2 & 3 III 5 X t

6 What is Fourier Analysis?: IV conclusion: by summing up lots of sinusoids with different amplitudes, can get artificial X t s that resemble actual X t s goal of Fourier analysis: given a time series X t, figure out how to construct it using sinusoids; i.e., to write X t = X A k sin(2πf k t) + B k cos(2πf k t), k where f k s are a collection of different frequencies above called Fourier representation for a time series allows us to reexpress time series in a standard way different time series will need different A k s and B k s can compare time series by comparing their A k s and B k s III 6

7 Some Notation, Conventions and Basic Facts: I easier to do Fourier theory by not dealing with sinusoids directly i 1 and hence i 2 = 1, i 3 = i & i 4 = 1 (note: means equal by definition ) if x & y are real-valued variables, z = x + iy is complex-valued z x iy, z x 2 + y 2 and z 2 = zz and e ix cos(x) + i sin(x) is definition of a complex exponential e ix 2 = 1 because cos 2 (x) + sin 2 (x) = 1 e i(x+y) = e ix e iy just expand out both sides (e ix ) n = e inx for integer n (de Moivre s theorem) e ix dx = eix i because i sin(x)+cos(x) cos(x) + i sin(x) dx = sin(x) i cos(x) = i WMTSA: III 7

8 Some Notation, Conventions and Basic Facts: II since e ix = cos(x) + i sin(x) & e ix = cos(x) i sin(x), have cos(x) = eix +e ix 2 and sin(x) = eix e ix 2i e ±iπ = 1 (trivial, but useful!) can write z = z e iθ (polar representation) z is magnitude of z ( z is nonnegative) θ = arg(z) is argument of z (defined if z 6= 0); θ = angle between positive x axis & line to (x, y) by convention π < θ π (x, y) θ z = x + iy z is length of line from (0, 0) to (x, y) WMTSA: III 8

9 Fourier Theory for Infinite Sequences: I let {a t : t =..., 1, 0, 1,...} = {a t } denote an infinite realvalued sequence satisfying t a2 t < (do not need stronger condition t a t < addendum to overheads has details) discrete Fourier transform (DFT) of {a t }: A(f) a t e i2πft t= f called frequency: e i2πft = cos(2πft) i sin(2πft) (controls how fast cosine & sine go up & down as t increases) A( ) called Fourier analysis of {a t } (note: A( ) is function, while A(f) is value of A( ) at f) WMTSA: III 9

10 Fourier Theory for Infinite Sequences: II A(f) defined for all f, but 0 f 1/2 of main interest because {a t } is real-valued, A( f) = X t= a t e i2πft = X t= a t e i2πft = A (f) A( ) periodic with unit period; i.e., A(f + 1) = A(f) since e i2π(f+1)t = e i2πft e i2πt = e i2πft [cos(2πt) i sin(2πt)] = e i2πft implies A(f + j) = A(f) for any integer j need only know A(f) for 0 f 1/2 to know it for all f low frequencies are those in lower range of [0, 1/2] high frequencies are those in upper range of [0, 1/2] WMTSA: 22 III 10

11 Fourier Theory for Infinite Sequences: III can reconstruct {a t } from its Fourier transform (Exercise [22c]): Z 1/2 1/2 A(f)e i2πft df = a t, t =..., 1, 0, 1,... left-hand side called inverse DFT of A( ) {a t } and A( ) are two representations for one thingy notation: {a t } A( ) means DFT of {a t } is A( ): A(f) = P t a te i2πft inverse DFT of A( ) is {a t }: a t = R 1/2 1/2 A(f)ei2πft df large A(f) says e i2πft important in synthesizing {a t }; i.e., {a t } resembles some combination of cos(2πft) and sin(2πft) WMTSA: III 11

12 Fourier Theory for Infinite Sequences: IV if {a t } A( ) & {b t } B( ), then X Z 1/2 a t b t = A(f)B (f) df t= 1/2 two sequence Parseval s theorem (Exercise [23a]) setting b t = a t yields one sequence Parseval: X Z 1/2 a 2 t = A(f) 2 df t= 1/2 (key to energy decomposition across frequencies) WMTSA: 23 III 12

13 Fourier Theory for Infinite Sequences: V suppose {a t : t = 0,..., N 1} is finite sequence extend to infinite sequence by setting a t = 0 for t < 0 & t N DFT is then A(f) X t= a t e i2πft = N 1 X t=0 a t e i2πft notation: {a t : t = 0,..., N 1} A( ); i.e., zero extension is implicit will use shorthand {a t } A( ) if t = 0,..., N 1 is clear from context WMTSA: 23 III 13

14 Examples of Fourier Transforms of Infinite Sequences consider a t = t t and bt = a t {a.... t }... {b t } t t 2 0 A( ) B( ) blue is real part red is imaginary part f f because {a t } is symmetric about t = 0 (i.e., a t = a t ), its DFT A( ) is real-valued {b t } is asymmetric, so its DFT B( ) is complex-valued III 14

15 Convolution of Infinite Sequences: I given {a t } A( ) and {b t } B( ), define X a b t a u b t u, t =..., 1, 0, 1,... u= note: a b t is just a fancy variable name (could have used c t ) sequence {a b t } is convolution of {a t } and {b t } reverse direction of {b t }, multiply by {a t } & sum to get a b 0 shift {b t } and repeat to get other values a b t a 1 a 0 a 1 a 2 a 3 a b 0 = P u= a ub u b 1 b 0 b 1 b 2 b 3 WMTSA: 24 III 15

16 Convolution of Infinite Sequences: I given {a t } A( ) and {b t } B( ), define X a b t a u b t u, t =..., 1, 0, 1,... u= note: a b t is just a fancy variable name (could have used c t ) sequence {a b t } is convolution of {a t } and {b t } reverse direction of {b t }, multiply by {a t } & sum to get a b 0 shift {b t } and repeat to get other values a b t a 1 a 0 a 1 a 2 a 3 a b 1 = P u= a ub 1 u b 2 b 1 b 0 b 1 b 2 WMTSA: 24 III 15

17 Convolution of Infinite Sequences: I given {a t } A( ) and {b t } B( ), define X a b t a u b t u, t =..., 1, 0, 1,... u= note: a b t is just a fancy variable name (could have used c t ) sequence {a b t } is convolution of {a t } and {b t } reverse direction of {b t }, multiply by {a t } & sum to get a b 0 shift {b t } and repeat to get other values a b t a 1 a 0 a 1 a 2 a 3 a b 2 = P u= a ub 2 u b 3 b 2 b 1 b 0 b 1 WMTSA: 24 III 15

18 Convolution of Infinite Sequences: I given {a t } A( ) and {b t } B( ), define X a b t a u b t u, t =..., 1, 0, 1,... u= note: a b t is just a fancy variable name (could have used c t ) sequence {a b t } is convolution of {a t } and {b t } reverse direction of {b t }, multiply by {a t } & sum to get a b 0 shift {b t } and repeat to get other values a b t a 1 a 0 a 1 a 2 a 3 a b 3 = P u= a ub 3 u b 4 b 3 b 2 b 1 b 0 WMTSA: 24 III 15

19 Convolution of Infinite Sequences: II DFT of {a b t } has a simple form, namely, X a b t e i2πft = A(f)B(f); t= i.e., just multiply two DFTs together (Exercise [24]) can state this result as {a b t } A( )B( ) related concept is (complex) cross-correlation: X X a? b t = a ub u+t = a u b u+t A (f)b(f) u= u= letting b t = a t yields autocorrelation: X a u a u+t A (f)a(f) = A(f) 2 u= WMTSA: III 16

20 Basic Concepts of Filtering: I convolution & linear time invariant filtering are same concepts: {b t } is input to filter {a t } represents the filter {a b t } is output from filter flow diagram for filtering: {b t } {a t } {a b t } or {b t } a t {a bt } since {a t } equivalent to A( ), can also express flow diagram as {b t } A( ) {a b t } WMTSA: 25 III 17

21 Basic Concepts of Filtering: II {a t } called impulse response sequence for filter A( ) called transfer function for filter in general A( ) is complex-valued, so write A(f) = A(f) e iθ(f) A(f) defines gain function A(f) A(f) 2 defines squared gain function θ(f) called phase function (well-defined at f if A(f) > 0) WMTSA: 25 III 18

22 consider b t = Example of a Low-Pass Filter 45 t t & at = 1 2, t = 0 1 4, t = 1 or 1 0, otherwise {b t } {a... t }... {a b t } t t t 2 1 B( ) A( ) A( )B( ) f f f note: A( ) & B( ) are both real-valued (equal to gain functions) WMTSA: III 19

23 Example of a High-Pass Filter consider same {b t }, but now let a t = 0 1 2, t = 0 1 4, t = 1 or 1 0, otherwise {b t }.... {a... t }.... {a b... t } t t t 2 1 B( ) A( ) A( )B( ) f f f might regard {a t } as highly discretized Mexican hat wavelet WMTSA: III 20

24 Cascade of Filters: I idea: output from one filter becomes input to another flow diagram for cascade with 2 filters (can have more!): {b t } A 1 ( ) 1. A 2 ( ) 2. {a b t } if {b t } B( ) and {a b t } C( ), then 1. output from A 1 ( ) has DFT A 1 (f)b(f) 2. output from A 2 ( ) has DFT A 2 (f)a 1 (f)b(f) so C(f) = A 2 (f)a 1 (f)b(f) let A(f) A 2 (f)a 1 (f) can reexpress overall effect of filter cascade as {b t } A( ) {a b t } WMTSA: III 21

25 Cascade of Filters: II A( ) is transfer function for equivalent filter for cascade let {a t } A( ), {a 1,t } A 1 ( ) and {a 2,t } A 2 ( ) to form {a t }, just need to convolve {a 1,t } and {a 2,t } 1 2, t = 1 1 2, t = 0 example: a 1,t = 1 2, t = 0 & a 2,t = 1 2, t = 1 0, otherwise 0, otherwise a 1, 3 a 1, 2 a 1, 1 a 1,0 a 1,1 a 1,2 a 2 = P a 2,1 a 2,0 a 2, 1 a 2, 2 a 2, 3 a 2, 4 u= a 1,ua 2, 2 u WMTSA: III 22

26 Cascade of Filters: II A( ) is transfer function for equivalent filter for cascade let {a t } A( ), {a 1,t } A 1 ( ) and {a 2,t } A 2 ( ) to form {a t }, just need to convolve {a 1,t } and {a 2,t } 1 2, t = 1 1 2, t = 0 example: a 1,t = 1 2, t = 0 & a 2,t = 1 2, t = 1 0, otherwise 0, otherwise a 2 = WMTSA: III 22

27 Cascade of Filters: II A( ) is transfer function for equivalent filter for cascade let {a t } A( ), {a 1,t } A 1 ( ) and {a 2,t } A 2 ( ) to form {a t }, just need to convolve {a 1,t } and {a 2,t } 1 2, t = 1 1 2, t = 0 example: a 1,t = 1 2, t = 0 & a 2,t = 1 2, t = 1 0, otherwise 0, otherwise a 1 = WMTSA: III 22

28 Cascade of Filters: II A( ) is transfer function for equivalent filter for cascade let {a t } A( ), {a 1,t } A 1 ( ) and {a 2,t } A 2 ( ) to form {a t }, just need to convolve {a 1,t } and {a 2,t } 1 2, t = 1 1 2, t = 0 example: a 1,t = 1 2, t = 0 & a 2,t = 1 2, t = 1 0, otherwise 0, otherwise a 0 = WMTSA: III 22

29 Cascade of Filters: II A( ) is transfer function for equivalent filter for cascade let {a t } A( ), {a 1,t } A 1 ( ) and {a 2,t } A 2 ( ) to form {a t }, just need to convolve {a 1,t } and {a 2,t } 1 2, t = 1 1 2, t = 0 example: a 1,t = 1 2, t = 0 & a 2,t = 1 2, t = 1 0, otherwise 0, otherwise a 1 = WMTSA: III 22

30 Cascade of Filters: II A( ) is transfer function for equivalent filter for cascade let {a t } A( ), {a 1,t } A 1 ( ) and {a 2,t } A 2 ( ) to form {a t }, just need to convolve {a 1,t } and {a 2,t } 1 2, t = 1 1 2, t = 0 example: a 1,t = 1 2, t = 0 & a 2,t = 1 2, t = 1 0, otherwise 0, otherwise a 2 = WMTSA: III 22

31 Cascade of Filters: III gives high-pass filter seen earlier: a t = filter {b t } with {a 1,t } to get, say, {a 1 b t }: 0 1 2, t = 0 1 4, t = 1 or 1 0, otherwise {b t } {a 1,t } {a b t } t t t 2 0 B( ) A 1 ( ) A 1 ( )B( ) f f f WMTSA: III 23

32 Cascade of Filters: IV filter {a 1 b t } with {a 2,t } to get same {a b t } as before: 0. {a b t }.... {a 2,t }.... {a b... t } t t t 2 0 A 1 ( )B( ) A 2 ( ) A( )B( ) = A 2 ( )A 1 ( )B( ) f f f cascade of M filters of widths L 1,..., L M has {a t } of width MX L = L m M + 1 (Exercise [28a]) m=1 (check on above example with M = 2: L = = 3) WMTSA: III 24

33 Fourier Theory for Finite Sequences: I let {a t : t = 0, 1,..., N 1} = {a t } denote a finite sequence (same shorthand as for infinite sequence don t get confused!) discrete Fourier transform (DFT) of {a t }: A k N 1 X t=0 a t e i2πf kt, with f k k N & k = 0, 1,..., N 1 note: can define A k for all k, but {A k : k = 0, 1,..., N 1} is DFT (sequence indexed by all integers k is periodic with a period of N; i.e., A k+n = A k ) A k is associated with frequency f k, and 0 f k < 1 A k for 0 f k 1/2 of main interest because A N k = A k (if N even, k = 0,..., N/2 index the frequencies of interest) WMTSA: 28 III 25

34 Fourier Theory for Finite Sequences: II can recover {a t } from its DFT {A k } (Exercise [29a]): 1 N N 1 X k=0 A k e i2πf kt = a t, t = 0, 1,..., N 1; left-hand side called inverse DFT of {A k } {a t } and {A k } are two representations for one thingy relationship between {a t } and {A k } denoted by {a t } {A k } or, less formally, by a t A k can define a t for t < 0 & t N via inverse DFT: {a t : t =..., 1, 0, 1,...} periodic with period N WMTSA: 29 III 26

35 Fourier Theory for Finite Sequences: III if {a t } {A k } & {b t } {B k }, then N 1 X t=0 a t b t = 1 N N 1 X k=0 A k B k two sequence Parseval s theorem (Exercise [29b]) setting b t = a t yields one sequence Parseval: N 1 X t=0 a 2 t = 1 N N 1 X k=0 A k 2 WMTSA: 29 III 27

36 Examples of Fourier Transforms of Finite Sequences two time series {X t } of length N = 16 and their DFTs {X k } X t X k t or k t or k blue is real part red is imaginary part series differ only at t = 13, but their DFTs differ at all k note that X 16 k = X k for k = 1, 2, 3, 4, 5, 6 and 7 WMTSA: 42, 49 III 28

37 Examples of Fourier Transforms of Finite Sequences two time series {X t } of length N = 16 and their DFTs {X k } X t X k t or k t or k blue is real part red is imaginary part series differ only at t = 13, but their DFTs differ at all k note that X 16 k = X k for k = 1, 2, 3, 4, 5, 6 and 7 WMTSA: 42, 49 III 28

38 Examples of Fourier Transforms of Finite Sequences two time series {X t } of length N = 16 and their DFTs {X k } X t X k t or k t or k blue is real part red is imaginary part series differ only at t = 13, but their DFTs differ at all k note that X 16 k = X k for k = 1, 2, 3, 4, 5, 6 and 7 WMTSA: 42, 49 III 28

39 Examples of Fourier Transforms of Finite Sequences two time series {X t } of length N = 16 and their DFTs {X k } X t X k t or k t or k blue is real part red is imaginary part series differ only at t = 13, but their DFTs differ at all k note that X 16 k = X k for k = 1, 2, 3, 4, 5, 6 and 7 WMTSA: 42, 49 III 28

40 Examples of Fourier Transforms of Finite Sequences two time series {X t } of length N = 16 and their DFTs {X k } X t X k t or k t or k blue is real part red is imaginary part series differ only at t = 13, but their DFTs differ at all k note that X 16 k = X k for k = 1, 2, 3, 4, 5, 6 and 7 WMTSA: 42, 49 III 28

41 Examples of Fourier Transforms of Finite Sequences two time series {X t } of length N = 16 and their DFTs {X k } X t X k t or k t or k blue is real part red is imaginary part series differ only at t = 13, but their DFTs differ at all k note that X 16 k = X k for k = 1, 2, 3, 4, 5, 6 and 7 WMTSA: 42, 49 III 28

42 Examples of Fourier Transforms of Finite Sequences two time series {X t } of length N = 16 and their DFTs {X k } X t X k t or k t or k blue is real part red is imaginary part series differ only at t = 13, but their DFTs differ at all k note that X 16 k = X k for k = 1, 2, 3, 4, 5, 6 and 7 WMTSA: 42, 49 III 28

43 Examples of Fourier Transforms of Finite Sequences two time series {X t } of length N = 16 and their DFTs {X k } X t X k t or k t or k blue is real part red is imaginary part series differ only at t = 13, but their DFTs differ at all k note that X 16 k = X k for k = 1, 2, 3, 4, 5, 6 and 7 WMTSA: 42, 49 III 28

44 Convolution/Filtering of Finite Sequences: I given {a t } & {b t } of length N with DFTs {A k } & {B k }, define a b t N 1 X u=0 a u b t u, t = 0, 1,..., N 1 assumes b t defined for t < 0 by periodic extension; thus b 1 = b N 1, b 2 = b N 2, b 3 = b N 3 etc equivalent definition, but with periodicity explicitly stated a b t N 1 X u=0 a u b t u mod N, t = 0, 1,..., N 1 k mod N k if 0 k N 1; if not, k mod N k + nn, where n is unique integer such that 0 k + nn N 1; thus b 0 mod N = b 0, b 1 mod N = b N 1, b 2 mod N = b N 2 etc. WMTSA: III 29

45 Convolution/Filtering of Finite Sequences: II sequence {a b t } called circular (cyclic) convolution a 0 a 1 a 7 b 0 b 7 a 6 b 1 b 2 b 6 a 2 a b 0 = P 7 u=0 a ub u mod 8 a 5 b 3 DFT of {a b t } again has a simple form (Exercise [30]): N 1 X t=0 i.e., {a b t } {A k B k } b 4 a 4 b 5 a 3 a b t e i2πf kt = A k B k ; WMTSA: III 30

46 Convolution/Filtering of Finite Sequences: II sequence {a b t } called circular (cyclic) convolution a 0 a 1 a 7 b 1 b 0 a 6 b 2 b 3 b 7 a 2 a b 1 = P 7 u=0 a ub 1 u mod 8 a 5 b 4 DFT of {a b t } again has a simple form (Exercise [30]): N 1 X t=0 i.e., {a b t } {A k B k } b 5 a 4 b 6 a 3 a b t e i2πf kt = A k B k ; WMTSA: III 30

47 Convolution/Filtering of Finite Sequences: II sequence {a b t } called circular (cyclic) convolution a 0 a 1 a 7 b 2 b 1 a 6 b 3 b 4 b 0 a 2 a b 2 = P 7 u=0 a ub 2 u mod 8 a 5 b 5 DFT of {a b t } again has a simple form (Exercise [30]): N 1 X t=0 i.e., {a b t } {A k B k } b 6 a 4 b 7 a 3 a b t e i2πf kt = A k B k ; WMTSA: III 30

48 Convolution/Filtering of Finite Sequences: II sequence {a b t } called circular (cyclic) convolution a 0 a 1 a 7 b 3 b 2 a 6 b 4 b 5 b 1 a 2 a b 3 = P 7 u=0 a ub 3 u mod 8 a 5 b 6 DFT of {a b t } again has a simple form (Exercise [30]): N 1 X t=0 i.e., {a b t } {A k B k } b 7 a 4 b 0 a 3 a b t e i2πf kt = A k B k ; WMTSA: III 30

49 Convolution/Filtering of Finite Sequences: II sequence {a b t } called circular (cyclic) convolution a 0 a 1 a 7 b 4 b 3 a 6 b 5 b 6 b 2 a 2 a b 4 = P 7 u=0 a ub 4 u mod 8 a 5 b 7 DFT of {a b t } again has a simple form (Exercise [30]): N 1 X t=0 i.e., {a b t } {A k B k } b 0 a 4 b 1 a 3 a b t e i2πf kt = A k B k ; WMTSA: III 30

50 Convolution/Filtering of Finite Sequences: II sequence {a b t } called circular (cyclic) convolution a 0 a 1 a 7 b 5 b 4 a 6 b 6 b 7 b 3 a 2 a b 5 = P 7 u=0 a ub 5 u mod 8 a 5 b 0 DFT of {a b t } again has a simple form (Exercise [30]): N 1 X t=0 i.e., {a b t } {A k B k } b 1 a 4 b 2 a 3 a b t e i2πf kt = A k B k ; WMTSA: III 30

51 Convolution/Filtering of Finite Sequences: II sequence {a b t } called circular (cyclic) convolution a 0 a 1 a 7 b 6 b 5 a 6 b 7 b 0 b 4 a 2 a b 6 = P 7 u=0 a ub 6 u mod 8 a 5 b 1 DFT of {a b t } again has a simple form (Exercise [30]): N 1 X t=0 i.e., {a b t } {A k B k } b 2 a 4 b 3 a 3 a b t e i2πf kt = A k B k ; WMTSA: III 30

52 Convolution/Filtering of Finite Sequences: II sequence {a b t } called circular (cyclic) convolution a 0 a 1 a 7 b 7 b 6 a 6 b 0 b 1 b 5 a 2 a b 7 = P 7 u=0 a ub 7 u mod 8 a 5 b 2 DFT of {a b t } again has a simple form (Exercise [30]): N 1 X t=0 i.e., {a b t } {A k B k } b 3 a 4 b 4 a 3 a b t e i2πf kt = A k B k ; WMTSA: III 30

53 Convolution/Filtering of Finite Sequences: III related concept is complex cross-correlation: a? b t N 1 X u=0 for which {a? b t } {A k B k} a ub u+t mod N t = 0, 1,..., N 1, a 0 a 1 a 7 a 2 b 0 b 7 b6 b 2 b 1 a? b 0 = P 7 a 6 u=0 a ub u b 3 b 4 b 5 a 3 a 5 a 4 WMTSA: III 31

54 Convolution/Filtering of Finite Sequences: III related concept is complex cross-correlation: a? b t N 1 X u=0 for which {a? b t } {A k B k} a ub u+t mod N t = 0, 1,..., N 1, a 0 a 1 a 7 a 2 b 1 b 0 b7 b 3 b 2 a? b 1 = P 7 a 6 u=0 a ub u+1 mod 8 b 4 b 5 b 6 a 3 a 5 a 4 WMTSA: III 31

55 Convolution/Filtering of Finite Sequences: III related concept is complex cross-correlation: a? b t N 1 X u=0 for which {a? b t } {A k B k} a ub u+t mod N t = 0, 1,..., N 1, a 0 a 1 a 7 a 2 b 2 b 1 b0 b 4 b 3 a? b 2 = P 7 a 6 u=0 a ub u+2 mod 8 b 5 b 6 b 7 a 3 a 5 a 4 WMTSA: III 31

56 Convolution/Filtering of Finite Sequences: III related concept is complex cross-correlation: a? b t N 1 X u=0 for which {a? b t } {A k B k} a ub u+t mod N t = 0, 1,..., N 1, a 0 a 1 a 7 a 2 b 3 b 2 b1 b 5 b 4 a? b 3 = P 7 a 6 u=0 a ub u+3 mod 8 b 6 b 7 b 0 a 3 a 5 a 4 WMTSA: III 31

57 Convolution/Filtering of Finite Sequences: III related concept is complex cross-correlation: a? b t N 1 X u=0 for which {a? b t } {A k B k} a ub u+t mod N t = 0, 1,..., N 1, a 0 a 1 a 7 a 2 b 4 b 3 b2 b 6 b 5 a? b 4 = P 7 a 6 u=0 a ub u+4 mod 8 b 7 b 0 b 1 a 3 a 5 a 4 WMTSA: III 31

58 Convolution/Filtering of Finite Sequences: III related concept is complex cross-correlation: a? b t N 1 X u=0 for which {a? b t } {A k B k} a ub u+t mod N t = 0, 1,..., N 1, a 0 a 1 a 7 a 2 b 5 b 4 b3 b 7 b 6 a? b 5 = P 7 a 6 u=0 a ub u+5 mod 8 b 0 b 1 b 2 a 3 a 5 a 4 WMTSA: III 31

59 Convolution/Filtering of Finite Sequences: III related concept is complex cross-correlation: a? b t N 1 X u=0 for which {a? b t } {A k B k} a ub u+t mod N t = 0, 1,..., N 1, a 0 a 1 a 7 a 2 b 6 b 5 b4 b 0 b 7 a? b 6 = P 7 a 6 u=0 a ub u+6 mod 8 b 1 b 2 b 3 a 3 a 5 a 4 WMTSA: III 31

60 Convolution/Filtering of Finite Sequences: III related concept is complex cross-correlation: a? b t N 1 X u=0 for which {a? b t } {A k B k} a ub u+t mod N t = 0, 1,..., N 1, a 0 a 1 a 7 a 2 b 7 b 6 b5 b 1 b 0 a? b 7 = P 7 a 6 u=0 a ub u+7 mod 8 b 2 b 3 b 4 a 3 a 5 a 4 WMTSA: III 31

61 Convolution/Filtering of Finite Sequences: IV with {a t } {A k }, can obtain {a? b t } by filtering {b t } with filter {A k } (Exercise [31]) flow diagram for circular filtering: {b t } {a t } {a b t } or {b t } {A k } {a b t } (latter cannot be mistaken for infinite sequence case) sometimes convenient to simplify the above to {b t } a t {a b t } or {b t } A k {a b t } or to just b t a t a b t or b t A k a b t WMTSA: III 32

62 Periodized Filters: I circular filters of length N often constructed implicitly let {b t : t = 0,..., N 1} be a finite sequence, and consider using {a t : t = 0, 1,..., M 1} to form a b t = M 1 X u=0 a u b t u mod N, t = 0,..., N 1 resembles circular filtering: input {b t }, output {a b t }, but {a t } is a sequence of width M (need not be equal to N) if M < N, can write a b t = M 1 X u=0 a u b t u mod N = N 1 X u=0 by defining a t = 0 for t = M,..., N 1 a u b t u mod N WMTSA: 32 III 33

63 Periodized Filters: II if M > N, define a t = 0 for all t M so that M 1 X X a b t = a u b t u mod N = a u b t u mod N u=0 u=0 split infinite sum into sum of sums over N terms: a b t = N 1 X u=0 a u b t u mod N + 2N 1 X u=n a u b t u mod N + rewrite each sum so that u goes from 0 to N 1: a b t = N 1 X u=0 a u b t u mod N + N 1 X u=0 a u+n b t u N mod N + WMTSA: III 34

64 Periodized Filters: III use fact that, for any integer n, to get a b t = t u nn mod N = t u mod N N 1 X u=0 a u b t u mod N + N 1 X u=0 a u+n b t u mod N + collect multipliers of b t u mod N together & call their sum a u: N 1 X X N 1 X a b t = a u+nn b t u mod N a ub t u mod N u=0 n=0 u=0 WMTSA: III 35

65 Periodized Filters: IV {a t } is {a t} periodized to length N and is formed by chopping zero-padded {a t } into finite sequences of length N: a 0, a 1,..., a {z N 1, a } N, a N+1,..., a 2N 1,... {z } block n=0 block n=1 adding finite sequences element by element: block n = 0: a 0 a 1 a N block n = 1: a N a N+1 a 2N periodized filter: a 0 a 1 a N 1 WMTSA: 33 III 36

66 Periodized Filters: V as example, let s periodize {a 0, a 1, a 2, a 3, a 4, a 5 } to length 4 extend with zeros and chop into blocks of length 4: a 0, a 1 {z, a 2, a } 3, a 4, a {z 5, 0, 0 }, 0, 0, {z 0, 0 },... block n=0 block n=1 block n=2 add blocks element by element: block n = 0: a 0 a 1 a 2 a block n = 1: a 4 a periodized filter: a 0 + a 4 a 1 + a 5 a 2 a 3 yields a 0 = a 0 + a 4, a 1 = a 1 + a 5, a 2 = a 2 and a 3 = a 3 III 37

67 Periodized Filters: VI as a second example, let s periodize a filter of width M = 46 to length N = 16, which, after padding with two zeros, goes as follows:... at, t = 0,..., 47 III 38

68 Periodized Filters: VI as a second example, let s periodize a filter of width M = 46 to length N = 16, which, after padding with two zeros, goes as follows:... at, t = 0,..., 47 block n = 0 n = 1 n = 2 III 38

69 Periodized Filters: VI as a second example, let s periodize a filter of width M = 46 to length N = 16, which, after padding with two zeros, goes as follows:... block n = 0 n = 1 n = a t, t = 0,..., 15 a t, t = 16,..., 31 a t, t = 32,..., 47 a t, t = 0,..., 47 III 38

70 Periodized Filters: VI as a second example, let s periodize a filter of width M = 46 to length N = 16, which, after padding with two zeros, goes as follows:... block n = 0 n = 1 n = a t, t = 0,..., 15 a t, t = 16,..., 31 a t, t = 32,..., 47 a t, t = 0,..., 15 a t, t = 0,..., 47 III 38

71 Periodized Filters: VII have set a t = 0 for all t M; now set a t = 0 for all t < 0 also DFT of infinite sequence {a t } given by A(f) = X a t e i2πft = M 1 X a t e i2πft t= t=0 Exercise [33]: DFT of {a t : t = 0,..., N 1} is given by {A( N k ) : k = 0,..., N 1} periodization equivalent to sampling in frequency domain result holds for M < N, M = N or M > N (and for starting values of t other than 0) WMTSA: 33 III 39

72 Periodized Filters: VIII in terms of a flow diagram, can thus express as a b t = M 1 X u=0 a u b t u mod N, t = 0,..., N 1, {b t } {A( k N )} {a b t} or {b t } A( k N ) {a b t} variation on the above: place N elements of {b t } into vector B place N elements of {a b t } into vector C can then reexpress flow diagram as B A( k N ) C above is most common form of flow diagram WMTSA: 33 III 40

73 Summary of Fourier/Filtering Theory: I {a t : t =..., 1, 0, 1,...} = {a t } has DFT X A(f) a t e i2πft inverse DFT says that a t = Z 1/2 1/2 t= A(f)e i2πft df relationship between {a t } and A( ) denoted by {a t } A( ) or, less formally, by a t A(f) WMTSA: III 41

74 Summary of Fourier/Filtering Theory: II given {a t } A( ) and {b t } B( ), their convolution X a b t a u b t u, t =..., 1, 0, 1,..., u= has a DFT given by X t= a b t e i2πft = A(f)B(f) {a b t } is output from filter with impulse response sequence {a t } and transfer function A( ) related by {a t } A( ) can express filtering operation in a flow diagram as either {b t } {a t } {a b t } or {b t } A( ) {a b t } WMTSA: III 42

75 Summary of Fourier/Filtering Theory: III {a t : t = 0, 1,..., N 1} = {a t } has DFT A k N 1 X t=0 inverse DFT says that a t e i2πf kt, with f k k N a t = 1 N N 1 X k=0 & k = 0, 1,..., N 1 A k e i2πf kt, t = 0, 1,..., N 1 relationship between {a t } and {A k } denoted by {a t } {A k } or, less formally, by a t A k WMTSA: III 43

76 Summary of Fourier/Filtering Theory: IV given {a t } & {b t } of length N with DFTs {A k } & {B k }, their circular convolution a b t N 1 X u=0 has a DFT given by a u b t u mod N, t = 0, 1,..., N 1, N 1 X t=0 a b t e i2πf kt = A k B k {a b t } is output from circular filtering operation expressed as {b t } a t {a b t } or {b t } A k {a b t } WMTSA: III 44

77 Summary of Fourier/Filtering Theory: V given {a t } of width M & {b t }, can express a b t = M 1 X u=0 a u b t u mod N, t = 0,..., N 1, as (assuming a t 0 for t < 0 and t M) a b t = N 1 X u=0 a ub t u mod N, where a u X a u+nn n= DFT of {a t } given by A( N k ), k = 0,..., N 1, where X M 1 A(f) a t e i2πft X = a t e i2πft t= t=0 WMTSA: III 45

78 Summary of Fourier/Filtering Theory: VI can represent this type of filtering operation as either {b t } A( k N ) {a b t} or B A( k N ) C where B & C are vectors of length N containing {b t } & {a b t } WMTSA: III 46

79 Summary of Fourier/Filtering Theory: VII can achieve effect of cascade with L filters {b t } A 1 ( ) A 2 ( ) A L ( ) {a b t } by using a single equivalent filter LY {b t } A( ) {a b t }, where A(f) = A l (f) similarly, effect of cascade with L circular filters B A 1 ( N k ) A 2( N k ) A L( N k ) C can be achieved using a single equivalent circular filter L B A( N k ) C, where A( N k ) = Y A l ( N k ) l=1 l=1 WMTSA: III 47

80 Do We Need t a t < for DFT to Exist?: I for real-valued infinite sequence {a t : t =..., 1, 0, 1,...}, have stated that t a2 t < is sufficient for DFT A(f) a t e i2πft to exist and to be well-defined t= note that t a2 t < does not imply t a t < might seem we need stronger condition t a t < since A(0) = a t = a t t= t= if a t 0 for all t, opening up possibility A(0) = if we only assume t a2 t < WMTSA: III Addendum 1

81 Do We Need t a t < for DFT to Exist?: II in fact, t a2 t < is sufficient, as per following argument (see, e.g., Section 1.3 of L.H. Koopmans, The Spectral Analysis of Time Series, Academic Press, 1974) let L 2 ( 1 2, 1 2 ) denote collection of all complex-valued functions A( ) such that 1/2 1/2 A(f) 2 df < (need to interpret above integral as Lebesgue integral) can regard L 2 ( 1 2, 1 2 ) as Hilbert space with inner product A( ), B( ) = 1/2 1/2 A(f)B (f) df III Addendum 2

82 Do We Need t a t < for DFT to Exist?: III can argue that E t (f) e i2πft for t = 0, ±1,... form a complete orthonormal sequence in L 2 ( 1 2, 1 2 ) hence A( ) L 2 ( 1 2, 1 2 ) if and only if there exists a sequence of complex numbers {a t, t = 0, ±1,...} with t a t 2 < such that A(f) = a t E t (f) = a t e i2πft where t= a t = A( ), E t ( ) = 1/2 1/2 t= A(f)e i2πft df III Addendum 3

83 Do We Need t a t < for DFT to Exist?: IV let l 2 be set all complex-valued sequences {a t } such that a t 2 < t= can regard l 2 as Hilbert space with inner product {a t }, {b t } = a t b t thus A(f) = t= t= a t e i2πft and a t = 1/2 1/2 A(f)e i2πft df give a one-to-one mapping (the DFT) from L 2 ( 1 2, 1 2 ) onto l 2 that can be shown to preserve inner products III Addendum 4

84 Do We Need t a t < for DFT to Exist?: V second heuristic proof (not based on Hilbert space theory) for integer m 0, let m A m (f) a t e i2πft, t= m i.e., DFT of finite sequence {a t : t = m,..., m} one-sequence Parseval s theorem says m 1/2 a 2 t = A m (f) 2 df t= m 1/2 (solution to Exercise [23a] gives rigorous proof for finite sums) III Addendum 5

85 Do We Need t a t < for DFT to Exist?: VI hence t= a 2 t = lim m m t= m a 2 t = = = 1/2 m lim 1/2 1/2 1/2 1/2 1/2 A m (f) 2 df lim m A m(f) 2 df A(f) 2 df (note: need to justify interchange of limit and integration using argument such as provided by Vitali convergence theorem) III Addendum 6

86 Do We Need t a t < for DFT to Exist?: VII hence A( ) is square-integrable over interval [ 1 2, 1 2 ] if B( ) is also square-integrable over [ 2 1, 1 2 ], Cauchy Schwarz inequality (CSI) says 1/2 2 A(f)B 1/2 1/2 (f) df A(f) 2 df B(f) 2 df 1/2 1/2 1/2 letting B(f) = e i2πft in above says that 1/2 2 A(f)e i2πft 1/2 df A(f) 2 df < 1/2 1/2 III Addendum 7

87 Do We Need t a t < for DFT to Exist?: VIII hence 1/2 is finite for all t 1/2 A(f)e i2πft df ã t final step is to argue that we must have ã t = a t for DFT A m ( ) of finite sequence {a t : t = m,..., m}, have 1/2 1/2 A m (f)e i2πft df = a t for all m t (solution to Exercise [22c] gives rigorous proof for finite sums) III Addendum 8

88 Do We Need t a t < for DFT to Exist?: IX thus, for m t, ã t a t 2 = = 1/2 1/2 1/2 1/2 1/2 1/2 m (A(f) A m (f)) e i2πft df 2 A(f) A m (f) df A(f) A m (f) 2 df a 2 u + a 2 u 0 u= u=m as m, which completes the proof 2 (using CSI) III Addendum 9

89 Do We Need t a t < for DFT to Exist?: X thus stronger condition t a t < is sufficient but not necessary for DFT to exist example of real-valued sequence for which t a t = but t a2 t < is a t = Γ(1 2 )Γ( t ) 2πΓ( t ), for which A(f) = 1 2 sin(πf) note that A(0) = since sin(0) = 0 above {a t } is autocovariance sequence for a fractionally differenced (FD) process with parameter δ = 1 4 (we ll be discussing FD processes later on) WMTSA: 284 III Addendum 10