Summary of Lecture 5

Transcription

1 Summary of Lecture 5 In lecture 5 we learnt how to pick the reproduction levels for the given thresholds. We learnt how to design MSQE optimal (Lloyd-Max) quantizers. We reviewed linear systems, linear shift invariant systems and the convolution sum. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 1

2 LSI Systems and Convolution S is a linear shift invariant system with input-output relationship H. H(A(i, j)) = H( k= l= = k= l= A(k, l)δ(i k, j l)) A(k, l)h(δ(i k, j l)) h(i, j) = H(δ(i, j)) the impulse response of the system S. H(δ(i k, j l)) = h(i k, j l) H(A(i, j)) = k= l= A(k, l)h(i k, j l) which is the the convolution sum. Everything about the LSI system S is in h(i, j). c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 2

3 Convolution B = A h: Properties: B(i, j) = k= l= A(k, l)h(i k, j l) (1) A h = h A. A δ = A. Finite extent 2 D sequences A (N 1 M 1 ), h (N 2 M 2 ): (for e.g., A(i, j) 0, 0 i N 1 1, 0 j M 1 1, etc.) C = A h is (N 1 + N 2 1) (M 1 + M 2 1). c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 3

4 Convolution and Linear Filtering B = A h A is convolved with h to produce B. A is linearly filtered with h to produce B. Of course using A h = h A we can also say: h is convolved with A to produce B. h is linearly filtered with A to produce B. We can solve many interesting image processing problems by cleverly choosing the filter h and filtering the image A. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 4

5 Example c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 5

6 Example - contd. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 6

7 The Fourier Transform of 2-D Sequences We will now review the Fourier Transform of 2-D sequences. Motivation: The convolution operation takes on a very special form in 2-D Fourier transform domain. The 2-D Fourier transform of images will reveal interesting properties that are shared by many images. This will allow us to distinguish natural images from nonimages (such as noise). We will be able to say what kind of linear filter is good for a certain processing application. The effect of sampling operations are understood more clearly in 2-D Fourier transform domain. This class will mostly discuss the required tools. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 7

8 Intuition - Orthogonal Coordinate Systems c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 8

9 Definition The 2-D Fourier Transform of a 2-D sequence A, F(A) is defined as: F(A) = F A (w 1, w 2 ) = m= n= A(m, n)e j(mw 1+nw 2 ) A can be recovered back from its transform F A (w 1, w 2 ) via the inverse 2-D Fourier Transform F 1 (A): π w 1, w 2 < π (2) A(m, n) = F 1 (A) = 1 4π F 2 π π A(w 1, w 2 )e +j(mw 1+nw 2 ) dw 1 dw 2 (3) w 1, w 2 vary in a continuum, i.e., the interval [ π, π). e j(mw 1+nw 2 ) = cos(mw 1 + nw 2 ) + jsin(mw 1 + nw 2 ). A F F A c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 9

10 Intuition - contd. Continuing with the previous intuition, consider the impulse representation of 2-D sequences: A(k, l) = and their Fourier transforms: F A (w 1, w 2 ) = m= n= m= n= A(m, n)δ(m k, n l) A(m, n)e j(mw 1+nw 2 ) These are actually the representations of the same sequence A in two orthogonal coordinate systems: The first coordinate system has basis vectors given by the δ(m k, n l). The second coordinate system has basis vectors given by the e j(mw 1+nw 2). The sums are inner or scalar products. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 10

11 Real-Complex Parts and Symmetry In general F A (w 1, w 2 ) is complex valued. Since we will be mainly be considering real 2-d sequences we can note some symmetry properties by using the inverse Fourier transform relationship. If A is real then: A(m, n) = 1 4π 2 π π F A(w 1, w 2 )e +j(mw 1+nw 2 ) dw 1 dw 2 F A (w 1, w 2 ) = F A( w 1, w 2 ) (4) F A (w 1, w 2 ) = F A ( w 1, w 2 ) (5) F A (w 1, w 2 ) = F A ( w 1, w 2 ) (6) R(F A (w 1, w 2 )) = R(F A ( w 1, w 2 )) (7) I(F A (w 1, w 2 )) = I(F A ( w 1, w 2 )) (8) c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 11

12 Periodicity F A (w 1, w 2 ) = m= n= A(m, n)e j(mw 1+nw 2 ) π w 1, w 2 < π F A (w 1, w 2 ) is periodic in w 1, w 2 with period 2π, i.e., for all integers k, l: To see this consider: F A (w 1 + k2π, w 2 + l2π) = F A (w 1, w 2 ) (9) e j(m(w 1+k2π)+n(w 2 +l2π)) = e j(mw 1+nw 2 ) e jk2π e jl2π = e j(mw 1+nw 2 ) integers k, l e j(mw 1+nw 2 ) = cos(mw 1 + nw 2 ) + jsin(mw 1 + nw 2 ). w 1, w 2 the frequencies of the periodic trigonometric functions. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 12

13 Example c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 13

14 Shifting and Modulation Shifting: F(A(m m 0, n n 0 )) = m= n= = k= l= = e j(m 0w 1 +n 0 w 2 ) A(m m 0, n n 0 ) F e j(m 0w 1 +n 0 w 2 ) F A (w 1, w 2 ). Similarly, modulation: A(m m 0, n n 0 )e j(mw 1+nw 2 ) A(k, l)e j((k+m 0)w 1 +(l+n 0 )w 2 ) k= l= A(k, l)e j(kw 1+lw 2 ) (10) e j(mw 01+mw 02 ) A(m, n) F F A (w 1 w 01, w 2 w 02 ) (11) c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 14

15 Inner Product and Energy Conservation Conservation of the inner product: m= n= = m= n= = 1 4π 2 = 1 4π 2 π π A(m, n)b (m, n) 1 A(m, n)[ 4π 2 π [ m= n= Hence, energy conservation: m= n= π π F B(w 1, w 2 )e j(mw 1+nw 2 ) dw 1 dw 2 ] A(m, n)e j(mw 1+nw 2 ) ]F B(w 1, w 2 )dw 1 dw 2 π F A(w 1, w 2 )F B(w 1, w 2 )dw 1 dw 2 (12) A(m, n) 2 = 1 4π 2 π π F A(w 1, w 2 ) 2 dw 1 dw 2 (13) c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 15

16 Convolution Let C = A B. C(m, n) = F C (w 1, w 2 ) = k= k= k= k= = k= k= A(k, l)b(m k, n l) A(k, l) = F B (w 1, w 2 ) m= n= A(k, l)f B (w 1, w 2 )e j(kw 1+lw 2 ) k= k= = F A (w 1, w 2 )F B (w 1, w 2 ) B(m k, n l)e j(mw 1+nw 2 ) A(k, l)e j(kw 1+lw 2 ) where we used the shifting property in the second step of the calculation. Thus we have the important result: A B F F A (w 1, w 2 )F B (w 1, w 2 ) (14) c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 16

17 Multiplication A dual property to convolution property can be derived for multiplication. Let C(m, n) = A(m, n)b(m, n). Thus: F(C(m, n)) = = m= n= = 1 4π 2 = 1 4π 2 π π m= n= 1 A(m, n)[ 4π 2 π F B(w 1, w 2)[ π A(m, n)b(m, n)e j(mw 1+nw 2 ) m= n= π F B(w 1, w 2)e j(mw 1 +nw 2 ) dw 1dw 2]e j(mw 1+nw 2 ) π F B(w 1, w 2)F A (w 1 w 1, w 2 w 2)dw 1dw 2 A(m, n)b(m, n) F 1 4π 2 π A(m, n)e j(m(w 1 w 1 )+n(w 2 w2 )) ]dw 1dw 2 π F B(w 1, w 2)F A (w 1 w 1, w 2 w 2)dw 1dw 2 (15) c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 17

18 Delta Functions The Fourier transform of a Kronecker delta function: F δ (w 1, w 2 ) = = 1 m= n= δ(m, n)e j(mw 1+nw 2 ) (16) The Fourier transform of A(m, n) = 1 can be found via the Dirac delta function: 1 4π 2 π A(m, n) = 1 F 4π 2 π δ(w 1, w 2 )e j(mw 1+nw 2 ) dw 1 dw 2 = 1 k= l= 4π 2 δ(w 1 k2π, w 2 l2π) (17) where δ(w 1, w 2 ) is the Dirac delta function and we used the fact that the Fourier transform has to be periodic with 2π. Note that δ(w 1, w 2 ) = 0 for w 1, w 2 0 and + + δ(w 1, w 2 )dw 1 dw 2 = 1 (18) c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 18

19 comb(m,n) Consider the Kronecker comb function comb(m, n): k= l= where S 1 > 0, S 2 > 0 are integers. δ(m ks 1, n ls 2 ) (19) comb(m, n) is very useful when discussing sampling. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 19

20 comb(m,n) - contd. The Fourier transform of a comb function can be computed as: F(comb(m, n)) = m= n= = [ m= n= k= l= = [ k= l= m= n= = k= l= = m= n= comb(m, n)e j(mw 1+nw 2 ) 1 e j(ks 1w 1 +ls 2 w 2 ) 1 e j(ms 1w 1 +ns 2 w 2 ) δ(m ks 1, n ls 2 )]e j(mw 1+nw 2 ) δ(m ks 1, n ls 2 )e j(mw 1+nw 2 ) ] (20) c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 20

21 comb(m,n) - contd. Note that Equation 20 is simply the Fourier transform of A(m, n) = 1 and hence: F(comb(m, n)) = m= n= = 4π 2 = 4π2 S 1 S 2 k= l= k= l= 1 e j(ms 1w 1 +ns 2 w 2 ) where the last line follows since for any regular function G(w 1, w 2 ): + + δ(s 1 w 1 k2π, S 2 w 2 l2π) δ(w 1 k2π S 1, w 2 l2π S 2 ) (21) δ(s 1w 1 k2π, S 2 w 2 l2π)g(w 1, w 2 )dw 1 dw 2 = 1 G(k2π/S 1, l2π/s 2 ) S 1 S = δ(w 1 k2π, w 2 l2π )G(w 1, w 2 )dw 1 dw 2 S 1 S 2 S 1 S 2 and Dirac delta functions are defined by integrals. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 21

22 Fourier Transform Types Let 0 < a 1 < b 1 < π and 0 < a 2 < b 2 < π. We will say that a Fourier transform F A (w 1, w 2 ) is low pass if F A (w 1, w 2 ) 0 when a 1 < w 1 < π and a 2 < w 2 < π. We will say that a Fourier transform F A (w 1, w 2 ) is high pass if F A (w 1, w 2 ) 0 when 0 < w 1 < a 1 and 0 < w 2 < a 2. Finally, we will say that a Fourier transform F A (w 1, w 2 ) is band pass if F A (w 1, w 2 ) 0 when 0 < w 1 < a 1, b 1 < w 1 < π and 0 < w 2 < a 2, b 2 < w 2 < π. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 22

23 Example - Low pass c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 23

24 Sampling and Aliasing Given a 2-D sequence A we would like to obtain a sequence C by sub-sampling A: where S 1, S 2 > 0 are integers. C(m, n) = A(S 1 m, S 2 n) (22) We would like C to have close resemblance to A. For example given a image we would like to obtain a image by picking every other pixel in the original image. Things may go very wrong in sampling with unexpected effects. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 24

25 Example c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 25

26 Fourier Transform of Sampled Sequence B(m, n) = A(m, n)comb(m, n) C(m, n) = B(S 1 m, S 2 n) First obtain the Fourier transform of B using the multiplication property: F B (w 1, w 2 ) = 1 4π 2 = 1 4π 2 = π k= l= π F π A(w 1, w 2)[ 4π2 S 1 S 2 1 S 1 S 2 π π F A(w 1, w 2)F comb (w 1 w 1, w 2 w 2)dw 1dw 2 δ(w 1 w 1 k2π, w 2 w 2 l2π )]dw k= l= S 1 S 1dw 2 2 F π A(w 1, w 2)δ(w 1 w 1 k2π, w 2 w 2 l2π )dw S 1 S 1dw 2 2 For w 1, w 2 {l w 2 l2π S 2 [ π, π), let K(w 1 ) = {k w 1 k2π S 1 [ π, π)} and L(w 2 ) = [ π, π)}. Then: F B (w 1, w 2 ) = 1 S 1 S 2 k K(w 1 ) l L(w 2 ) F A (w 1 k2π S 1, w 2 l2π S 2 ) (23) c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 26

27 Example Suppose S 1 = S 2 = 2, i.e., we are sub-sampling by 2. Then for w 1, w 2 ( π, π), and we have: K(w 1 ) = {k w 1 kπ ( π, π)} = { 1, 0, 1} L(w 2 ) = {l w 2 lπ ( π, π)} = { 1, 0, 1} F B (w 1, w 2 ) = 1 1 k= 1 l= 1 F A (w 1 kπ, w 2 lπ) (24) If during this process there is overlapping, i.e., say (F B (w 1, w 2 ) F A (w 1, w 2 ))F A (w 1, w 2 ) 0 then we will say that there is aliasing in the sub-sampling operation. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 27

28 Example - contd. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 28

29 F-T of Sampled Sequence - contd. Going back to the transform we were calculating: We can now obtain F C (w 1, w 2 ) F C (w 1, w 2 ) = = m= n= B(S 1 m, S 2 n)e j(mw 1+nw 2 ) m=..., S 1,0,S 1,... n=..., S 2,0,S 2,... = F B (w 1 /S 1, w 2 /S 2 ) = 1 S 1 S 2 k K(w 1 ) l L(w 2 ) B(m, n)e j(mw 1/S 1 +nw 2 /S 2 ) F A ( w 1 S 1 k2π S 1, w 2 S 2 l2π S 2 ) (25) c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 29

30 Example - contd. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 30

31 Aliasing It is clear that unless we are careful, the sampled signal can be very different from the original. For no aliasing to occur after sampling F A (w 1, w 2 ) must be: F A (w 1, w 2 ) = 0, so that there is no overlap. But what if there is? π S 1 < w 1 < π, π S 2 < w 2 < π (26) Then we have to low-pass filter A to make sure things become conformant to the above. Such a low-pass filter is called an antialiasing filter. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 31

32 Summary In this lecture we learnt the equivalence between convolution and linear filtering. We reviewed two dimensional Fourier transforms of 2-d sequences. We discussed various properties of Fourier transforms and in particular we saw that the Fourier transform converts convolution to multiplication. Using the Fourier transform properties of Kronecker and Dirac delta functions we learnt about sampling and aliasing. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 32

33 Homework VI 1. Show the modulation property of the Fourier transform. 2. Show that if a two dimensional sequence is separable then so is its Fourier transform, i.e., if A(m, n) = A 1 (m)a 2 (n) then F A (w 1, w 2 ) = F A1 (w 1 )F A2 (w 2 ) where F A1 (w 1 ), F A2 (w 2 ) are one dimensional Fourier transforms such as F a (w) = 1 a(n)e jwn. 2π n= 3. Obtain the Fourier transform of the 2-D sequence A(m, n) given by: n = m = Simplify your answer as much as possible. 4. Calculate the Fourier transform of the limited extent sequence A(m, n) = 1, 0 m < 8, 0 n < 8 and A(m, n) = 0 otherwise. 5. Calculate the Fourier transform of the limited extent sequence A(m, n) = ( 1) m+n, 0 m < 8, 0 n < 8 and A(m, n) = 0 otherwise. 6. Based on the two items above, find the Fourier transform of the checkerboard image(d(m, n)). (Hint: Assume gray=2, black=0). For sampling with S 1 = S 2 = 2 show that there is aliasing. Calculate the Fourier transform of the sub-sampled sequence C(m, n) = D(S 1 m, S 2 n). (Do this by using the F D (w 1, w 2 ). Take the inverse transform and verify with direct sub-sampling in the sequence domain. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 33

34 7. Find the periodicity of F B (w 1, w 2 ) and F C (w 1, w 2 ) as obtained in sampling and aliasing slides. Draw a figure showing both the aliasing and periodicity involved in F B (w 1, w 2 ), F C (w 1, w 2 ). Take F A (w 1, w 2 ) of the original sequence and S 1, S 2 anything you like as long as there is antialiasing. Do a better job than I did. c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 34

35 References [1] A. K. Jain, Fundamentals of Digital Image Processing. Englewood Cliffs, NJ: Prentice Hall, c Onur G. Guleryuz, Department of Electrical and Computer Engineering, Polytechnic University, Brooklyn, NY 35