Fourier series: Any periodic signals can be viewed as weighted sum. different frequencies. view frequency as an

Transcription

1 Image Enhancement in the Frequency Domain Fourier series: Any periodic signals can be viewed as weighted sum of sinusoidal signals with different frequencies Frequency Domain: view frequency as an independent variable

2 Image Enhancement in the Frequency Domain Background- From the observation: any periodic function may be written as the sum of sines and consines of varying amplitudes and frequencies f ( x ) = sin x = f2 ( x) = sin 3x 3 f ( 3 x ) = sin 5x 5 f ( x) = sin x + sin 3x + sin 5x + sin 7x f4 ( x) = sin 7x

3 Image Enhancement in the Frequency Domain The more terms of the series we have, the closer of the sum will approach the original function f ( x) = sin x + sin 3x + sin 5x + sin 7x If f(x) is a function period 2T, then we can write ( ) = 0 + nπx nπx f x a ( an cos + bn sin ) = T T where n = T a 0 ( x) dx 2 f T T T nπx an = f ( x)cos dx, n =,2,3... T T T T nπx bn = f ( x)sin dx, n =,2,3... T T T

4 Image Enhancement in the Frequency Domain The Fourier series expansion of f(x) can be expressed in complex form inπx f ( x) = cn exp( ) dx, T n= where T in x c π n = f ( x)exp( ) dx 2T T T If the function is non-periodic, we need all frequencies to represent. f ( x) = [ a( ω)cosωx + jb( ω) sinωx] where a( ω ) = f ( x)cosωxdx, π b( ω) = f ( x)sinωxdx π dω ( ) = 0 + nπx nπx f x a ( an cos + bn sin ) T T where a a b 0 n n = 2T = T = T T T T T T T n= f ( x ) dx nπx f ( x )cos dx, n =,2,3...,3... T nπx f ( x)sin dx, n =,2,3... T θ e j = cosθ + j sinθ

5 Image Enhancement in the Frequency Domain Fourier Tr. Time, spatial Frequency Domain Domain Signals Signals Inv Fourier Tr. -D, Continuous case Fourier Tr.: Inv. Fourier Tr.: F( u) f ( x) j2 πux = f ( x) e 2 dx j πux = F( u) e 2 du

6 Image Enhancement in the Frequency Domain -D, Discrete case f=,,,,-,-,-,- f = [ f, f, f,... f ] = 0 2 N F = [ F0, F, F2,... FN ] where N F u N = = x 0 exp xu 2πi N fx

7 Image Enhancement in the Frequency Domain -D, Discrete case Fourier Tr.: F ( u ) = N N x= 0 f ( x ) e j2πux / N u = 0,,N- Inv. Fourier Tr.: f ( x) = N u= 0 F( u) e j2πux / N x = 0,,N-

8 Image Enhancement in the Frequency Domain -D, Discrete case Suppose f x = [ f0, f, f2,... f N ] is a sequence of length N. we define DFT to be the sequence F F u = [ F0, F, F2,... F N ] where u N = N x= 0 exp xu 2 π i f N As a matrix multiplication: x F = If f where mn I m, n = exp 2 πi N N mn mn I m, n = cos( 2π i ) + isin( 2π i ) N N N

9 -D Discrete Fourier Transforms D Discrete case S f [ 2 3 4] th tn 4 -D, Discrete case x f N f f f f = 2 0 ],...,, [ Suppose f=[,2,3,4] so that N=4 N N u f xu i F F F F F F = = exp ],...,, [ π = I i i i i i n m, x x u f N i N F = 0 2 exp π I = f F i i i i 2 = I 2 exp, N mn i N where n m π = i i i i i F = I ) 2 sin( ) 2 cos(, N mn i i N mn i N N N n m π π + = i F i 2 2 4

10 -D Discrete Fourier Transforms -D, Discrete case using Matlab >>f=[,2,3,4] >> fft(f ) ans = i i >> ifft(ans) ans = 2 3 4

11 Image Enhancement in the Frequency Domain From Euler s formular e j θ = cos θ + j sin θ M F ( u ) = N x= 0 F(u) ) can be written as where f ( x )[cos(2 π ux Sines and Consines have various frequencies, The domain (u) called frequency domain / N ) j sin(2 π ux / N )] F ( u ) = R ( u ) + ji ( u ) F ( u ) = F ( u ) e or jφ ( u) = I( u) 2 2 R( u) I( ) φ ( u ) = tan R( u) F ( u) + u

12 Example of -D Fourier Transforms Notice that the longer the time domain signal, The shorter its Fourier transform (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2 nd Edition.

13 Relation Between Δx and Δu For a signal lf( f(x) ) with N points, it let spatial resolution Δx be space between samples in f(x) and let frequency resolution Δu be space between frequencies components in F(u), we have f ( x) f ( x0 + xδx) = Δ For x = 0,,2,... N- F ( u) F ( uδu) = Δ u = 0,,2, N- Δu u = NΔx Example: for a signal f(x) with sampling period 0.5 sec, 00 point, it we will get frequency resolution equal to Δu = = 0.02 Hz This means that in F(u) we can distinguish 2 frequencies that are apart by 0.02 Hertz or more.

14 2-Dimensional Discrete Fourier Transform 2-D DFT For an image of size MxN pixels F( u, v) = MN M N x = 0 y = 0 f ( x, y) e j2π ( ux / M + vy / N ) u = frequency in x direction, u = 0,, M- v = frequency in y direction, v = 0,, N- 2-D IDFT f ( x, y) = M N j2π ( ux / M + vy / N ) F( u, v) e u= 0 v= 0 x = 0,, M- y = 0,, N-

15 2-Dimensional i Discrete Fourier Transform (cont.) F(u,v) can be written as F ( u, v) = R( u, v) + ji( u, v) where or F( u, v) = F( u, v) e jφ ( u, v) 2 2 I ( u, v ) = R ( u, v ) I ( u, ) φ ( u, v ) = tan R( u, v) F ( u, v ) + v For the purpose of viewing, we usually display only the Magnitude part of F(u,v)

16 Relation Between Spatial and Frequency Resolutions where Δu = MΔx Δv = NΔy Δx = spatial resolution in x direction Δy = spatial resolution in y direction ( Δx and Δy are pixel width and height. ) Δu = frequency resolution in x direction Δv = frequency resolution in y direction N,M = image width and height

17 Displaying transforms Having obtained the Fourier transform F(u) of a signal f(x), what does it look like? Having obtained the Fourier transform F(u,v) of an image f(x,y), what does it look like? F(u,v) are complex number, we can t view them directly, but we view their magnitude F(u,v). We have 3 approaches. Find the maximum value m of F(u,v) ( DC term), and use imshow to view F(u,v) /m 2. use mat2gray to view F(u,v) (,) directly 3. use imshow to view log(+ F(u,v) )

18 Periodicity of -D DFT From DFT: F( u) = N M x=00 f ( x) e j2πux / N -N 0 N 2N DFT repeats itself every N points (Period = N) but we usually We display only in this range DFT repeats itself every N points (Period N) but we usually display it for n = 0,, N-

19 Conventional Display for -D DFT f(x) DFT F(u) 0 N- 0 N- Time Domain Signal High frequency area Low frequency area The graph F(u) is not The graph F(u) is not easy to understand!

20 Conventional Display for DFT : FFT Shift F(u) 0 N- FFT Shift: Shift center of the graph F(u) to 0 to get better Display which is easier to understand. F(u) High frequency area Low frequency area -N/2 0 N/2-

21 Periodicity of 2-D DFT 2-D DFT: F( u, v) = MN M N j2π ( ux / M + vy / N ) x= 0 y= 0 -M f ( x, y) e g(x,y) -N 0 N 2N 0 M 2M For an image of size NxM pixels, its 2-D DFT repeats itself every N points in x- direction and every M points in y-direction. We display only in this range (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2 nd Edition.

22 Conventional Display for 2-D DFT F(u,v) has low frequency areas at corners of the image while high frequency areas are at the center of the image which is inconvenient to interpret. High frequency area Low frequency area

23 2-D FFT Shift : Better Display of 2-D DFT 2-D FFT Shift is a MATLAB function: Shift the zero frequency of F(u,v) v)tothecenterofanimage the of image. 2D FFTSHIFT (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2 nd Edition. High frequency area Low frequency area

24 2-D FFT Shift (cont.) : How it works -M 0 M Display of 2D DFT After FFT Shift -N 0 N 2N 2M Original display of 2D DFT

25 Example of DFT Fourier transforms in Matlab The relevant Matlab functions for us are: fft which takes the DFT of a vector, ifft which h takes the inverse DFT of a vector, fft2 which takes the DFT of a matrix, ifft2 which takes the inverse DFT of a matrix, fftshift which shifts a transform.

26 Example of DFT Example Note that the DC coecient is indeed the sum of all the Note that the DC coecient is indeed the sum of all the matrix values.

27 Example of DFT Example 2

28 Example 3 Chapter 4 Example of DFT

29 Fourier transforms of images Example >> a=[zeros(256,28) ones(256,28)]; >> af=fftshift(fft2(a)); >> fm = max(af(:)); >> imshow(im2uint8(af/fm)); (

30 Example 2 >> a=zeros(256,256); >> a(78:78,78:78)=; >> imshow(a) () >> af=fftshift(fft2(a)); >> figure,fftshow(af,'abs'), Chapter 4 Fourier transforms of images

31 Example 3 Chapter 4 Fourier transforms of images >> [x,y]=meshgrid(:256,:256); >> b=(x+y<329)&(x+y>82)&(x-y>-67)&(x-y<73); 73) >> imshow(b) >> bf=fftshift(fft2(b)); fftshift(fft2(b)); >> fm = max(bf(:)); >> imshow(im2uint8(bf/fm));

32 Example 4 Chapter 4 Fourier transforms of images >> [x,y]=meshgrid(-28:27,-28:27); >> z=sqrt(x.^2+y.^2); >> c=(z<5); >> imshow(c); >> cf=fft2shift(fft2(c)); >> figure,fftshow(af,'abs')

33 Example of 2-D DFT Notice that the longer the time domain signal Notice that the longer the time domain signal, The shorter its Fourier transform

34 Example of 2-D DFT Notice that direction of an object in spatial image and Its Fourier transform are orthogonal to each other.

35 Example of 2-D DFT 2D DFT Original image 2D FFT Shift

36 Example of 2-D DFT f = zeros(30,30); f(5:24,3:7) = ; figure;imshow(f,'notruesize'); F = fft2(f); () F2 = log(abs(f)+); figure;imshow(f2,[- 5],'notruesize'); F3=log(abs(fftshift(F))+); (fft figure;imshow(f3,[- 5],'notruesize');

37 Example of 2-D DFT 2D DFT Original image 2D FFT Shift

38 Basic Concept of Filtering in the Frequency Domain From Fourier Transform Property: g ( x, y) = f ( x, y) h( x, y) F( u, v) H ( u, v) = G( u, v) We cam perform filtering process by using M ltiplication in the freq enc domain Multiplication in the frequency domain is easier than convolution in the spatial Domain.

39 Filtering in the Frequency Domain with FFT shift F(u,v) H(u,v) (User defined) g(x,y) FFT shift X 2D IFFT 2D FFT FFT shift f(x,y) G(u,v) In this case, F(u,v) and H(u,v) must have the same size and have the zero frequency at the center.

40 Multiplication in Freq. Domain = Circular Convolution f(x) ) DFT F(u) ) G(u) = F(u)H(u) IDFT g(x) h(x) DFT H(u) Multiplication of DFTs of 2 signals 0.5 f(x) ) is equivalent to 0 perform circular convolution in the spatial domain h(x) ) Wrap around effect g(x)

41 convolution_discrete_b.swf

42

43 Applications of the Fourier Transform Locating Image Features The Fourier transform can also be used to perform correlation, which h is closely related to convolution. Correlation can be used to locate features within an image; in this context correlation is often called template matching. bw = imread('text.png'); a = bw(32:45,88:98); C = real(ifft2(fft2(bw) 2(b.* fft2(rot90(a,2),256,256))); figure, imshow(c,[]); max(c(:)) ans = thresh = 60; figure, imshow(c > thresh)

44 Multiplication in Freq. Domain = Circular Convolution Original image H(u,v) Gaussian Lowpass Filter with D0 = 5 Filtered image (obtained using circular convolution) Incorrect areas at image rims

45 Filtering i in the Frequency Domain : Example In this example, we set F(0,0) to zero which means that the zero frequency component tis removed. Nt Z f Note: Zero frequency = average intensity of an image

46 Filtering in the Frequency Domain : Example Lowpass Filter Highpass Filter (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2 nd Edition.

47 Filtering in the Frequency Domain : Example (cont.) Result of Sharpening Filter

48 Filter Masks and Their Fourier Transforms (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2 nd Edition.

49 Ideal Lowpass Filter Ideal Lowpass Filter Ideal LPF Filter Transfer function > = 0 0 ), ( 0 ), ( ), ( D v u D D v u D v u H where D(u,v) = Distance from (u,v) to the center of the mask.

50 Examples of Ideal Lowpass Filters The smaller D 0, the more high frequency components are removed.

51 Results of Ideal Lowpass Filters Ringing effect can be obviously seen! (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2 nd Edition.

52 How ringing effect happens H ( u, v) = 0 D( u, v) D( u, v) > D D 0 0 Surface Plot Ideal Lowpass Filter with D 0 = 5 Abrupt change in the amplitude

53 How ringing effect happens (cont.) x 0-3 Surface Plot 5 Spatial Response of Ideal Lowpass Filter with D 0 = Ripples that cause ringing effect

54 How ringing g effect happens (cont.)

55 Butterworth Lowpass Filter Transfer function H ( u, v) = 2 + [ D( u, v) / D ] N Where D 0 = Cut off frequency, N = filter order. 0

56 Results of Butterworth Lowpass Filters There is less ringing effect compared to those of ideal lowpass filters!

57 Spatial Masks of the Butterworth Lowpass Filters Some ripples can be seen.

58 Gaussian Lowpass Filter Transfer function H ( u, v ) = e Where D 0 = spread factor. D 2 ( u, v) / 2D 2 0 (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2 nd Edition. Note: the Gaussian filter is the only filter that has no ripple and hence no ringing effect.

59 Gaussian Lowpass Filter (cont.) H ( u, v) = e D 2 ( u, v) / 2D 0.8 Gaussian lowpass 0.6 filter with D 0 = Spatial respones of the Gaussian lowpass filter with D 0 = Gaussian shape

60 Results of Gaussian Lowpass Filters No ringing effect!

61 Application of Gaussian Lowpass Filters Original image Better Looking Th GLPF b d t j d d The GLPF can be used to remove jagged edges and repair broken characters.

62 Application of Gaussian Lowpass Filters (cont.) Remove wrinkles Original image Softer-Looking

63 Application of Gaussian Lowpass Filters (cont.) Original image : The gulf of Mexico and Filtered image Florida from NOAA satellite. (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2 nd Edition. Remove artifact lines: this is a simple but crude way to do it!

64 Highpass Filters H hp = - H lp

65 Ideal Highpass Filters Ideal Highpass Filters Ideal Highpass Filters Ideal Highpass Filters = 0 ), ( 0 ) ( D v u D v u H Ideal HPF Filter Transfer function > = 0 ), ( ), ( D v u D v u H where D(u,v) = Distance from (u,v) to the center of the mask. (, ) (, )

66 Butterworth Highpass Filters Transfer function H ( u, v ) = ) [ D / D( u, v ] N Where D 0 = Cut off frequency, N = filter order.

67 Gaussian Highpass Filters Transfer function H ( u, v) Where D 0 = spread factor. = e D 2 ( u, v) / 2D 2 0

68 Gaussian Highpass Filters (cont.) H ( u, v) = e D 2 ( u, v) / 2D Gaussian highpass 0.4 filter with D 0 = Spatial respones of the Gaussian highpass filter with D 0 =

69 Spatial Responses of Highpass Filters Ripples

70 Results of Ideal Highpass Filters Ringing effect can be obviously seen!

71 Results of Butterworth th Highpass Filters

72 Results of Gaussian Highpass Filters

73 Laplacian Filter in the Frequency Domain From Fourier Tr. Property: n d f ( x) n ( ju ) F(u( ) dx n Then for Laplacian operator We get f f ( u v ) F( u, v ) 2 2 f = + x y ( 2 2 u + v ) Image of (u 2 +v 2 ) Surface plot

74 Laplacian Filter in the Frequency Domain (cont.) Spatial response of (u 2 +v 2 ) Cross section Laplacian mask in Chapter 3

75 Sharpening Filtering in the Frequency Domain Sharpening Filtering in the Frequency Domain Sharpening Filtering in the Frequency Domain Sharpening Filtering in the Frequency Domain Spatial Domain ), ( ), ( ), ( y x f y x f y x f lp hp = ), ( ), ( ), ( y x f y x Af y x f lp hb = ) ( ) ( ) ( ) ( ) ( y x f y x f y x f A y x f + ), ( ), ( ), ( ) ( ), ( y x f y x f y x f A y x f lp hb + = ), ( ), ( ) ( ), ( y x f y x f A y x f hp hb + = Frequency Domain Filter ), ( ), ( ) ( ), ( y f y f y f hp hb q y ), ( ), ( v u H v u H lp hp = ), ( ) ( ), ( v u H A v u H hp hb + =

76 Sharpening Filtering in the Frequency Domain (cont.) p 2 P 2 P P 2 P

77 Sharpening Filtering in the Frequency Domain (cont.) f ( x, y ) = ( A ) f ( x, y ) f ( x, y ) hb + hp f f 2 hp = P A = 2 A = 2.7

78 Original High Frequency Emphasis Filtering H ( u, v ) = a bh ( u, v ) hfe( + hp Butterworth highpass gpass filtered image High freq. emphasis filtered image After Hist Eq. a = 0.5, b = 2

79 Homomorphic Filtering An image can be expressed as i(x,y) = illumination component r(x,y) = reflectance component f ( x, y) = i( x, y) r( x, y) We need to suppress effect of illumination that cause image Intensity changed slowly.

80 Homomorphic Filtering

81 Homomorphic Filtering i More details in the room can be seen!

82 Retinex Filter j c I k c Ik + The Relative reflectance of pixel i to j: c c I + k R ( i, j ) = δ log c k I k i = Average Relative Reflectance at i: R c j = ( i) N c R ( i, j ) = N c Ik + if log Threshold > { c δ I k c I k + 0 if log < Threshold c I I k j= j=2 j=4 i j=3 j=5

83

84

85 How to Perform 2-D DFT by Using -D DFT -D DFT by row f(x,y) by row F(u,y) -D DFT by column F(u,v)

86 How to Perform 2-D DFT by Using -D DFT(cont.) Alternative method f(x,y) -D DFT by column F(x,v) -D DFT by row F(u,v)

87 Linear Convolution by using Circular Convolution and Zero Padding f(x) ) Zero padding DFT F(u) ) h(x) Zero padding DFT H(u) G(u) = F(u)H(u) IDFT Concatenation g(x) Padding zeros Before DFT Keep only this part

88 Linear Convolution by using Circular Convolution and Zero Padding The discrete Fourier transform assumes a digital image exists on a closed surface, a torus.

89 Linear Convolution by using Circular Convolution and Zero Padding

90 Linear Convolution by using Circular Convolution and Zero Padding Filtered image Zero padding area in the spatial Domain of the mask image (the ideal lowpass filter) Only this area is kept.

91 Correlation Application: Object Detection

92 2-D DFT Properties

93 2-D DFT Properties (cont.)

94 2-D DFT Properties (cont.)

95 2-D DFT Properties (cont.)

96 Computational ti Advantage of FFT Compared to DFT

97 Image Enhancement in the Frequency Domain

98 Image Enhancement in the Frequency Domain

99 Image Enhancement in the Frequency Domain

100 Image Enhancement in the Frequency Domain

101 Image Enhancement in the Frequency Domain

102 Image Enhancement in the Frequency Domain

103 Image Enhancement in the Frequency Domain

104 Image Enhancement in the Frequency Domain

105 Image Enhancement in the Frequency Domain

106 Image Enhancement in the Frequency Domain

107 Image Enhancement in the Frequency Domain

108 Image Enhancement in the Frequency Domain

109 Image Enhancement in the Frequency Domain

110 Image Enhancement in the Frequency Domain

111 Image Enhancement in the Frequency Domain

112 Image Enhancement in the Frequency Domain

113 Image Enhancement in the Frequency Domain

114 Image Enhancement in the Frequency Domain

115 Image Enhancement in the Frequency Domain

116 Image Enhancement in the Frequency Domain

117 Image Enhancement in the Frequency Domain

118 Image Enhancement in the Frequency Domain

119 Image Enhancement in the Frequency Domain

120 Image Enhancement in the Frequency Domain

121 Image Enhancement in the Frequency Domain

122 Image Enhancement in the Frequency Domain

123 Image Enhancement in the Frequency Domain

124 Image Enhancement in the Frequency Domain

125 Image Enhancement in the Frequency Domain

126 Image Enhancement in the Frequency Domain

127 Image Enhancement in the Frequency Domain

128 Image Enhancement in the Frequency Domain

129 Image Enhancement in the Frequency Domain

130 Image Enhancement in the Frequency Domain

131 Image Enhancement in the Frequency Domain

132 Image Enhancement in the Frequency Domain

133 Image Enhancement in the Frequency Domain

134 Image Enhancement in the Frequency Domain

135 Image Enhancement in the Frequency Domain

136 Image Enhancement in the Frequency Domain

137 Image Enhancement in the Frequency Domain

138 Image Enhancement in the Frequency Domain

139 Image Enhancement in the Frequency Domain