Lecture 4 Filtering in the Frequency Domain. Lin ZHANG, PhD School of Software Engineering Tongji University Spring PDF Free Download

Lecture 4 Filtering in the Frequency Domain Lin ZHANG, PhD School of Software Engineering Tongji University Spring 2016

Outline Background From Fourier series to Fourier transform Properties of the Fourier transform Discrete Fourier transforms The basics of filtering in the frequency domain Image smoothing using frequency domain filters Image sharpening using frequency domain filters

Background Fourier analysis (Fourier series and Fourier transforms) is quite useful in many engineering fields Linear image filtering can be performed in the frequency domain A working knowledge of the Fourier analysis can help us have a thorough understanding of the image filtering

Background Jean Baptiste Joseph Fourier was born in 1768, in France Most famous for his work La Théorie Analitique de la Chaleur published in 1822 Translated into English in 1878: The Analytic Theory of Heat 21 March 1768 16 May 1830

Fourier Series For any periodic function f(t), how to extract the component of f at a specific frequency? is composed of the following components

Fourier Series For any periodic function f(t), how to extract the component of f at a specific frequency?

Fourier Series For any periodic function f(t), how to extract the component of f at a specific frequency? Fourier Series Any periodic function can be expressed as a sum of sines and cosines of different frequencies each multiplied by a different coefficient a0 f () t ancosntbnsinn t 2 n1 more details

Fourier Series For a periodic function f () t, with period T Fourier Series where Redundant! 2 a f() t dt, 0 T 2 T T 2 a f t a n t b nt 0 () ncos nsin 2 n1 2 T 2 an f ()cos t ntdt T T 2 T 2 T 2 T 2 2 bn f ()sin t ntdt T

Fourier Transforms Fourier transform of f(t) (maybe is not periodic) is defined as j2t F( ) f( t) e dt Inverse Fourier transform 2 f () t F( ) e j t d How to get these formulas? Let s start the story from Fourier series to Fourier transform

From Fourier Series to Fourier Transforms j According to Euler formula e cos jsin Easy to have e e e e cos nt,sin nt j 2 2 Then, Fourier series become a0 f() t ancosntbnsinn t 2 jnt jnt jnt jnt n1

From Fourier Series to Fourier Transforms j According to Euler formula e cos jsin Easy to have e e e e cos nt,sin nt j 2 2 Then, Fourier series become Then, let Then, jnt jnt jnt jnt a e e e e jnt jnt jnt jnt 0 f() t an jbn 2 n1 2 2 a a jb a jb e a0 an jbn an jbn c0, cn, d 2 2 2 0 n n jnt n n e 2 n1 2 2 0 n jnt n1 jnt jnt f() t c c e d e (1) n n

c 0 From Fourier Series to Fourier Transforms 1 T T 2 T 2 f() t dt, 1 1 jnt cn f() t cosn t jsin n t dt f() t e dt (2) T T T 2 2 T T 2 T 2 1 1 jnt dn f() t cosn t jsin n t dt f() t e dt T T T 2 2 T T 2 T 2 We can see that Thus, d n c n 1 jnt jnt jnt de n cne ce n n1 n1 n (3)

From Fourier Series to Fourier Transforms 0 0 0 jnt jnt f( t) c c e d e (according to (1)) n1 n j0t jnt jnt n n n1 n1 ce ce de 1 j0t jnt jnt n n n1 n ce ce ce jnt ce n,where c n is defined by (2) n This is the Fourier series in complex form How about a non-periodic function? n (according to (3))

From Fourier Series to Fourier Transforms f () t is a non periodic function () t We make a new function which is periodic and the period T is T ft () t f(), t if t [ T /2, T /2] T f () t f () t If, becomes According to Fourier series Let T T jnt 2 T() n, n T T() n T 2 sn n f 1 jnt f t c e c f te dt f t 1 f te dt e 1 f te dt e T T 2 jsnt jsnt 2 jsnt jsnt T() T T() T T() n T 2 T n 2

From Fourier Series to Fourier Transforms T 1 2 jsnt jsnt ft() t T ft() te dte T n 2 when T T 1 2 jsnt n f ( t) lim ft( t) lim T ft( te ) dte T T T n 2 2 2 s sn sn 1 T T s T 2 () lim T T () s 0 2 n 2 jsnt f t f te dt e T 1 lim 2 T T ( ) s 0 n 2 2 s jsnt jsnt f te dt e s js t n js t

From Fourier Series to Fourier Transforms T 1 2 jsnt jsnt f () t lim T ft () te dte s 2 s 0 n 2 when T ( s0) sn s s ds,, 1 jst jst f () t f() t e dt e ds 2 Denote by F() s jst F() s f() t e dt 1 jst f () t F() s e ds 2 Fourier transform Inverse Fourier transform

From Fourier Series to Fourier Transforms jst F() s f() t e dt 1 jst f () t F() s e ds 2 s here actually is the angular frequency In the signal processing domain, we usually use another form by substituting s by s 2, where is the frequency (measured by Herz) j2t F( ) f( te ) dt j2t f () t F( ) e d

Related Concepts to Fourier Transform Fourier transform F( ) is complex in general F( ) f ()cos(2 t t) dt j f ()sin(2 t t) dt R( ) ji( ) j ( ) F( ) F( ) e where 2 2 1/2 I( ) F( ) ( R ( ) I ( )), ( u) atan2 R ( ) Fourier Spectrum Phase Angle In polar form, it can be expressed as 2 2 2 P( ) F( ) R ( ) I ( ) is called the power spectrum

Related Concepts to Fourier Transform Implementation Tips 1) For computing the image s Fourier transform, you can use fft2() 2) ifft2() can compute the inverse Fourier transform 3) abs() can compute the Fourier spectrum 4) angle() can compute the phase angle

Symmetry Properties of the Fourier Transform Real f(t) Fourier transform F( ) Symmetry of F( ) general complex conjugate symmetric F * ( ) F ( ) even f () t f ( t) only real even F( ) F( ) odd f () t f ( t) only imaginary odd F( ) F( ) Proof?

Impulse Function Impulse (Dirac) function Considered as an infinitely high, infinitely thin spike at the origin, with total area one under the spike It physically represents an idealized point mass or point charge Paul Adrien Maurice Dirac (Aug. 08, 1902 Oct. 20, 1984)

Impulse Function Impulse (Dirac) function () t 0, t 0 Definition 1: () tdt1 Definition 2: f () t () t dt f(0) Sift property: () t f () t ( tt ) dt f( t ) 0 0 f(t) t 0 t Cannot be computed using normal integral methods!

Impulse Function Impulse (Dirac) function ( x) lim a0 a 1 e x a 2 2 The Dirac delta function can be seen as the limit of the sequence of zero centered normal distributions

Impulse Function The Fourier transform of function Proof: () t 1 j2t j2t j20 F = ( t) e dt e e 1 t0 Similarly, we have Why? j2t0 ( tt ) e 0

Fourier Transform of f(t) = 1 The Fourier transform of the function 1 is Proof: 1 2 2 () j t j t ( ) 01 f t e d e

Fourier Transform of The Fourier transform of the function is jat a e 2 Proof: e jat a f t e d e e 2 2 = j t j t jat a 2 2 e jat

Fourier Transform of The Fourier transform of the function is a a 2 2 sin at 2 j Proof: sin at sin at jat jat e cosat j sin at, e cosat j sinat sin at e jat e 2 j jat

Fourier Transform of sin at sin at The Fourier transform of the function is a a 2 2 sin at 2 j Proof: jat jat e e j2t F( ) e dt 2 j 1 2 j jat j2t jat j2t e e dt e e dt a a 2 2 2 j

Fourier Transform of cosat cosat The Fourier transform of the function is a a 2 2 cos at 2 Can you work it out?

Why Study Fourier Transform? Observe the image in the frequency domain; has some related applications, e.g., de noising and phase based image matching; directly manipulating the image in the frequency domain We can make use of Fourier transform to compute the convolution efficiently; thanks to FFT The underlying theory is the convolution theorem!

Convolution Theorem Still remember the convolution? (Lecture 3) For 1D continuous case, it is defined as Convolution theorem f ()* t h() t f( ) h( t) d The Fourier transform of a convolution is the point wise product of Fourier transforms if then Proof: f() t F( ), ( h()) t H( ) f()* t h() t F( ) H( )

Convolution Theorem ()* () ( ) ( ) j2t f t h t f h t d e dt j2 t f( ) h( t ) e dt d (Let x t ) j2 ( x ) f( ) h( x) e dx d j2x j2 f( ) h( x) e dx e d j2 f( ) H( ) e d j2 ( ) ( ) ( ) ( ) H f e d H F

Revisit Gaussian Filter In spatial domain 1 x y Gxy (, ) exp 2 2 2 2 2 2 In frequency domain ( uv, ) exp exp 2 1 2 2 2 2 2 2 ( u v ) ( u v ) 2 The Fourier transform of a Gaussian function is also of a Gaussian shape in the frequency domain

Revisit Gaussian Filter Gaussian filter is a low pass filter Consider the 1D case f ( x) if filtered by 2 1 x gx ( ) exp 2 2 2 What will happen to the frequency components of f(x)? FT f ( x)* g( x) F( ) G( ) F( ) exp 1 0.8 0.6 F( ) 0.4 magnitude 2 1 2 2 0.2 0 w -4-2 0 2 4

Revisit Gaussian Filter original smoothed (5x5 Gaussian) Why does this work? smoothed original

Discrete Fourier Transform (DFT) in 1D Case Given a discrete sequence with M points f [ f0 f1,..., fm 1] Regard it as a periodic signal, thus its basis frequency is For its frequency components, the frequencies are, 1 2 M 1,,... 1,2,..., M M M M M Its DFT is computed as M 1 x0 1 j2 ux M Fu ( ) f( xe ), u1,2,..., M Usually, we write it as, M 1 x0 1 j2 ux M Fu ( ) f( xe ), u0,1,2,..., M1 1 M

Discrete Fourier Transform (DFT) in 1D Case Thus, f s DFT also has M points F [ F0 F1,..., FM 1] and M 1 j2 ux/ M Fu ( ) f( xe ), u0,1,2,..., M1 x0 IDFT 1 f x F u e x M M 1 j2 ux/ M ( ) ( ), 0,1,2,..., 1 M u0 For DFT, there is a fast algorithm for computation, FFT (Fast Fourier Transform)

Discrete Fourier Transform (DFT) in 2D Case In continuous case j2 ( uxvy) Fuv (, ) f( xye, ) dxdy j2 ( uxvy) f ( x, y) F( u, v) e dudv In discrete case M1N1 j2 ( ux/ Mvy/ N) (, ) f( xye, ), Fuv x0 y0 where u 0,1,..., M 1; v 0,1,..., N 1 1 MN M1N1 j2 ( ux/ Mvy/ N) (, ) (, ), f x y u0 v0 F u v e where x 0,1,..., M 1; y 0,1,..., N 1

Some Notes on DFT Visualization For DFT, the origin is not at the center of the matrix Assume the original spectrum is divided into four quadrants; the small gray filled squares in the corners represent positions of low frequencies Due to the symmetries of the spectrum the quadrant positions can be swapped diagonally and the low frequencies locations appear in the middle of the image original spectrum low frequencies in corners shifted spectrum with the origin at (M/2, N/2)

Some Notes on DFT Visualization h h l l h h l l

Some Notes on DFT Visualization For visualization, we usually rearrange the DFT matrix to make its low frequencies at the center of the rectangle; it equals to (, )( 1) x f x y y x y f( x, y)( 1) F( um /2, vn /2) Implementation Tips In Matlab, it can simply implemented by using fftshift

DFT Visualization Samples Since each field of the Fourier transform is a complex number, we cannot show Fourier map in a single figure; instead, magnitude and phase maps are shown separately im = imread('im.bmp'); figure; imshow(im,[]); imfft = abs(fft2(im)); imfftlog = log10(1+imfft); figure; imshow(imfftlog,[]); imfftshifted = fftshift(imfftlog); figure; imshow(imfftshifted,[]);

DFT Visualization Samples An example Original image Spectrum without using fftshift Spectrum using fftshift

DFT Visualization Samples DFT DFT Any relationship?

The Basics of Filtering in the Frequency Domain To filter an image in the frequency domain: 1. Compute F(u,v), the DFT of the image 2. Multiply F(u,v) by a filter function H(u,v) 3. Compute the inverse DFT of the result

Directly Filtering in the Frequency Domain 1. Given an image f(x, y) of size M N, set P = 2M and Q = 2N 2. Form a padded image, f ( p x, y ) of size P Q by appending the necessary number of zeros to f(x, y) 3. Multiply f ( by ( 1) x y p x, y ) to center its transform 4. Compute F(u, v) of f ( p x, y ) 5. Generate a filter function H(u, v) of the size P Q 6. Get the modified Fourier transform Guv (, ) FuvHuv (, ) (, ) 7. Obtain the processed image 1 (, ) ( (, ))( 1) x g y p x y G u v 8. Obtain the final result g(x, y) by extracting the M N region from the top, left corner of g (, ) p x y

Directly Filtering in the Frequency Domain Example f ( x, y ) f p( x, y ) F( uv, ) H ( uv, ) gxy (, ) g (, ) p x y Guv (, ) FuvHuv (, ) (, ) Source codes are available on our course website

Convolution via Fourier Transform 1. Given an image f(x, y) of size A B, and a spatial filter h(x, y) of size C D ; set P >= A+C-1 and Q >=B+D-1 2. Form a padded image f p of size P Q by appending the necessary number of zeros to f(x, y); form a padded filter h p of size P Q in a similar way 3. Compute the DFT F(u, v) of the image, and H(u, v) of the filter 4. Get the modified Fourier transform Guv (, ) FuvHuv (, ) (, ) 5. Obtain the processed image (, ) 1 gp x y ( G ( u, v )) 6. Obtain the final result g(x, y) by extracting the central A B region from g (, ) p x y

Convolution via Fourier Transform Example f ( xy, ) f ( x, y) p F( uv, ) Guv (, ) g (, ) p x y g( x, y) hxy (, ) h (, ) p x y H ( uv, ) Source codes are available on our course website

Some Tips on Filtering via Fourier Transform When the filter kernel is small, it s better to implement the filtering in the spatial domain; otherwise, you can realize the filtering via the Fourier transform In practice, when padding the images or filters, it s better to make it has a size which is the power of 2; this criterion is based on the computer architecture

Smoothing is Low Pass Filtering Image smoothing actually is performing a low pass filtering to the image Edges and other sharp intensity transitions, such as noise, in an image contribute significantly to the high frequency content of its Fourier transform Three commonly used low pass filtering techniques Ideal low pass filters Butterworth low pass filters Gaussian low pass filters

Ideal Low Pass Filter Simply cut off all high frequency components that are within a specified distance D 0 from the origin of the transform Its drawback is that the filtering result has obvious ringing artifacts Ideal low pass filter is rarely used in practice

Ideal Low Pass Filter The transfer function for the ideal low pass filter can be given as: H ( u, v) 1 0 if if D( u, v) D D( u, v) D where D(u,v) is the distance of (u, v) to the frequency centre (0, 0) and it is given as: Duv (, ) [ u v] 0 0 2 2 1/2

Ideal Low Pass Filter Above we show an image, it s Fourier spectrum and a series of ideal low pass filters of radius 5, 15, 30, 80 and 230 superimposed on top of it

Ideal Low Pass Filter Original image Result of filtering with ideal low pass filter of radius 5 Result of filtering with ideal low pass filter of radius 15 Result of filtering with ideal low pass filter of radius 30 Result of filtering with ideal low pass filter of radius 80 Result of filtering with ideal low pass filter of radius 230

Butterworth Low pass Filters It was proposed by the British engineer Stephen Butterworth Filter order can change the shape of the Butterworth filter; for high order values, the Butterworth filter approaches the ideal filter; for low order values, it approaches the Gaussian filter

Butterworth Low pass Filters The transfer function of a Butterworth low pass filter of order n with cutoff frequency at distance D 0 from the origin is defined as: 1 1 H ( u, v) 2n [ D ( u, v ) / D ] 0

Butterworth Low pass Filters Original image Result of filtering with Butterworth filter of order 2 and cutoff radius 5 Result of filtering with Butterworth filter of order 2 and cutoff radius 15 Result of filtering with Butterworth filter of order 2 and cutoff radius 30 Result of filtering with Butterworth filter of order 2 and cutoff radius 80 Result of filtering with Butterworth filter of order 2 and cutoff radius 230

Butterworth Low pass Filters Original image Filtering result of Butterworth Source codes are available on course website

Gaussian Low pass Filters The transfer function of a Gaussian lowpass filter is defined as: H ( u, v) e D 2 ( u, v)/ 2D where D(u,v) is the distance from the center of the frequency rectangle 2 0

Gaussian Lowpass Filters Original image Result of filtering with Gaussian filter with cutoff radius 5 Result of filtering with Gaussian filter with cutoff radius 15 Result of filtering with Gaussian filter with cutoff radius 30 Result of filtering with Gaussian filter with cutoff radius 85 Result of filtering with Gaussian filter with cutoff radius 230

Low pass Filtering Examples A low pass Gaussian filter is used to connect broken text

Low pass Filtering Examples Gaussian filters used to remove blemishes in a photograph for publishing

Sharpening in the Frequency Domain Edges and fine detail in images are associated with high frequency components High pass filters only pass the high frequencies, drop the low ones High pass filters are precisely the reverse of low pass filters, so, H 1 H ( u, v) HP LP

Ideal High Pass Filters The ideal high pass filter is given as: 0 if D( u, v) D H ( u, v) 1 if D( u, v) D where D 0 is the cut off distance as before 0 0

Ideal High Pass Filters Results of ideal high pass filtering with D 0 = 15 Results of ideal high pass filtering with D 0 = 30 Results of ideal high pass filtering with D 0 = 80

Butterworth High Pass Filters The Butterworth high pass filter is given as: 1 H ( u, v) 2n 1[ D0 / D( u, v)] where n is the order and D 0 is the cut off distance as before

Butterworth High Pass Filters Results of Butterworth high pass filtering of order 2 with D 0 = 15 Results of Butterworth high pass filtering of order 2 with D 0 = 30 Results of Butterworth high pass filtering of order 2 with D 0 = 80

Butterworth High Pass Filters Original image Filtering result of Butterworth high pass filtering Source codes are available on course website

Gaussian High Pass Filters The Gaussian high pass filter is given as: H ( u, v) 1 e D where D 0 is the cut off distance as before 2 ( u, v)/ 2D 2 0

Gaussian High Pass Filters Results of Gaussian high pass filtering with D 0 = 15 Results of Gaussian high pass filtering with D 0 = 80 Results of Gaussian high pass filtering with D 0 = 30

Highpass Filter Comparison Results of ideal high pass filtering with D 0 = 15

Highpass Filter Comparison Results of Butterworth high pass filtering of order 2 with D 0 = 15

Highpass Filter Comparison Results of Gaussian high pass filtering with D 0 = 15

Fast Fourier Transform The reason that Fourier based techniques have become so popular is the development of the Fast Fourier Transform (FFT) algorithm Allows the Fourier transform to be carried out in a reasonable amount of time Reduces the amount of time required to perform a Fourier transform by a factor of 100 600 times!

Fourier Domain Filtering & Spatial Domain Filtering Similar jobs can be done in the spatial and frequency domains Filtering in the spatial domain can be easier to understand Filtering in the frequency domain can be much faster especially for large images

Summary In this lecture we examined image filtering in the frequency domain Background From Fourier series to Fourier transform Properties of the Fourier transform Discrete Fourier transforms The basics of filtering in the frequency domain Image smoothing using frequency domain filters Image sharpening using frequency domain filters

Thanks for your attention

Lecture 4 Filtering in the Frequency Domain. Lin ZHANG, PhD School of Software Engineering Tongji University Spring 2016