Convolution Spatial Aliasing Frequency domain filtering fundamentals Applications Image smoothing Image sharpening

Size: px

Start display at page:

Download "Convolution Spatial Aliasing Frequency domain filtering fundamentals Applications Image smoothing Image sharpening"

Marilynn Wade
6 years ago
Views:

1 Frequency Domain Filtering Correspondence between Spatial and Frequency Filtering Fourier Transform Brief Introduction Sampling Theory 2 D Discrete Fourier Transform Convolution Spatial Aliasing Frequency domain filtering fundamentals Applications Image smoothing Image sharpening Selective filtering

2 Spatial Domain Image Filter Convolution Processed Image Frequency Domain Image Filter Fourier Transform Image Filter Multiplication Processed Image Inverse Fourier Transform Processed Image Remember the Laplace Transform and the s domain? Transform ODEs to polynomial equations Remember Spatial filters/masks? (Gaussian, sobel, etc m n mask (. Apply onto Or in vector format: 1 h x y image ( f x y g ( x. y T R = wz + wz wz = w z This operation can be succinctly expressed by a b ( ( ( ( h x. y f x, y = h s, t f x s, y t = g( x, y a = ( m 1/2, b= ( n 1/2 s= a t= b w 1 w 2 w 3 w 4 w 5 w 6 w 7 w 8 w 9

3 Any periodic function can be represented by a sum of sines and cosines: Fourier Series Extension to non periodic functions : Fourier Transform Operation that transforms one complex valued function of a real variable into another Continuous FT Discrete FT Single/Multivariate FT Representing complex numbers: C = R+ ji 2 2 ( cos θ sin θ, C = C + j C = R + I jθ jθ C = C e, e = cosθ + jsinθ Fourier Series: 2πn 2πn j t 1 T /2 j t T T f ( t = cne, cn = f( t e dt T T /2 n= C = R+ ji

4 Continuous single variate: j2 { (} ( μ ( I = = πμt f t F f t e dt Conversely, Inverse Fourier Transform: { ( μ } ( μ ( 1 j 2 πμt f t F F e dt =I = Fourier Transform Pair Using Euler s formula, ( μ = ( cos ( 2 πμ sin ( 2 πμ F f t t j t dt Find the Fourier Transform of A W /2 t W /2 f ( t = otherwise W /2 2 πμ 2 ( μ = ( = j t j πμ t F f t e dt Ae dt W /2 A j2πμt W /2 A jπμw jπμw = e = e e j 2πμ W /2 j 2πμ = AW ( πμw ( πμw W sin

5 Visualizing the Fourier Transform and Fourier Spectrum (frequency spectrum The convolution of two functions is defined as ( ( ( ( f t h t = f τ h t τ dτ Taking Fourier Transform j2πμt I f ( t h( t f ( τ( h t τ dτ = e dt And reversing order of integration: Similarly: j2πμt I f ( t h( t f ( τ h( t τ e dt = dτ ( ( μ j 2πμτ = f τ H μ e dτ = F μ H μ commutative ( ( ( ( ( ( μ I f t h t = F μ H Multiplication in t domain is analogous to convolution in μ domain

6 Computers process data discretely and so are images! Sampled function ( = ( ( f t f t s t ΔT Sampling function = f ( t δ ( t nδt n= Value of each sample: f = f( t δ ( t kδt dt k = f( kδt { Δ } ( μ =I { } =I ( F f( t f( t s T t ( μ S( μ = F 1 n It can be shown that S ( μ = δ μ Δ T n= Δ T F μ F τ S μ τ dτ ( = ( ( (verify it as an exercise 1 n = F ( τ δ μ τ d τ T Δ n= T Δ 1 n = F μ ΔT n= ΔT Hence the FT of the sampled function is a continuous infinite and periodic sequence of duplicates of the FT of the continuous function Duplicates are separated and individually scaled by by 1/ΔT

7 Lets see it visually! Δ T <ΔT C Over sampling Δ T =ΔT C Critically sampling Δ T >ΔT C Under sampling Need only one complete lt period to recover f(t using IFT Single variate ( { } ( { Δ } ( ( F μ =I f( t =I f( t s T t = F μ S μ = f ( t e j2 t F πμ ( μ = f ( t δ( t nδt e dt n = j2πμt j2πμ nδt = fe = n where n ( δ ( Δ n= ( dt Remember? f t f t s t ( = ( ( ΔT = f ( t δ ( t nδt n= f f t t n T dt = f( nδt Discrete sampling of F μ over interval μ= and μ=1/δt (only 1 needed dd M equally spaced samples m M 1 j2π mn Given a set of M samples of μ = M M Δ T F m = f ne f n, a discrete corresponding n= complex expression is F m=,1,2,,m 1 m

8 Conversely given a set of F m discrete data points, a sample set of f n can be recovered. IDFT: f n M 1 j2π mn M 1 j2π mn 1 M M = Fme DFT: Fm = fne m,n=,1,2,,m 1 M n= = m Discrete Fourier Transform Pair In image (spatial processing, the more natural and widely used expression is: M 1 1 f x F u e M j2πux M ( = ( ( ( u= M 1 F u = f x e x= j2πux M u,x=,1,2,,m 1 Image Processing: xyare x,y spatial variables and u,vare spatial frequency variables Signal Processing: t is a temporal variable and μ is a frequency variable Periodicity F( u = F( u+ km f ( x f( x km = + k integer Discrete version of convolution (aka circular convolution M 11 ( ( = ( ( f x h x f m h x m m= m=,1,2,,m 1

9 Given following 4 data sampled function, what is F(1? 3 F( 1 = f ( x e x= j2π x 4 jπ 3 jπ 2 jπ 2 = 1e + 2e + 4e + 4e Similarly F( = 11, F(2 = 1 and F(3 = 3 2j hence M=4 Reconstruct f( g given F(, ( F(1 (, F(2 ( and F( ( ( [ ] f = F u = j 1 3 2j = u= Two variate M 1N 1 ux uy j2π + M N 1 f x y F u v e MN IDFT: (, (, = u,v=,1,2,,m 1 u= v= M 1N 1 ux vy j2π + M N DFT: (, (, F u v f x y e = x,y=,1,2,,m 1 x= y= In image processing, f(x,y is a digital image of size N x M Image Processing: x,y are spatial variables and u,v are spatial frequency variables

10 Translation & Rotation Periodicity Symmetry And. Also known as 2 D circular convolution M 1N 1 (, (, = (. (, f x y h x y f mn h x m y n m = n = x=,1, 2,..., M 1 y =,1, 2,..., N 1 As with 1 D case, (, (, (, (, I f x y h x y = F u v H u v (, (, (, (, I f x y h x y = F u v H u v The latter expression is the foundation of linear filtering i in the frequency domain.

11 Before foraging into frequency domain filtering, lets do a quick digress into aliasing in digital imaging What is Aliasing? High frequency components masquerading as lower frequencies Temporal Aliasing Pertains to time intervals between images in a sequence The wagon wheel effect Capturing frame rate too low Spatial Aliasing Under sampling of scenes in digital images due to finite pixels Jaggies

12 When image shrinking is achieved by row column deletion (reduce 5% by deleting ever other row and column, aliasing i can occur Reduce aliasing by first smoothing image before resampling Or use interpolation during resampling (bicubic: done in photoshop Filter function Input image h x. y f x, y = g( x, y Taking the DFT: ( ( Recall: Filtered image { ( ( } (, (, (, (, I f xy h xy = F uv H uv I h x. y f x, y = Guv (, Filter TF We have: ( ( F u, v H u, v = G( u, v And finally, 1 { FuvH ( ( uv } gxy (, =I,, Hence, Filtering in the frequency domain requires altering the DFT of the input image and computing the IDFT to obtain the filtered/processed image

Each term of F(u,v contains all values of f(x,y

image properties difficult However F(u=,v= is

Increasing rate of change of pixel intensities u

impulse, f x y δ x y F( u v Then: 1 { H ( uv, } h (

As this filter is obtained from the response of a

13 Each term of F(u,v contains all values of f(x,y making direct association of component values to image properties difficult However F(u=,v= is proportional to average intensity of image v Increasing rate of change of pixel intensities u F(u,v From gxy (, =I FuvH (, ( uv, 1 { } If f is an impulse, f x y δ x y F( u v Then: 1 { H ( uv, } h ( xy, I = (, = (,, = 1 where h(x,y is a spatial filter As this filter is obtained from the response of a frequency domain filter to an impulse, h(x,y is also called the impulse response of H(u,v. And since filter contain only finite quantities, they are also called Finite Impulse Response (FIRfilters

14 Edges and noise contribute to the high frequency components of an image s FT Smoothing or blurring is achieved by high frequency attenuation Low Pass Filters Ideal lowpass filters (sharp Butterworth lowpass filter Gaussian Lowpass filter (smooth Mathematically: (, H u v 1 if D( u, v D = if D( u, v > D Cutoff Frequency Where D(u,v is the distance from (u,v to origin. Remove Keep Cutoff Frequency

15 Spatially: (looks like a 2 D sinc function Mathematically: H ( u, v = 1 + [ Duv (, / D] 2 1 n Where D(u,v is the distance from (u,v to origin. Remove Keep

16 Spatially: n=1 n=2 n=5 n=25 Mathematically: (, H u v = e 2 2 (, /2 D uv D Where D(u,v is the distance from (u,v to origin. Remove Keep Note: IFT of GLPF is also a Gaussian. (spatial filter is also Gaussian

17 Ideal LPF Butterworth LPF Gaussian LPF Machine recognition systems (OCR Bridge small gaps in alphabets of text Printing/publishing/advertising Photoshopping to remove blemishes/lines and obtain a smoother, softer result Aerial/Satellite imagery Aerial/Satellite imagery Removing unwanted scan lines in images

18 Amplify edges by removing low frequency content Achieved by low frequency attenuation High pass Filters Ideal highpass filters Butterworth highpass filter Gaussian highpassfilter Highpass/Lowpass p Identity H ( u, v = 1 H ( u, v HP LP Ideal: H ( uv, if D( u, v D = 1 if D( u, v > D Butterworth H( u, v = 1 + [ D / D ( u, v ] 2 1 n H u, v = 1 e Gaussian ( 2 2 (, /2 D uv D

19 Ideal Butterworth Gaussian

y Fingerprint g recognition g y Enhance ridges and

clarity l i in i X rays X y X rays cannot be

blurred y Astronomy y y Accentuating surface

planets Highpass Filter With thresholding Gaussian

20 y Fingerprint g recognition g y Enhance ridges and reduce smudges y Improving I i bone b structure clarity l i in i X rays X y X rays cannot be focused like optical lens y Images tend to be blurred y Astronomy y y Accentuating surface contours/features in images of extraterrestrial planets Highpass Filter With thresholding Gaussian Highpass Filtering High frequency g q y Emphasis Filtering With thresholding

21 Process specific bands / small regions of frequencies Bandreject filters / Bandpass filters Ideal filters Butterworth filter Gaussian filter Notch filters Ideal filters Butterworth filter Gaussian filter Bandreject/Bandpass Gaussian Filter Bandreject Bandpass Relationship: H ( u, v = 1 H ( u, v BP BR

22 NF reject (or passes frequencies in a predefined neighborhood about the center of the frequency rectangle Constructed as products of highpass filters whose centers have been translated to the centers of the notches ( Q H uv, = H ( uvh, ( uv, NR k k k 11 where H k (u,v and H k (u,v are HP filters centered at (u k,v k & ( u k, v k Reducing unwanted patterns in scanned newspaper images Isolating and removing corruption caused by AC signals (vertical sinusoidal patterns

23 Implementing filtering in frequency domain 2 D DFT and IDFT is computationally intensive (MN 2 order of summations and additions 124 x 124 image would require ~ a trillion operations for 1 DFT Fast Fourier Transform (FFT MNlog 2 (MN operations 124 x 124 image would require ~ 2 million operations Filter design in frequency domain Use frequency domain for prototyping of filter Find equivalent spatial filter Implement filter in spatial domain Digital Image Processing using Matlab, R. C. Gonzalez, R. E. Woods and S. Eddins, Prentice Hall, 24 Digital Image Processing, R. C. Gonzalez and R. d E. Woods, Prentice Hall, 3 rd Edition, 28

ECG782: Multidimensional Digital Signal Processing

Professor Brendan Morris, SEB 3216, brendan.morris@unlv.edu ECG782: Multidimensional Digital Signal Processing Filtering in the Frequency Domain http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Background