Cris Luengo TD396 fall 4 cris@cbuuse Today s lecture Local neighbourhood processing smoothing an image sharpening an image The convolution What is it? What is it useful for? How can I compute it? Removing uncorrelated noise from an image The Fourier transform What is it? What is it useful for?
Cris Luengo TD396 fall 4 cris@cbuuse Neighbourhoods
Cris Luengo TD396 fall 4 cris@cbuuse Local neighbourhood operation For each pixel, examine its neighbourhood and compute an output value (mean) 4 6 3 9 6 9 6 7 4 3 46 47 4 37 4 4 6 8 7 4 6 3 4 46 3 7 8 4 6 7 6 3 6 3 3 8
Cris Luengo TD396 fall 4 cris@cbuuse Local neighbourhood operation Possible operations to do for each neighbourhood: Neighbourhood size and shape is very important average (mean, median, etc) weighted average other statistics (variance, maximum, etc) difference (to compute derivative) round neighbourhood gives rotation invariance Adaptive filtering: changing size, shape and/or operation depending on local image properties
Cris Luengo TD396 fall 4 cris@cbuuse Smoothing an image Input image mean filter median filter
Cris Luengo TD396 fall 4 cris@cbuuse Smoothing an image Input image mean filter weighted mean filter
Cris Luengo TD396 fall 4 cris@cbuuse How to define Gaussian weights σ determines the amount of smoothing the neighbourhood size should be large enough to contain the whole Gaussian bell! rule of thumb: ceil(3σ) + sum of all weights normalised to x + y exp πσ σ ( ) ceil(3σ) +
Cris Luengo TD396 fall 4 cris@cbuuse Weighted mean filter For each pixel, multiply the values in its neighbourhood with the corresponding weights, then sum /9 /9 /9 /9 /9 /9 4 6 3 9 6 9 6 7 4 3 46 47 4 37 4 4 6 8 7 4 6 3 4 46 3 7 8 4 6 7 6 3 6 3 3 8 /9 /9 /9
Cris Luengo TD396 fall 4 cris@cbuuse Applications? Write down as many applications of a smoothing filter as you can come up with
Cris Luengo TD396 fall 4 cris@cbuuse Application: noise reduction input image Normally distributed noise Salt & pepper noise 3x3 mean filter 3x3 median filter
Cris Luengo TD396 fall 4 cris@cbuuse Application: abstraction Sometimes you just don t want all those details
Cris Luengo TD396 fall 4 cris@cbuuse Application: shading correction Gaussian smoothing, σ = pixels
Cris Luengo TD396 fall 4 cris@cbuuse Sharpening an image Unsharp masking original smoothed (3x3) sharpened (α = 9) sharpened = (+α) original α smoothed
Cris Luengo TD396 fall 4 cris@cbuuse Sharpening an image sharpened = (+α) original α smoothed sharpened = original + α ( original smoothed ) original /9 /9 /9 smoothed - - - /9 /9 /9-8 - /9 /9 /9 - - - 9 diff
Cris Luengo TD396 fall 4 cris@cbuuse Laplace filter Laplace operator: Δ= = + x y -4-8 sharpened = original + 9 ( original smoothed ) sharpened = original - Laplace
Cris Luengo TD396 fall 4 cris@cbuuse Sobel filter Approximates the first derivatives: - Sx, x y - - - - - Sy
Cris Luengo TD396 fall 4 cris@cbuuse Detecting edges Approximates the gradient magnitude: ( + x y sqrt ( Sx^ + Sy^ ) ) ( )
Cris Luengo TD396 fall 4 cris@cbuuse Adaptive filtering Many non-linear filters are meant to reduce noise without blurring the edges One common technique is to adapt the kernel so that it does not extend across any edges The bilateral filter is the most common one input image median filter bilateral filter
Cris Luengo TD396 fall 4 cris@cbuuse Bilateral filter A new kernel is designed for each output pixel Kernel weights are reduced if the corresponding pixel in the input image has a large difference in intensity with the central pixel h x ( x ) = Gσ ( x x ) Gσ ( f ( x ) f ( x )) x f
Cris Luengo TD396 fall 4 cris@cbuuse What happens at the image edge?
Cris Luengo TD396 fall 4 cris@cbuuse What happens at the image edge? Write down as many different ways of extending the edge as you can think of
Cris Luengo TD396 fall 4 cris@cbuuse What happens at the image edge? Mean padding f[end+x] = mean(f) Zero order hold f[end+x] = f[end]
Cris Luengo TD396 fall 4 cris@cbuuse What happens at the image edge? Periodic boundary condition f[end+x] = f[x] Symmetric boundary condition f[end+x] = f[end-x]
Cris Luengo TD396 fall 4 cris@cbuuse Linear neighbourhood operation For each pixel, multiply the values in its neighbourhood with the corresponding weights, then sum /9 /9 /9 /9 /9 /9 4 6 3 9 6 9 6 7 4 3 46 47 4 37 4 4 6 8 7 4 6 3 4 46 3 7 8 4 6 7 6 3 6 3 3 8 /9 /9 /9
Cris Luengo TD396 fall 4 cris@cbuuse Linear neighbourhood operation For each pixel, multiply the values in its neighbourhood with the corresponding weights, then sum (-,-) (,-) (,-) f(x,y) h(i,j) g(x,y) (-,) (,) (,) (-,) (,) (,) (x,y) (x,y)
Cris Luengo TD396 fall 4 cris@cbuuse Convolution h is: impulse response function point-spread function convolution kernel g (t ) = f (t ) h(t ) g (t ) = f (t τ) h( τ) d τ b g [n] = f [n k ] h[k ] k =a [a,b] is the interval where h is defined, eg [-,]
Cris Luengo TD396 fall 4 cris@cbuuse Convolution properties Linear: Scaling invariant: C f h = C f h Distributive: f g h = f h g h Time Invariant: shift f h = shift f h Commutative: f h = h f Associative: f h h = f h h (= shift invariant)
Cris Luengo TD396 fall 4 cris@cbuuse Associativity of convolution f (h h ) = (f h ) h if h = h h then f h = (f h ) h thus: you can decompose h to speed up the operation! Eg the Gaussian can be decomposed into two one-dimensional filters: G( x, y ) = e π σ x + y σ = e π σ x σ e π σ y σ
Cris Luengo TD396 fall 4 cris@cbuuse Kernel decomposition G = G x G y original convolved with Gx Gx and Gy are both a kernel with 3x values G is a kernel with 3x3 values convolved with Gy 3+3 = 6 MADs 3x3 = 96 MADs
Cris Luengo TD396 fall 4 cris@cbuuse Sequence of filters f (h h h3 ) = (((f h ) h ) h3 ) 3+3 ops 4(3+3) ops = 4 ops 9+9 ops = 8 ops
Cris Luengo TD396 fall 4 cris@cbuuse Sequence of filters 3+3 ops 4(3+3) ops = 4 ops 9+9 ops = 8 ops
Cris Luengo TD396 fall 4 cris@cbuuse Sequence of filters a b e f (example from Section 37) b = Laplace(a) c=a-b d = Sobel(a) e = smooth(d) f=c*e g=a+f h = gamma(g) c d g h
Cris Luengo TD396 fall 4 cris@cbuuse Jean Baptiste Joseph Fourier Born March 768, Auxerre (Bourgogne region) Died 6 May 83, Paris Same age as Napoleon Bonaparte Permanent Secretary of the French Academy of Sciences (8-83) Foreign member of the Royal Swedish Academy of Sciences (83)
Cris Luengo TD396 fall 4 cris@cbuuse The Fourier transform = + + +
Cris Luengo TD396 fall 4 cris@cbuuse The Fourier transform Can you think of a function that cannot be decomposed into Fourier basis functions?
Cris Luengo TD396 fall 4 cris@cbuuse Fourier analysis
Cris Luengo TD396 fall 4 cris@cbuuse Fourier example
Cris Luengo TD396 fall 4 cris@cbuuse Fourier example
Cris Luengo TD396 fall 4 cris@cbuuse Fourier example
Cris Luengo TD396 fall 4 cris@cbuuse Fourier example
Cris Luengo TD396 fall 4 cris@cbuuse Complex numbers i= i i= x =a+ i b x * =a i b (complex conjugate) * (Euler s formula) x x =a + b = x a= x cos( x ) b x=arctan( ) a b= x sin( x ) e i φ=cos φ+ i sin φ x = x cos( x ) + i x sin( x ) = x e i x
Cris Luengo TD396 fall 4 cris@cbuuse Fourier basis function e iωx = cos(ω x )+ i sin(ω x ) ω= π f = T π T
Cris Luengo TD396 fall 4 cris@cbuuse Fourier transform F (ω) = f ( x ) e i ω x d x f (x) = = F (ω )e + i ω x F (ω )e + i ω x F (ω3 )e + i ω3 x + + +
Cris Luengo TD396 fall 4 cris@cbuuse Fourier transform F (ω) = complex value f ( x ) e i ω x d x f (x) = complex function F (ω )e + i ω x F (ω )e + i ω x F (ω3 )e + i ω3 x complex function???
Cris Luengo TD396 fall 4 cris@cbuuse Fourier basis function A ei ω x + A* e i ω x is a real-valued function Thus: we need negative frequencies! For real-valued images: At frequency ω we have weight A At frequency -ω we have weight A* F ( ω) = F * (ω)
Cris Luengo TD396 fall 4 cris@cbuuse Inverse Fourier transform f (x) = π F (ω) e iωx dω normalization no minus sign Compare with the forward transform: F (ω) = i ω x f (x) e dx
Cris Luengo TD396 fall 4 cris@cbuuse Fourier transform pairs Spatial cosine impulses sine impulses box sinc sinc box Gaussian white noise Notice the symmetry! Gaussian white noise Frequency impulse
Cris Luengo TD396 fall 4 cris@cbuuse Properties of the Fourier transform Spatial scaling Linear Amplitude scaling Addition Translation Convolution phase change ℱ {f h} = ℱ {f } ℱ {h} = ℱ {f } ℱ {h} ℱ {f h}
Cris Luengo TD396 fall 4 cris@cbuuse Summary of today s lecture Virtually all filtering is a local neighbourhood operation Convolution = linear and shift-invariant filters Many non-linear filters exist also eg mean filter, Gaussian weighted filter kernel can sometimes be decomposed eg median filter, bilateral filter The Fourier transform decomposes a function (image) into trigonometric basis functions (sines & cosines) The Fourier transform is used to analyse frequency components of an image
Cris Luengo TD396 fall 4 cris@cbuuse Reading assignment Filtering The Fourier transform Sections 34, 3, 36, 37, 4, 3 Section 4 Exercises: 3, 36, 39, 33, 37, 38, 43