I Chen Lin, Assistant Professor Dept. of CS, National Chiao Tung University Computer Vision: 4. Filtering
Outline Impulse response and convolution. Linear filter and image pyramid. Textbook: David A. Forsyth and Jean Ponce, Computer Vision: A Modern Approach, Prentice Hall, New Jersey, 23. Some contents are from the reference lecture notes: Prof. D. Lowe, Computer Vision, UBC, CA. Prof. D.A. Forsyth, Computer Vision, UIUC. Prof. J. Rehg, Computer Vision, Georgia Inst. of Tech. Prof. T. Darrell, Computer Vision and Applications, MIT.
What s linear filtering? Construct a new image whose pixels are a weighted sum of the original pixel values. Using thesame set ofweights at each point. Mainly based on neighborhood of the target pixels. For example, 6 5 3 Some function 4 6 3 5/4 2 7
Linear filtering The simplest and most useful case: Replace each pixel with a linear combination of its neighbors. The prescription for the linear combination is called the convolution kernel. Similar to correlation, but 6 5 3 4 6 3 * 5/4 2 7 /4 /2 /4 = kernel
Correlation I 9 9 2 9 9 9 9 99 F 9 9 O X X X X X X X X X X X X X X X X X X X X /9 /9.(x + x + x + 9x + x + x + x + 9x + x) = /9.( 9) = Slides from Prof. D. Lowe, Computer Vision, UBC, CA. Credit: Christopher Rasmussen
Normalized correlation Zero mean image, : scale Zero mean image, max:max scale
Normalized correlation Zero mean image, g, : scale Zero mean image, g, max:max scale
Finding hands Figure from Computer Vision for Interactive Computer Graphics, W.Freeman et al, IEEE Computer Graphics and Applications, 998.
Convolution The same as correlation, but with reversed kernels. Represent the linear weights as an image, F, called the kernel. Convolution operation: Center origin of the kernel F at each pixel location Multiply l weights iht by corresponding pixels Set resulting value for each pixel Image, R, resulting from convolution of F with image H, where u,v range over kernel pixels: R ij = u,v H i u,j v F uv Warning: the textbook mixes up H and F
Correlation compared to convolution Correlation compared to convolution Correlation + + = k k j y i x I j i F y x O ), ( ), ( ), ( = = k j k i j y j y ) ( ) ( ) ( Convolution = k k j y i x I j i F y x O ), ( ), ( ), ( = = k j k i j y j y ) ( ) ( ) ( + + = k k j y i x I j i F ), ( ), ( = = k j k i j y j ) ( ) ( If F(i j) F( i j) th l ti i i l tt lti If F(i, j) == F( i, j), then convolution is equivalent to correlation.
Convolution and impulse response Convolution can be viewed as a sequence of impulse responses. k O ( x) = h( i) I( x i) i= k Figures from cnx.org/content/m539/latest/
Examples Original Filtered (no change) Original Shifted left By pixel
Examples (cont.) Original Blur (with a box filter) 2 Original Sharpening filter
Sharpening Sharpening filter Accentuates differences with local average
Examples: Smoothing with a box filter
Smoothing with a Gaussian Averaging does not model defocussed lens well Impulse response should be fuzzy blob, but a box filter would give a little square. Gaussian gives a better model of a fuzzy blob.
Gaussian kernel Wiht Weight contributions tib ti of neighboring ihb i pixels by nearness. For efficiency, usually use integers as weights, e.g.:.25 * Don t forget normalizing the weight sum to. 4.3.3.22.3.3.3.59.97.59.3.22.97.59.97.22.3.59.97.59.3.33.3 3.22 22.3 3.3 3
Smoothing with a Gaussian
Additive Gaussian noise σ = σ = 6
Noise and smoothing Smoothing reduces pixel noise: Each row shows smoothing with Gaussians of different width Each column shows different amounts of Gaussian noise.
Noise and smoothing Filtered noise is sometimes useful E.g. in textures synthesis.
Noise and smoothing Figures from lecture notes D.A. Forsyth, Computer Vision, UIUC.
Efficient implementation Both the box filter and the Gaussian filter are separable into two D convolutions: First convolve each row with a D filter Then convolve each column with a D filter. G x + y 2 2σ σ ( x, y) = e 2 = 2πσ 2 2 2 x y 2 2σ 2σ e 2πσ 2πσ A 2D Gaussian can be expressed as the product of Gaussian of X and Y. e 2 2
Differentiation Recall, for 2D function, f(x, y): ( ) f( x,y) f f x + ε, y = lim x ε ε ε This is linear and shift invariant, so must be the result of a convolution. We could approximate this as symmetric ti finite it difference: h h i +, j h i, j H = x 2 2 Issue: noise Smooth before differentiation Two convolutions to smooth, then differentiate?
Differentiation Because differentiation is convolution, and convolution is associative. ((f**g)**h)=(f**(g**h))(f**(g**h)) We can use a derivative of Gaussian filter.
Scale affects derivatives pixel 3 pixel 7 pixel
Fourier Transform Represent function on a new basis. Think of functions as vectors, with many components. We now apply a linear transformation to transform the basis dot product with each basis element ( ) Basis elements have the form i2π ux +vy e Fgx, ( ( y) ) u, v ( )= gx, ( y)e R 2 i2π ( ux+vy ) dxdy
Discrete Fourier Transform Figures from lecture notes of Prof. T. Darrell, Computer Vision and Applications, MIT.
Phase and magnitude Fourier transform of a real function is complex difficult to visualize. Instead, think ofphase and magnitude Phase = phase of the complex transform Magnitude = magnitude of complex transform E.g. x(t) = ae -bt u(t) X(w) = a/(b + jw) X(w) = a /(b 2 + w 2 ) /2 X(w) = tan -(w/b), if a > X(w) = π + tan -(w/b), if a <
Phase and magnitude Reconstruct with one s phase, and magnitude of the other.
Aliasing Can t shrink hi an image by taking every second pixel If we do so, characteristic errors appear Typically, small phenomena look bigger; fast phenomena can look slower Common phenomenon Wagon wheels rolling the wrong way in movies Checkerboards misrepresented in ray tracing Striped shirts look funny on color television
Re sampling Resample the checkerboard dby tki taking one sample at each circle.
Simple sampling, but
Smoothing as low pass filtering High frequencies lead to trouble with sampling. Suppress high frequencies before sampling! Truncate high frequencies in FT or convolve with a low pass filter.
The Gaussian pyramid Create each level from previous one: smooth and sample Smooth with Gaussians, in part because a Gaussian*Gaussian = another Gaussian G(x) * G(y) = G(sqrt(x 2 + y 2 )) Gaussians are low pass filters, so the representation is redundant once smoothing has been performed
The Gaussian pyramid