Introduction to Linear Image Processing 1 IPAM - UCLA July 22, 2013 Iasonas Kokkinos Center for Visual Computing Ecole Centrale Paris / INRIA Saclay
Image Sciences in a nutshell 2 Image Processing Image to Image da da Imaging Physics to Image Computer Graphics Symbols to Image Computer Vision Image to Symbols
Images as functions Continuous Discrete d=1: Gray d=3: Color 3
Image Denoising 4
Image Denoising 5 Key assumption: clean image is smooth
Moving Average in 2D 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 0 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Slide Source: S. Seitz
Moving Average in 2D 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 0 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Slide Source: S. Seitz
Moving Average in 2D 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 20 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 0 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Slide Source: S. Seitz
Moving Average in 2D 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 20 30 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 0 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Slide Source: S. Seitz
Moving Average in 2D 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 0 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Slide Source: S. Seitz
Moving Average in 2D 11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 0 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10 0 20 40 60 60 60 40 20 0 30 60 90 90 90 60 30 0 30 50 80 80 90 60 30 0 30 50 80 80 90 60 30 0 20 30 50 50 60 40 20 10 20 30 30 30 30 20 10 10 10 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Slide Source: S. Seitz
Denoising: input 12
Denoising: first application of averaring filter 13
Denoising: tenth application of denoising filter 14
Denoising: application of larger box filter 15
Weighted averaging 16
Weighting kernel 17 Gaussian function: Standard deviation, σ: determines spatial support σ = 2 σ = 5 σ = 10
Moving average 18
Gaussian blur 19
Image Processing 20 image filter image
Linear Image Processing 21 Linearity Translation Invariance Linear, Translation-Invariant (LTI) system
Linear Image Processing 22 image filter image From time-invariance: useful bases.
Linear Image Processing 23 image filter image From time-invariance: useful bases.
Linear algebra reminder 24 Basis: N linearly independent vectors Expansion on basis: Orthonormal basis: Expansion coefficients: Expansion:
Canonical basis 25
Canonical basis for 2D signals 26 Kronecker delta
Canonical basis for 2D signals 27 Kronecker delta
Canonical basis for 2D signals 28 Kronecker delta
Canonical basis for signals: expansion 29 Signal expansion: Identify terms: Rewrite: Unit sample function Sifting property:
Canonical basis for signals and LTI filters 30 unit sample impulse response Translation-invariance Any signal: By linearity: Convolution sum Output of any LSI filter for any input: convolution of input with filter s impulse response
Convolution discrete and continuous 31 2D convolution sum: 2D convolution integral:
Linear Image Processing 32 image filter image From time-invariance: useful bases.
Associative property & efficiency 33 Associative Property: Separability of Gaussian: Slow Fast
Associative property & accuracy 34 Associative Property: Derivative of Gaussian: exact approximate
Associative property & multi-scale processing 35 Associative Property: Semi-group property of Gaussian:
Denoising: first application of averaging kernel 36
Denoising: 10 th application of denoising kernel 37
Distributive property & efficiency 38 Distributive property: Steerable fliter: W. Freeman and E. Adelson, The design and use of steerable filters, PAMI, 1991
Linear algebra reminder: eigenvectors 39 Eigenvectors: Full-rank, real and symmetric: eigenbasis
Eigenvectors and eigenfunctions 40 Eigenvector: Eigenfunction: Input: Output:
Eigenfunctions for LTI filters 41 LTI filter: Let s guess: It works: Frequency response:
Expansion on harmonic basis 42 From orthonormality: Inner product for complex functions: Discrete-time: Continuous-time:
Change of basis 43 Canonical expansion: Eigenbasis expansion: Rotation matrix from eigenbasis: Fourier transform: change of basis Rotation from canonical basis to eigenfunction basis
Fourier Analysis 44
Fourier synthesis equation 45 Continuous-time: Discrete-time:
Convolution theorem of Fourier transform 46 Input expansion: Output: Expansions:
Linear Image Processing 47 image filter image From time-invariance: useful bases.
Convolution theorem 48 Fourier Analysis Fourier Synthesis
Convolution theorem and efficiency 49 Fast Fourier Transform Fast Fourier Transform
Gaussian blur Time 50 Frequency
Moving average Time 51 Frequency
Modulation property and Gabor filters 52 Modulation property: Gaussian: Time Frequency
Modulation property and Gabor filters 53 Modulation property: Gaussian: Gabor: Time Frequency
2D Gabor filterbank 54 Consider many combinations of and Increasing Increasing Frequency responce isocurves
2D Gabor filterbank and texture analysis 55
2D Gabor filterbank and texture analysis 56
Summary 57 Linear Time-Invariant filters Convolution Fourier Transform (Derivative-of) Gaussian filters Steerable filters Gabor filters Thursday s lecture: Pyramids, Scale-Invariant Blobs/Ridges, SIFT, HOG, Log-polar features, Harmonic analysis on surfaces Further reading: Fast recursive filters: Recursively implementing the Gaussian and its Derivatives - R. Deriche, 1993 Recursive implementation of the Gaussian filter. I. Young, L. Vliet, 1995 Fast IIR Isotropic 2D Complex Gabor Filters with Boundary Initialization, A Bernardino, J. Santos-Victor, TIP, 2006 Wavelets: A Wavelet Tour of Signal Processing, S. Mallat, 2008