ECE Digital Image Processing and Introduction to Computer Vision. Outline

Transcription

1 ECE Digital mage Processing and ntroduction to Computer Vision Depart. of ECE, NC State University nstructor: Tianfu (Matt) Wu Spring Recap Outline 2. Thinking in the frequency domain Convolution for Linear, Time-nvariant System (LT) Fourier series Fourier transform Convolution theorem 1

2 1. Recap, Linear Spatial Filtering Linear spatial filtering can be defined w.r.t. either convolution or crosscorrelation. t is a matter of preference to chose one vs the other. n the literature, it is likely to encounter the terms, convolution filter, convolution mask or convolution kernel. As a rule, these terms are used to denote a spatial filter, and not necessarily that the filter will be used for true convolution. 2. g[x, y] = 3012 /01. w s, t f[x + s, y + t], as cross-correlation if image origin at left-top = w s, t f[x s, y t], as convolution if image origin at right-bottom (i.e. reverse) / Recap, Unsharp Masking and Highboost Filtering = original smoothed (5x5) detail + α = original detail sharpened Source: S. Lazebnik 2

3 1. Recap a)? = D * B b) A =? *? c) F = D *? d)? = D * D F E G A C B H D Source: D. Hoiem 2. Thinking in the Frequency Domain Motivation : Why does the Gaussian give a nice smooth image, but the square filter give edgy artifacts? Gaussian Box filter 3

4 2. Thinking in the Frequency Domain Motivation : Why do we get different, distancedependent interpretations of hybrid images?? Source: D. Hoiem 2. Thinking in the Frequency Domain Motivation : Why does a lower resolution image still make sense to us? What do we lose? Why can we compress an image significantly? mage: mage pyramid Source: D. Hoiem 4

5 2. Thinking in the Frequency Domain Motivation V: How to represent signals in a physically meaningful coordinate system? E.g., Taylor series represents any function using polynomials, Which are unstable and not very physically meaningful Source: S. Narasimhan 2. Thinking in the Frequency Domain Motivation V: How to represent signals in a physically meaningful coordinate system? t is easier to talk about signals in terms of frequencies (i.e., how fast/often signals change, etc.) and decompose them accordingly. 5

6 Convolution t describes the output of a Linear and Time- nvariant (LT) system. (or, Translation-nvariant) f(t) LT g(t) Linearity: 8 c 8 f 8 t = 8 c 8 g 8 t Time-nvariance: f t Δt ==> g(t Δt) Any LT system can be characterized entirely by a single function called the system's impulse response h(t) δ(t) LT h(t) mpulses and Their Sifting Properties mpulses (distributions), if t = 0 δ t = A 0, if t 0, δ t = δ t, F δ t dt = 1, x t δ t = x 0 δ(t) 1 Sifting property: F 1 f(t)δ t dt = f(0), F f(t)δ t t J dt = f(t J ) 1 Similarly, we can define discrete impulses x t is continuous at x = 0 6

7 Convolution t describes the output of a Linear and Time- nvariant (LT) system. (Translation-nvariant) Any LT system can be characterized entirely by a single function called the system's impulse response h(t) f(t) LT h(t) g t = f t h(t) g t = F f τ h t τ dτ 1 F δ t τ h τ dτ 1 = F f t τ h t dτ 1 = F δ τ t h τ dτ 1 = h(t) Convolution Exponential functions are Eigenfunctions of a LT system. Ae 3/ LT h(t) H s Ae 3/ s, A C Background: An eigenfunction is a function for which the output of the operator/system H is a scaled version of the same input function f, Hf = λf, (in the spirit similar to the Eigenvectors of a matrix in linear algebra, Ax = λx) eigenvalues 7

8 Thinking in the Frequency Domain Motivation V: How to represent signals in a physically meaningful coordinate system? t is easier to talk about signals in terms of frequencies (i.e., how fast/often signals change, etc.) and decompose them accordingly. e XY/ LT h(t) H u e XY/ Orthonormal basis functions: {e X8[ \/, k Z} in one periodic interval < T = 2π/u J > < e X8[ \/, e Xe[ \/ > = F e X8[ \/ e Xe[ \/ dt fgh = T, if l = k, 0, otherwise Some Background Complex Numbers c = R + i, R, R, i o = 1 Conjugate: c = R i Similarly, Complex functions F(u) = R(u) + i (u) Geometrically: a point (R, ) in a complex plan spanned by real axis and imaginary axis c = c (cos θ + i sin θ), c = R o + o, θ = arctan /R Euler s Formula using Taylor series expansions: e X{ = 1 + ix + (ix)o 2! + (ix)} 3! + (ix) 4! + = 1 + ix xo 2! ix} 3! + x 4! + cos x = 1 xo 2! + x 4! x x} +, sin x = x 6! 3! + x 5! x 7! + So, e X{ = cos x + i sin x, cos x = c = c e X ˆ Š ˆŒ Š o, sin x = ˆ Š1ˆŒ Š o 8

9 Thinking in the Frequency Domain Can we represent any signals in the coordinate system spanned by e XY/, < u <? f so, we can describe convolution in the new space in a simpler way. e XY/ LT h(t) H u e XY/ Building blocks: Asin(ut + φ) Jean Baptiste Joseph Fourier ( )...the manner in which the author arrives at these equations is not exempt of difficulties and...his analysis to integrate them still leaves something to be desired on the score of generality and even rigor. Any univariate function can be rewritten as a weighted sum of sines and cosines of different frequencies. had crazy idea (1807): Don t believe it? Neither did Lagrange, Laplace, Poisson and other big wigs Laplace Not translated into English until 1878! But it s (mostly) true! called Fourier Series there are some subtle restrictions Lagrange Legendre 9

10 The building block: A sin(ut + φ) Add enough of them to get any signal f(t) you want! A Sum of Sinusoids Frequency Spectra Example : g(t) = sin(2πv t) + (1/3)sin(2π(3v) t) = + Slides: Efros 10

11 Frequency Spectra Frequency Spectra = + = 11

12 Frequency Spectra = + = Frequency Spectra = + = 12

13 Frequency Spectra = + = Frequency Spectra = + = 13

14 Frequency Spectra = A å k= 1 1 sin(2 p kt ) k Frequency Spectra A general trigonometric sum (with amplitude and phase shown explicitly), 0 A sin(2πnt + φ ) For the simplicity in calculation, it is rewritten as 0 a cos 2πnt + b sin(2πnt) f a constant term is included (n = 0, as the DC component) a J 2 + a cos 2πnt + b sin(2πnt) 0 Use complex exponentials, 01 c e Xo /, c C 14

15 Fourier Series For a function f t of a continuous variable t that is periodic with period T, the Fourier series is defined as, Where f t = c = 1 T F 01 c e Xo g / = c [cos 2πn 2πn t + i sin T T t] 01 g/o 1g/o f t e1xo g / dt, n = 0, ±1, ±2, Fourier Transform For a function f t of a continuous variable t, the Fourier transform is defined as, F{f t } = F f t e 1Xo [/ dt F(μ) 1 The inverse Fourier transform, f t = F 1 {F(μ)}, f t = F F μ e Xo [/ dμ 1 Symmetry: F F t = f( μ) 15

16 Fourier Transform Some sufficient conditions for the existence Dirichlet conditions f t dt < 1 f t has finite maxima and minima within any finite interval f t has finite number of discontinuities within any finite interval Square integrable function [f(t)] o dt < 1 Fourier Series and Transform Fourier series: representing and analyzing periodic signals Fourier transform: representing and analyzing nonperiodic signals Connection: Considering nonperiodic phenomena (and thus just about any general function) as a limiting case of periodic phenomena as the period tends to infinity. A discrete set of frequencies in the periodic case becomes a continuum of frequencies in the nonperiodic case, the spectrum is born, and with it comes the most important principle of the subject: Every signal has a spectrum and is determined by its spectrum. You can analyze the signal either in the time (or spatial) domain or in the frequency domain. Recall: The true logic of this world is the calculus of probabilities. J. Maxwell 16

17 Example: box function Fourier spectrum Frequency spectrum Sinc function: sinc m = œ ž Ÿ Ÿ Example: impulse 17

18 Example: Music We think of music in terms of frequencies at different magnitudes Slide: Hoiem Other signals We can also think of all kinds of other signals the same way xkcd.com 18

19 Properties of the Fourier Transform Let f t, g t and h t be functions satisfying the sufficient conditions of the Fourier transform. Denote by F u, F u, and F (u) the Fourier transforms respectively. Linearity h t = af t + bg t F u = af u + bf (u), a, b C Time/Translation shifting h t = f t t J F u = e 1Xo / \Y F u, t J R Frequency/Modulation shifting h t = e 1Xo /Y \f x F u = F u u J, u J R Properties of the Fourier Transform Time Scaling h t = f at F u = 2 F Conjugation Y 2 principle), a R, a 0 (uncertainty h t = f t F u = F u ntegration Differentiation F F 0 = F f t dt 1 u = i2πu F (n) 19

20 Convolution Theorem f t h t = f τ h t τ dτ F f t h t = f τ h t τ dτ e 1Xo [/ dt = F f(τ) F h t τ e 1Xo [/ dt 1 1 = F f τ [F u e 1Xo Y ] dτ 1 = F (u) F f τ e 1Xo Y dτ 1 = F u F (u) f t h t F u F (u) dτ Convolution Theorem Convolution Multiplication Spatial Forward FT Frequency Frequency nverse FT Spatial 20

21 2-D mpulses and Their Sifting Properties mpulses (distributions), if t = z = 0 δ t, z = A 0, otherwise, F F δ t, z dt dz = 1, f t, z δ t, z = f 0,0 δ(t, z) 1 1 Sifting property: F F f(t, z)δ t, z dt dz = f(0,0), 1 1 F F f(t, z)δ t t J, z z J dt dz = f(t J, z J ) 1 1 Similarly, we can define discrete impulses 2-D Fourier Transform For a function f t, z of two continuous variable t and z, the Fourier transform is defined as, F{f t, z } = F F f t, z e 1Xo [/ ª dt dz F(μ, v) 1 1 The inverse Fourier transform, f t, z = F 1 {F(μ, v)}, f t, z = F F F μ, v e Xo [/ ª dμ dv

22 Fourier analysis in images ntensity mage Fourier mage Signals can be composed + = More: 22

23 Summary Thinking in the frequency domain Convolution for Linear, Time-nvariant System (LT) Fourier series Fourier transform Convolution theorem 23