Image Processing 1 (IP1) Bildverarbeitung 1

Size: px

Start display at page:

Download "Image Processing 1 (IP1) Bildverarbeitung 1"

Gabriel Conley
5 years ago
Views:

1 MIN-Fakultät Fachbereich Informatik Arbeitsbereich SAV/BV (KOGS) Image Processing 1 (IP1) Bildverarbeitung 1 Lecture 7 Spectral Image Processing and Convolution Winter Semester 2014/15 Slides: Prof. Bernd Neumann Slightly revised by: Dr. Benjamin Seppke & Prof. Siegfried Stiehl

2 Spectral Image Properties An image function may be considered a sum of spatially sinusoidal components of different frequencies. The frequency spectrum indicates the magnitudes of the spatial frequencies contained in an image. f y = v Principle: y x f x = u Important qualitative properties of spectral information: spectral information is independent of image locations sharp edges give rise to high frequencies noise (= disturbances of image signal) is often high-frequency University of Hamburg, Dept. Informatics 2

3 Illustration of 1-D Fourier Series Expansion original waveform approximation of a rectangular pulse with sinusoidal components sinusoidal components add up to original waveform University of Hamburg, Dept. Informatics 3

4 Discrete Fourier Transform (DFT) Computes image representationas a sum of sinusoidals. Discrete Fourier Transform: G uv = 1 MN M 1 N 1 g mn e m=0 n=0 mu 2π j( M +nv N ) for u = 0,, M 1 and v = 0,, N 1 Notation for computing the Fourier Transform: G uv = F g mn g mn = F 1 { } { G uv } Transform is based on periodicity assumption! à periodic continuation may cause boundaryeffects Inverse Discrete Fourier-Transform: M 1 N 1 g mn = G uv e u=0 v=0 mu 2π j( M +nv N ) for m = 0,, M 1 and n = 0,, N University of Hamburg, Dept. Informatics 4

and an imaginary (odd) part: G uv = R uv + i I uv Euler s formula: r e iz = r cos(z) + r i sin(z) 05.11.

5 Basic Properties of DFT Linearity: F{ a g 1mn + b g 2mn } = a F{ g 1mn } + b F{ g 2mn } Symmetry: G -u,-v = G uv for real g mn (such as images) In general, the Fourier transform is a complex function with a real (even) and an imaginary (odd) part: G uv = R uv + i I uv Euler s formula: r e iz = r cos(z) + r i sin(z) University of Hamburg, Dept. Informatics Recommended reading: Gonzalez/Wintz Digital Image Processing Addison Wesley 87 5

6 Measures of DFT Freuqency/amplitude spectrum G(u,v) = Re{ G(u,v) } 2 + Im{ G(u,v) } 2 Power spectrum G(u,v) 2 Phase spectrum Frequency " Φ = arctan Im(u,v) % $ ' # Re(u,v) & f =1/ p mit p = u 2 + v 2 Direction " Ψ = arctan$ v % # u & ' University of Hamburg, Dept. Informatics 6

7 A g(x, y) Illustrative Example of Fourier Transform x X 2D image function Y y Frequency spectrum Note that large spectral amplitudes occur in directions vertical to prominent edges of the image function! frequency spectrum as an intensity function University of Hamburg, Dept. Informatics 7

8 Examples of Fourier Transform Pairs University of Hamburg, Dept. Informatics 8

9 Example of a Real-world Amplitude Spectrum University of Hamburg, Dept. Informatics 9

FFT (Fast Fourier Transform) is based on recursive decomposition of g mn into subsequences.

10 Fast Fourier-Transformation Ordinary DFT needs ~(MN) 2 operations for an image of size M x N. Example: M = N = 1024, sec/operation à 1,1 s. FFT (Fast Fourier Transform) is based on recursive decomposition of g mn into subsequences. Due to multiple use of partial results à ~ MN log 2 (MN) Operations. Same example with FFT needs only about seconds. The next slides will: introduce the decomposition scheme and give one-dimensional examplesof the FFT University of Hamburg, Dept. Informatics 10

Fast Fourier-Transformation Principle of decomposition for the 1D-DFT (Cooley & Tukey, 1965): For r = 0,..., N-1 and N=2n: G r = 2n 1 g k=0 n e # n 1 % = $ g k=0 k &% n 1 g (1) k k 2π jr 2n e 2π 2k jr!

11 Fast Fourier-Transformation Principle of decomposition for the 1D-DFT (Cooley & Tukey, 1965): For r = 0,..., N-1 and N=2n: G r = 2n 1 g k=0 n e # n 1 % = $ g k=0 k &% n 1 g (1) k k 2π jr 2n e 2π 2k jr! = g (1) k e 2π jr k n!## k=0 " $ # % = $ % & G (1) r 2n + g 2 k+1! e 2π jr g (2 ) k 1 ## + jr e π n G r (1) + e π jr 1 n G r (2) G r (1) e π jr 1 n G r (2) n 1 (2k+1) 2n ' % ( )% g (2) k e 2π jr k n!## k=0 "## $ if r < n if r n G (2 ) r Decomposition in odd/even part Decomposition in frequency space G r = G r (1) + e π jr 1 n G r (2) G r = G r (1) e π jr 1 n G r (2) r = 0,..., n 1 r = n,..., 2n 1 All G r my be computed by (N/2) 2 instead of (N) 2 operations! University of Hamburg, Dept. Informatics 11

12 Cooley & Tukey Decomposition I Example with two values (N=2) G 0 = g 0 + e 2π j 0 g 1 = g 0 + g 1 G 1 = g 0 e 2π j 0 g 1 = g 0 g 1 Graphical representation: g 0 G 0 = g 0 + g 1 positive negative g 1 G 1 = g 0 g University of Hamburg, Dept. Informatics 12

13 Cooley & Tukey Decomposition II g 0 g 1 Example with four values (N=4) + - product G 0 = (g 0 + g 2 )+ (g 1 + g 3 ) G 2 = (g 0 + g 2 ) (g 1 + g 3 ) g 2 g 3 w 0 w 1 G 1 = (g 0 g 2 )w 0 + (g 1 g 3 )w 1 G 3 = (g 0 g 2 )w 0 (g 1 g 3 )w 1 With weights: w 0 = e 2π j 1 4 = e 1 2 π j w 1 = e 2π j 3 4 = e 3 2 π j University of Hamburg, Dept. Informatics 13

14 Cooley & Tukey Dekomposition III Example with eight values (N=8) + - product g 0 G 0 g 1 g 2 g 3 w 0 w 1 G 4 G 2 G 6 g 4 w 0 G 1 g 5 w 1 G 5 g 6 w 2 w 0 G 3 g 7 w 3 w 1 G University of Hamburg, Dept. Informatics 14

15 Convolution Convolutionis an important operationfor describingand analyzinglinear operations, e.g. filtering. Definition of2d convolution for continuous signals: g(x, y) = f (r, s) h(x r, y s) dr ds = f (x, y) h(x, y) Convolutionin the spatial domain is dual to multiplication in the frequency domain: F{ f (x, y) * h(x, y) } = F(u, v) H(u, v) F{ f (x, y) h(x, y) } = F(u, v) * H(u, v) H can be interpreted as attenuatingor amplifyingthe frequencies of F. à Convolutiondescribes filtering in the spatial domain University of Hamburg, Dept. Informatics 15

16 Filtering in the Frequency Domain A filter transforms a signal by modifying its spectrum. G(u, v) = F(u, v) H(u, v) F H G Typical filters: Fourier transform of the signal frequency transfer function of the filter modified Fourier transform of signal low-pass filter lowfrequenciespass, high frequenciesare attenuated or removed high-pass filter high frequenciespass, lowfrequenciesare attenuated or removed band-pass filter frequencies within a frequency band pass, other frequencies below or above are attenuated or removed Often (but not always) the noise part of an image is high-frequency and the signal part is low-frequency. Low-pass filtering then improves the signal-to-noise ratio University of Hamburg, Dept. Informatics 16

17 Filtering in the Spatial Domain Filtering in the spatial domain is described by convolution. g(x, y) = f (r, s) h(x r, y s) dr ds = f (x, y) h(x, y) Commonly used description for the effect of technical components in linear signal theory: s!(t) = + h(r) s(t r) dr s 1 (t) h s 1 (t) s 2 (t) h s 2 (t) a s 1 (t) + b s 2 (t) h a s 1 (t) + b s 2 (t) An impulse δ as input generates the filter function h(x, y) as output: h(x, y) = h(r, s) δ(x r, y s) dr ds = h(x, y) δ(x, y) h(x, y) is often called "impulse response w.r.t. LTI systems University of Hamburg, Dept. Informatics 17

18 Ideal low-pass filter Low-pass Filters H v " $ H(u, v) = # 1 for u 2 + v 2 W u %$ 0 else W All frequencies above W are annihilated Note that the filter function h(x, y) is rotation symmetricand h(r) ~ sinc(2pwr) = sin 2pWr / (2pWr) with r 2 = x 2 + y 2 à impuls-shapedinput structuresmay produce ring-like structures as output Gaussian filter optimallysmooth boundary, both in the frequency and the spatial domain. important for several advanced image analysis methods, e.g. generating multiscale images. H(u, v) = e 1 2 (u2 +v 2 )σ 2 h(x, y) = σ 1 2π e 1 2 x 2 +y 2 σ University of Hamburg, Dept. Informatics 18

19 Discrete Filters For periodicdiscrete2d signals (e.g. discreteimages), the convolution operator which describes filtering is Each pixel g ij of the filtered image is the sum of the products ofthe original image with the mirror filter h -m,-n placed at location ij. M 1 g ij = f m,n m=0 N-1 n=0 h i m, j n Example h mn = h -m,-n is a bell-shaped function, e.g. Gaussian The filtering effect is a smoothing operation by weighted local averaging. The choice of weights of a local filter - the convolution mask - may influence the properties of the output image in important ways, e.g. with regard to remaining noise, blurred edges, artificial structures, preserved or discarded information University of Hamburg, Dept. Informatics 19

20 Matrix Notation for Discrete Filters M 1 N-1 The convolution operation g ij = f m,n h i m, j n m=0 n=0! may be expressed as matrix multiplication: g = H f!!! Vectors g and f are obtained bystackingrows (or columns) onto each other:! g T = ( g 00 g 01! g 0 N 1 g 10 g 11! g 1N 1! g M 1 0 g M 1 1! g M 1 N 1 )! f T = f 00 f 01! f 0 N 1 f 10 f 11! f 1N 1! f M 1 0 f M 1 1! f M 1 N 1 ( ) The filter matrix H is obtained byconstructing a matrix H j for each row j of h ij : " $ $ H j = $ $ $ # h j0 h j N 1 h j N 2! h j 1 h j0 h j N 1! " " " # h 1 N 1 h 1 N 2 h 1 N 3! h j 1 h j 2 " h j0 % ' ' ' ' ' & " $ $ H = $ $ # H 0 H M 1 H M 2! H 1 H 0 H N 1! " " " # H M 1 H N 2 H M 3! H 1 H 2 " H 0 % ' ' ' ' & University of Hamburg, Dept. Informatics 20

Avoiding Wrap-around Errors Wrap-around errorsresultfrom filter responses due to the periodic continuation of image and filter à periodicity (cf. slide 4).

21 Avoiding Wrap-around Errors Wrap-around errorsresultfrom filter responses due to the periodic continuation of image and filter à periodicity (cf. slide 4). To avoid wrap-around errors, image and filter have to be extended e.g. by zeros. A B original image size C D original filter size M N extended image and filter size Example: M A + C - 1 N B + D University of Hamburg, Dept. Informatics 21

22 Discrete Convolution Using the FFT Convolution in the spatial domain maybe performed moreefficiently usingthe FFT. M 1 g ij! = g mn m=0 N-1 n=0 h i m, j n (MN) 2 operations needed Using the FFT andfilteringin the frequencydomain: FFT H uv FFT -1 g mn G uv G uv g mn MN log(mn) MN MN log(mn) # of operations Example with M = N = 512: straight convolution needs ~ operations convolution usingthe FFT needs ~10 7 operations University of Hamburg, Dept. Informatics 22

23 Convolution and Correlation The crosscorrelation function of 2 stationary stochastic processes f and h is: g(x, y) = f (r, s) h(r x, s y) dr ds = f (x, y)! h(x, y) = f (x, y) h( x, y) Compare with convolution: filter function is not mirrored! CorrelationusingFourier Transform: F{ f(x, y) o h(x, y) } = F*(u, v) H(u, v) F{ f*(x, y) h(x, y) } = F(u, v) o H(u, v) F*, f* are complex conjugates Correlation is particularly important for matching problems, e.g. matching an image with a template. Correlation maybecomputedmoreefficiently byusingthe FFT University of Hamburg, Dept. Informatics 23

Correlation and Matching Matching a template with an image: image template find degree of match for all locations of template find location of best match For

j): Sinceimageenergy andtemplate energy are constant, correlation measures distance M 1 N-1 ( ) 2 d ij = f mn h m i.

24 Correlation and Matching Matching a template with an image: image template find degree of match for all locations of template find location of best match For (periodic) discrete images, crosscorrelation at (i, j) is M 1 N-1 c ij = f mn m=0 n=0 h m i,n j Compare with Euclidean distance between f and h at location (i, j): Sinceimageenergy andtemplate energy are constant, correlation measures distance M 1 N-1 ( ) 2 d ij = f mn h m i.n j m=0 M 1 n=0 N-1 ( ) 2 = f mn 2 f mn h m i.n j + h m i.n j! m=0 # n=0 "## $! m=0 ### n=0 "### $! m=0## n=0 "## $ Energy University of Hamburg, Dept. Informatics M 1 N-1 c ij M 1 N-1 ( ) 2 Energy 24

25 Principle of Image Restoration Typical degradationmodel ofa continuous 1-dimensional signal: g(t) h(t) z(t) + g (t) g(t) h(t) z(t) g (t) original signal degrading filter additive noise degraded signal How can one process g (t) to obtain a g (t) which best approximates g(t)? Note that a perfect restoration g (t) = g(t) may not be possible even if z(t) = 0.? r(t) restoring filter g (t) r(t) g (t) g (t) restored signal The ideal restoring filter H (f) = 1/H(f) may not exist because of zeros of H(f). H(f) G(f) G (f) H (f) G (f) University of Hamburg, Dept. Informatics 25

26 Image Restoration by Minimizing the MSE!! Degradation in matrix notation:! g! = H g + z Restored signal g must minimizethe mean squareerror J(g ) of the remainingdifference: min g!! H g!!! 2 J( g! "" )!!! = 2H T g" H g"" g"" ( ) = 0 g! "" = ( H T H ) 1! H T g" pseudoinverse of H If H is a squarematrix, and if H -1 exists, we can simplify: The matrix H -1 gives a perfect restorationif z = 0.! g!! = H 1! g! University of Hamburg, Dept. Informatics 26

27 Discrete Convolution of Masked Images Scenario: Greyvalues exist onlyfor a partial domain ofthe image function Examples: Cloud coverage in aerial and satellite images, Segmented areas Sensor malfunctions Problems arise at the boundaryofthe convolution kernel (Titmarsh 1926). In these areas, discrete convolutionresults in undesiredeffects. Easy fixes suffer from additional problems: 1. Exclude the boudary areas à (Strong) reduction of the resulting image space! 2. Set to zero à Errorneous values are introduced! If iterative algorithms are used, errors may also be propagatedand enhanced! Wanted: An approach, which treats masked pixel as no information instead of no intensity! University of Hamburg, Dept. Informatics 27

28 Normalized Convolution I Approach of Knutsson und Westin (1993): # % I ' = K * I I ' = K * M,A I = $ % & A K *(I M ) if A K * M 0 A K * M 0 else. with: K I A M Convolution kernel Image Applicable kernel Image mask Example: I = {250, 225, 200, 175, 150, 125, 100, 75, 50, 25, 0} M = {1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0} K = {1, 2, 4, 8, 4, 2, 1}/ 22 A =1 K * I = {,,, 175, 150, 125, 100, 75,,, } K *(I M ) = {,,, , , 52.27, 21.59, 6.82,,, } K * M,A I = {,,, , , , , 150,,, } University of Hamburg, Dept. Informatics 28

29 IP1 - Lecture 7: Spectral Image Processing and Convolution Normalized Convolution II Example of Knutsson und Westin 10% Sampling A=1 K*I K: 2D discrete Gaussian mit σ=2, Größe:17x K *M,A I A locally restricted University of Hamburg, Dept. Informatics 29

30 Summary Normalized Convolution III Combines mask and discrete convolution, Executionspeed may beenhancedby the use offft Derives results for masked areas if at least one non-masked pixel exists within the current neighborhood à may also be used for reconstruction purpose Provides a base to extend generic algorithms to with with masked images in a clearly defined way. Restricted to certain convolution kernels (non-zero power)! Differential convolution kernels require an additional normalized convolution à Normalized differential convolution University of Hamburg, Dept. Informatics 30

Empirical Mean and Variance!

Empirical Mean and Variance! Global Image Properties! Global image properties refer to an image as a whole rather than components. Computation of global image properties is often required for image enhancement, preceding image analysis.!