Midterm Summary Fall 8 Yao Wang Polytechnic University, Brooklyn, NY 2
Components in Digital Image Processing Output are images Input Image Color Color image image processing Image Image restoration Image Image enhancement Image Image acquisition Wavelets and and Multiresolution processing Knowledge base base Compression Morphological processing Segmentation Representation & description Object Object recognition Output are image attributes Yao Wang, NYU-Poly EL523: Midterm Summary 2
Topics Covered Image representation Color representation Quantization Contrast enhancement Spatial Filtering: noise removal, sharpening, edge detection Frequency domain representations FT, DTFT, DFT Implementation o linear iltering using DTFT and DFT Yao Wang, NYU-Poly EL523: Midterm Summary 3
Color Image Representation Light is the visible band o the EM wave Color hue o a light depends on the wavelength o the EM wave Illuminating light vs. relecting light Color attributes: Brightness luminance Hue depends on the wavelength Saturation purity Any color can be reproduced by mixing three primary colors Illuminating and relecting light sources ollow dierent mixing rules Color can be represented in many dierent coordinates or models with 3 components Yao Wang, NYU-Poly EL523: Midterm Summary 4
Tri-chromatic Color Mixing Tri-chromatic color mixing theory Any color can be obtained by mixing three primary colors with a right proportion Primary colors or illuminating sources: Red, Green, Blue RGB Color monitor works by exciting red, green, blue phosphors using separate electronic guns ollows additive rule: R+G+BWhite Primary colors or relecting sources also known as secondary colors: Cyan, Magenta, Yellow CMY Color printer works by using cyan, magenta, yellow and black CMYK dyes ollows subtractive rule: R+G+BBlack Yao Wang, NYU-Poly EL523: Midterm Summary 5
Color Models Speciy three primary or secondary colors Red, Green, Blue. Cyan, Magenta, Yellow. Speciy the luminance and chrominance HSB or HSI Hue, saturation, and brightness or intensity YIQ used in NTSC color TV YCbCr used in digital color TV Amplitude speciication: 8 bits per color component, or 24 bits per pixel Total o 6 million colors A kxk true RGB color requires 3 MB memory Yao Wang, NYU-Poly EL523: Midterm Summary 6
Color Image Processing Apply contrast enhancement or iltering linear or non-linear to each primary color component independently using the techniques or monochrome images May change the color hue o the original image Convert the tri-stimulus representation into a luminance / chrominance representation, and modiy the luminance component only. For certain applications, dierent operations may be applied to dierent color components either in RGB domain or in YCC2 domain to obtain special desired eects Yao Wang, NYU-Poly EL523: Midterm Summary 7
Pseudo Color Processing Given a gray-scale image, we may display it in pseudo colors to better reveal certain eatures Color depends on image pixel value Oten used in medical image display, weather maps, vegetation maps Decompose an image into dierent requency bands and represent each band with dierent colors Yao Wang, NYU-Poly EL523: Midterm Summary 8
Quantization Q Quantizer Q r 8 Decision Levels {t k, k,, L+} Reconstruction Levels {r k, k,, L} I t k, t [ k + Then Q r k t t 2 t 3 t 4 t 5 t 6 t 7 t 8 L levels need bits x R log2 returns the smallest integer that is bigger than or equal to x L r 7 r 6 r 5 r 4 r 3 r 2 r Quantizer error r r r 2 r 3 r 4 r 5 r 6 r 7 r 8 r t t 2 t 3 t 4 t 5 t 6 t 7 t 8 Yao Wang, NYU-Poly EL523: Midterm Summary 9
Uniorm Quantization Equal distances between adjacent decision levels and between adjacent reconstruction levels t l t l- r l r l- q Parameters o Uniorm Quantization L: levels L 2 R B: dynamic range B max min q: quantization interval step size q B/L B2 -R Yao Wang, NYU-Poly EL523: Midterm Summary
Uniorm Quantization: Functional Representation Q stepsize q max - min /L r 7 max -q/2 r 6 q Q + q 2 min q + min r 5 r 3 r 4 x returns the biggest integer that is smaller than or equal to x r 2 r r min +q/2 t t t 2 t 3 t 4 t 5 t 6 t 7 t 8 min max min I is called the reconstruction level index, which indicates which reconstruction level is used or. q Yao Wang, NYU-Poly EL523: Midterm Summary
Contrast Enhancement An requently used, important operation How to tell the contrast o an image rom its histogram? Given a histogram, can sketch a transormation that will enhance the contrast Histogram equalization Histogram speciication Yao Wang, NYU-Poly EL523: Midterm Summary 2
Histogram vs. Contrast p p p a Too dark b Too bright c Well balanced Yao Wang, NYU-Poly EL523: Midterm Summary 3
Enhancement o Too-Dark Images H g g max New histogram max Hg Original histogram max Transormation unction: log unction: gc log + Or Power law: g c ^r, <r< g max g Yao Wang, NYU-Poly EL523: Midterm Summary 4
Enhancement o Too-Bright Images H g g max New histogram max Hg Original histogram max Transormation unction: Power law: gc ^r, r> g max g Yao Wang, NYU-Poly EL523: Midterm Summary 5
Enhancement o Images Centered near the Middle Range H g g max New histogram max Hg Original histogram max Transormation unction g max g Yao Wang, NYU-Poly EL523: Midterm Summary 6
Histogram Equalization Transorms an image with an arbitrary histogram to one with a lat histogram Suppose has PDF p F, Transorm unction continuous version g g is uniormly distributed in, pf t dt Histogram Equalization Yao Wang, NYU-Poly EL523: Midterm Summary 7
Histogram Speciication What i the desired histogram is not lat? p F Histogram Equalization g s Histogram Equalization z p Z z p G g g g P S s s s z Speciied Histogram p F d z p Z z dz z s g s g Yao Wang, NYU-Poly EL523: Midterm Summary 8
Image Filtering Applications: Noise removal image smoothing, low-pass iltering Edge enhancement deblurring, high-emphasis Edge detection high-pass iltering Contrast enhancement is accomplished by point operation, i.e., each pixel value is changed based on its original value, not its neighboring pixels but the transormation unction depends on the overall histogram o the image Image iltering reers to changing the color value o one pixel based on the color values o this pixel and its neighbors Yao Wang, NYU-Poly EL523: Midterm Summary 9
Three Ways o Implementing Linear Filtering Spatial domain Weighted average o adjacent pixels linear convolution Weights or the ilter depends on the desired iltering eect Frequency domain FT, DTFT Design spatial ilter based on desired requency response Low pass, high pass, high emphasis Convolution theorem: h* H F Frequency domain DFT Perorm iltering in DFT domain Convolution theorem: circulation convolution h@ H F Relation between circulation convolution and linear convolution Filter masks in the DFT domain must be designed properly so that the corresponding ilter in the spatial domain is real The ilter mask should enjoy the symmetry property o real signals Yao Wang, NYU-Poly EL523: Midterm Summary 2
Yao Wang, NYU-Poly EL523: Midterm Summary 2 Linear Convolution o Continuous Signals D convolution Equalities 2D convolution α α α α α α d x h d h x x h x, x x x x x x x δ δ β α β α β α β α β α β α d d y x h d d h y x y x h y x,,,,,,
Yao Wang, NYU-Poly EL523: Midterm Summary 22 Linear Convolution o Discrete Signals D convolution 2D convolution Separable iltering Row irst, then columns m m m n h m m h m n n h n * k l k l l n k m h l k l k h l n k m n m h n m,,,,, *,
Interpretation as weighted average o neighboring samples How should we design the weights? Smoothing: sum, coe > High emphasis: sum, some coe < Edge detection high pass: sum Yao Wang, NYU-Poly EL523: Midterm Summary 23
Ex: Smoothing by Averaging Replace each pixel by the average o pixels in a square window surrounding this pixel g m, n 9 + + m, n + m, n + m +, n + m, n + m, n + m, n + m +, n + m, n + m +, n + Trade-o between noise removal and detail preserving: Larger window -> can remove noise more eectively, but also blur the details/edges Yao Wang, NYU-Poly EL523: Midterm Summary 24
Yao Wang, NYU-Poly EL523: Midterm Summary 25 Directional Edge Detector High pass in one direction and low pass in the orthogonal direction Prewitt edge detector Sobel edge detector [ ] [ ] 3 3 ; 3 3 y H x H [ ] [ ] 2 4 2 2 4 ; 2 4 2 2 4 y H x H
Special Considerations in Implementation Boundary treatment I is MxN, h is KxL, convolved image g is M+K- xn+l- Instead o expanding the image size, we modiy iltering operations at the boundary Symmetric expansion Leave the boundary pixels unchanged Renormalization Filtered values may not be integer, and may have negative values, may have a smaller or larger dynamic range than original To save resulting image in an 8-bit unsigned char ormat, we normalize all values to -255 g g-g_min/g_max-g_min * 255 Yao Wang, NYU-Poly EL523: Midterm Summary 26
D Fourier Transorm For Discrete Time Sequence DTFT Forward Transorm Inverse Transorm j2πun n e n / 2 j2πun n F u e du / 2 F u 2D Forward Transorm F u, v m n m, n e j2π mu+ nv Inverse Transorm / 2 / 2 j2π mu+ nv m, n F u, v e dudv / 2 / 2 Separable implementation Transorm each row irst with D DTFT, then each column Yao Wang, NYU-Poly EL523: Midterm Summary 27
Design Filters Based on Desired Frequency Response Convolution theorem x, y* h x, y F u, v H u, v Filter hx,y designed based on desired requency response Hu,v Sharp transitions in requency domain very long ilters in spatial domain Apply a window unction to smooth the transition band shorter ilters Giver a spatial ilter, we can use DTFT to better understand its iltering eect requency response Separable ilters Design horizontal and vertical ilters separately For images, we usually use very short ilters Yao Wang, NYU-Poly EL523: Midterm Summary 28
Discrete Fourier Transorm DFT: I DTFT or Finite Duration Signals the signal is only deined or n Fourier transorm becomes : F' N n nexp,,..., N j2πn,, : Sampling F' at Forward transorm DFT : N k F k F' N N n Inverse transorm IDFT : n N N k k/n, k k F kexp j2π N,,...,N-, and rescaling yields: nexp n, k j2π N n, n,,..., N k,,..., N Yao Wang, NYU-Poly EL523: Midterm Summary 29
Circular Convolution Theorem Circular convolution n h n N k n k N h k n-k N n hn 2 3 4 n 2 3 4 n n n- Convolution theorem n h n F k H k n+ n+2 n+4 n+3 k Yao Wang, NYU-Poly EL523: Midterm Summary 3
2D Discrete Fourier Transorm Deinition Assuming m, n, m,,, M-, n,,, N-, is a inite length 2D sequence F k, l m, n MN MN M N m n M N k l m, n e F k, l e Comparing to DTFT F u, v m n m, n e km ln j2π + M N km ln j2π + M N,, j2π mu+ nv k,,..., M m,,..., M, u k M, l, n, v / 2 / 2 j2π mu+ nv m, n F u, v e dudv / 2 / 2 l N,,..., N,,..., N ;. Yao Wang, NYU-Poly EL523: Midterm Summary 3
Periodicity Property o 2D DFT Periodicity F k, l F k, l, k < or k M, l < or l N. M N -M -M/2 A C B D A C B D Low Frequency k M/2 M -N A C B D,,N- A C M-, B D M-,N- -N/2 N/2 N l High Frequency Yao Wang, NYU-Poly EL523: Midterm Summary 32
Calculate Linear Convolution Using DFT D case n is length N, hn is length N 2 gn n*hh is length N N +N 2 -. To use DFT, need to extend n and hn to length N by zero padding. n * hn Convolution gn DFT DFT DFT Fk x Hk Multiplication Gk Yao Wang, NYU-Poly EL523: Midterm Summary 33
Midterm Exam Logistics Scheduled time: /5 3-5:4, LC2 Closed-book, sheet o notes allowed double sided OK Oice hour by the Grader Zhi Zhao, Tuesday /4-2AM Wed. /5-2 AM LC22 Zhi Zhao [zhaozhi429@yahoo.com.cn] Yao Wang, NYU-Poly EL523: Midterm Summary 34