Problem Session #5 EE368/CS232 Digital Image Processing
1. Solving a Jigsaw Puzzle Please download the image hw5_puzzle_pieces.jpg from the handouts webpage, which shows the pieces of a jigsaw puzzle. Since we know that the completed puzzle should show a picture of the famous Mona Lisa painting, we can download an image of the painting from the web to use as a reference when solving the puzzle; you can download hw5_puzzle_reference.jpg from the handouts webpage.
Step 1: We binarize the pieces image, with the result shown below. Step 2: We extract an edge map from the binary image (result shown below), which we will use later to draw the boundaries of the jigsaw pieces on top of the reference image.
Step 3: When the jigsaw pieces are fully assembled, the assembled image may have a different size than the reference image. To account for this mismatch, we resize the reference image (maintaining the same aspect ratio) to have approximately the same number of pixels as the total number of white pixels in the binary pieces image. Step 4: We perform region labeling on the binary pieces image.
Step 5: For each region in the binary pieces image, we compare the corresponding region in the original color pieces image to different regions in the reference image. Note that the regions being compared are non-rectangular in shape, because we utilize the regional shapes obtained from the segmentation in Step 1. We find the region in the reference image that best matches (has the smallest mean squared difference relative to) each piece region. Reference Template + - Mask x Final Difference Mask
Step 6: For the matching region in the reference image, we (i) label that region with the same number that identifies the jigsaw piece (ii) overlay the boundary of the jigsaw piece using the edge map from Step 2 + EOP
2. Eigenfaces and Fisherfaces for Recognition of Glasses In this problem, we will study the use of eigenfaces and fisherfaces for recognizing glasses in images. Please download the following files from the homework webpage: hw5_non_glasses_training.zip: 153 training images of people not wearing glasses hw5_glasses_training.zip: 67 training images of people wearing glasses hw5_non_glasses_testing.zip: 128 testing images of people not wearing glasses hw5_glasses_testing.zip: 52 testing images of people wearing glasses
Part A: Compute, display, and submit the mean face image over the training set. Mean Face
Part B: Using the Sirovich and Kirby method on the training set, compute the 100 eigenfaces with the largest eigenvalues. Please use mean-removed training samples to compute the eigenfaces and make sure each eigenface has unit L2 norm. Display and submit the top 10 eigenfaces. (MATLAB function: eig) Eigenface 1 Eigenface 2 Eigenface 3 Eigenface 4 Eigenface 5 Eigenface 6 Eigenface 7 Eigenface 8 Eigenface 9 Eigenface 10
Part C: Pick one of the eigenfaces from the top 10 eigenfaces which seems to accentuate features around the eyes. Create a 1-d classifier which distinguishes between non-glasses and glasses images by choosing a threshold for the eigenface coefficient that minimizes the error rate for the training images. Report the overall error rate on the training set. Then, evaluate the classifier on the testing images, and report the numbers of non-glasses images and glasses images that are correctly classified, as well as the overall error rate on the testing set. Eigenface 60 50 Non-glasses Glasses Frequency 40 30 20 T = 0.75 Eigenface 10 10 0-5 0 5 Transform Coefficient
Part D: Using the training set, compute, display, and submit the fisherface for recognizing glasses. Perform Fisher LDA in a 100-dimensional eigenface space (using the top 100 eigenfaces computed previously) and make sure the fisherface has unit L2 norm. Repeat part (c), except replace the single eigenface by the fisherface. Does the fisherface obtain a lower overall error rate on the testing set than a single eigenface? (MATLAB function: eig) Image Perform PCA 100 PCA Coefficients Perform Fisher LDA Fisher LDA Coefficient
60 50 Non-glasses Glasses Fisherface Frequency 40 30 20 T = 0.45 Fisherface 10 0-5 0 5 Transform Coefficient
3. Frequency Responses of Edge Templates Part A: Derive expressions for the frequency responses of the horizontal and vertical central difference, Prewitt, and Sobel edge templates and of the two diagonal Roberts edge templates. For each template, submit a plot of the magnitude of the frequency response. Which frequency components are boosted and which frequency components are attenuated by each edge template? (MATLAB functions: meshgrid, mesh, surf, abs)
Central Difference Operator h hor [x, y] = 1 0 h ver [x, y] = 1 [0] 1 [ ] 1 H hor H ver e jω x,e jω y ( ) = e jω x e jω x = 2 j sin ( ω x ) e jω x,e jω y ( ) = 2 j sin( ω y )
Part B: In some edge detection algorithms, e.g., the Canny edge detector, convolution with the edge template is combined with convolution with a Gaussian filter for noise suppression. Generate a Gaussian impulse response (function: fspecial) of standard deviation σ = 1 and finite extent (6σ +1) (6σ +1). Compute the Gaussian filter s frequency response (function: freqz2) using the sample frequency-domain sampling grid that you used in Part A. For each edge template, submit a plot of the magnitude of the product of the Gaussian filter s frequency and the edge template filter s frequency response.!
Without Gaussian Prefilter With Gaussian Prefilter
4. Document Image Restoration Please download the image hw5_insurance_form.jpg from the homework webpage. During faxing, the form has suffered some image distortions: it is slightly rotated and some of the table s horizontal and vertical lines have been erased. Design and implement an algorithm to automatically rotate the form to an upright orientation and repair the gaps in the broken lines. Clearly describe the steps in your algorithm, showing intermediate results for clarification if necessary. Display and submit the final restored image.
Binarization with Otsu s Method
Hough Transform MATLAB: hough, houghpeaks Orientation Histogram for Peaks in Hough Transform
Rotated to Upright Orientation MATLAB: imrotate
Erosion by 35-pixel Vertical Line MATLAB: imerode
Closing by 201-pixel Vertical Line MATLAB: imclose
Restoration of Vertical Lines in Table
Erosion by 49-pixel Horizontal Line MATLAB: imerode
Closing by 301-pixel Horizontal Line MATLAB: imclose
Restoration of Horizontal Lines in Table
Final Restoration Result EOP