Final Exam Due on Sunday 05/06 The exam should be completed individually without collaboration. However, you are permitted to consult with the textbooks, notes, slides and even internet resources. If you need hints, please send the instructor emails or come by the office for discussions. Problems are equally weighted. Problem 1 Terms and Definitions Briefly answer the following questions. 1. What are camera s intrinsic and extrinsic (calibration) parameters? 2. What are epipoles and epipolar lines? 3. Explain why the rank of the fundamental matrix is two. 4. What is the Rayleigh quotient? How does it associate with the generalized eigenvalue problem? 5. What is disparity? And what are the epipolar and continuity constraints in stereo correspondence? Problem 2 LevelSet and Segmentation We discussed image segmentation using active contour in class. In this problem, we will work out the mathematics in some details. The main idea is to define a speed function on the image using information extracted from the image and the geometry of the contour. Starting with an initial contour, the speed function F tells us how to evolve the contour in order to get to the target. For example, suppose a contour c = c(t) is given with t the parameter. If p denotes a point on the contour, F (p) gives the instantaneous speed in the normal direction to the contour at that point. That is, if n is the unit normal of the contour at p and V (p) is the velocity of the point p, F (p) = V (p) n. The idea of the level-set based technique is to embed the contour c as the zero level-set of a function Φ(x, y, t) = Φ t (x, y). At any time t = T, the contour is given as c = Φ 1 t=t (0), i.e., the points at which Φ t=t is zero. 1 For example, let Φ t (x, y) = x 2 + y 2 + t 100. What are the zero level-sets of Φ at t = 0, 19 and 36? 2 Determine the speed function F for the example above. In the example above, we are given the values of Φ at every t. In general, only the values of Φ at t = 0 are given, and its values at other time instances are determined by solving a partial differential equation, which we will derive. 3 Let c α denote the contour Φ 1 α (0). Show that the unit normal vector at a point p on the contour is given by Φ α Φ α. 4 Show that the partial differential equation for Φ is given by Φ t + F Φ = 0. 1
This is a straightforward problem: take the total derivatives of Φ(x(t), y(t), t) with respect to t and use (3) above. See the figure. Figure 1: The evolving contour at two different times t = a, b. One point p on the contour is marked and the blue path is the path it travelled between t = a and t = b. Let (x(t), y(t)) denote this path. We have Φ(x(t), y(t), t) = 0 for all t (Why?). Of course, (x (t), y (t)) gives the velocity (not speed!) of the point p at time t. Problem 3 Epipolar Geometry Recall that a camera matrix P is a 3 4 matrix. For a point x given in the homogeneous coordinates x = [x 1 x 2 x 3 x 4 ] t, the corresponding point (again in homogeneous coordinates) on the image is given by P x. One important and useful result that we did not discuss in details is how to find a correct pair of camera matrices given a fundamental matrix F. This result is important because it allows us to start the projective reconstruction (See class slides). 1. We know that the fundamental matrix is a 3 3 matrix with rank two. This implies that its determinant is zero. What is the degree of freedom of a fundamental matrix? 2. Briefly describe how to compute the fundamental matrix using pairs of corresponding pixels (image points). 3. A 4 4 matrix S is skew-symmetric if and only if for every 4 1 vector X, X t SX = 0. Show that a non-zero matrix F is the fundamental matrix corresponding to a pair of camera matrices P 1, P 2 if and only if P t 2F P 1 is skew-symmetric. (This is a very simple problem. Argue geometrically from the fact that every pair of corresponding image points arises from a 3D point, which is expressed in homogeneous coordinates.) 4. Let (P 1, P 2 ) and (P 1, P 2) be two pairs of camera matrices such that P 1 = P 1 H, P 2 = P 2 H for some 4 4 nonsingular matrix H. Show that the fundamental matrices corresponding to these two pairs of camera matrices are the same. 5. Let P = K[R t] denote a camera matrix, where K and R are nonsingular 3 3 matrices that give the intrinsic camera parameters and the rotation (with respect to some world frame), respectively. t 2
is the translational component. Show that there is a nonsingular 4 4 matrix H that puts P into the standard form P H = [I 0], where I is the 3 3 identity matrix. 6. Let P 1 = [I 0] and P 2 = [M m] be two camera matrices, where M is a 3 3 (nonsingular) matrix. Show that their fundamental matrix is given by the formula F = [m] M, where [m] is the matrix [m] = 0 m z m y m z 0 m x m y m x 0 Notice that given any vector u, [m] u is the cross product between m and u. straightforward problem. See the slides.). (Again, this is a 7. Given F in the form above. What can you say about the last column m of P 2? (Hint: It is one of the two epipoles. Which one?) 8. Finally, let e denote the epipole determined above. If F denotes a given fundamental matrix, show that a correct pair of camera matrices compatible with F is given by P 1 = [I 0], P 2 = [[e] F e]. (You have to use (3) above.) Problem 4 Shape and Illumination In this problem, we will prove a simplified version of the following result: Given two images I and J, and two linearly independent vectors s, l IR 3, there exists a Lambertian surface S such that the images of S taken under lighting conditions (point sources at infinity) specified by s and l are I and J, respectively. A direct and (rather) counterintuitive consequence of this result is that given two images, it is not possible with absolute certainty to determine whether they are created by the same or different objects! A. Recall that for a Lambertian surface, the radiance (pixel intensity) I at each surface point under a point source at infinity is given by the formula I = ρ max{ L ˆn, 0}, where ρ is the albedo, L is the lighting direction and strength and ˆn is the unit normal vector at the given surface point. We assume infinitesimal pixels; therefore, an image is considered as a nonnegative function defined on a rectangular domain in IR 2, and the two images have the same domain. For this problem, you need to come up with a surface z = f(x, y) for some smooth function f(x, y) that will account for the two given images taken under the two lighting conditions. You have to describe a method (or procedure) that computes f(x, y) given I, J, s and l. For simplicity, you can assume the followings: 1. Intensity is given by the simplified formula I = ρ L ˆn. That is, we don t consider the backside of the surface. 3
2. The two vectors l, s are [1, 0, 1] t and [ 1, 0, 1] t, respectively. The general case follows from a coordinate transform. 3. You do not need to consider the shadow cast by the object onto itself. 4. The two images do not vanish simultaneously, I(x, y) + J(x, y) > 0 for all x, y. 5. Given an ordinary differential equation (ODE) and an one-parameter family of smooth initial conditions (with smooth functions g and h) dz(x) dx = g(x, y), z(0) = h(y), there exists a smooth function Z(x, y) such that Z(0, y) = h(y) and for each fixed y, Z(x, y) is the solution of the above ODE. Hint: The problem statement looks daunting at first, but the problem is not difficult. See the figure. Find an ODE that does the reconstruction over one scanline and patch these curves together using (5) to yield the surface z = f(x, y). The main point is to come up with the correct ODE and argue with it. Use these two equations I(x, y) = ρ(x, y) s ˆn(x, y) = ŝ n(x, y), J(x, y) = ρ(x, y) l ˆn(x, y) = ˆl n(x, y), to figure out the correct tangent vector along each scanline. Recall also that the tangent vector of a curve c(t) = (x(t), y(t), z(t)) is c (t) = (x (t), y (t), z (t)). B. Is there an analogous result for three arbitrary images? Explain. Figure 2: Find an ordinary differential equation that computes the curve c(t) over one horizonal scanline. 4
Problem 5 Principal Component Analysis (PCA) Principal Component Analysis is so ubiquitous these days that I feel obliged to include it. For simplicity, we will work in IR 2. Let p 1,, p K be K data points in IR 2 centered at the origin. We want to find a line L passing through the origin that best approximates the data points, i.e., the line L minimizes the following error function, K E(L) = dist(l, p i ) 2, where dist is the distance between the line L and p i. The line L can be represented by a unit vector u parallels to it. Using Pythagorean theorem, the above sum of squares becomes K K K E(L) = p t ip i u t p i p t i u = p t ip i u t S u, where S = K p ip t i is the covariance matrix. 1. For each data point p i, show that Tr(p i p t i ) = p i 2, where Tr is the trace of the matrix. 2. Show that Tr(S) is K pt i p i. Note that S is symmetric and positive definite (assuming the data is not degenerated); therefore, it has two positive eigenvalues. 3. Show that u ( and L) can be determined by taking the (unit) eigenvector of S corresponding to the largest eigenvalue. 4. Let L be the line (and u the corresponding unit vector) determined above. What is E(L)? Hint: Use the results above, and in case you forgot an important property of Tr that is needed here, the following website should be helpful, http://en.wikipedia.org/wiki/trace_%28linear_algebra%29 5. Suppose now that we know the line L is given by the equation x y = 0. Furthermore, we also know that K pt i p i = 2 and E(L) = 1 2. What is the covariance matrix S? 5