L. Vandenberghe ECE236B Spring Homework 1. ((u i u c ) 2 +(v i v c ) 2 R 2 ) 2 i=1

Transcription

1 L. Vandenberghe ECE6B Spring 8 Homework. Least squares fit of a circle to points. In this problem we use least squares to fit a circle to given points (u i,v i ) in a plane, as shown in the figure. The variables (u c,v c ) denote the center of the circle and R its radius. A point (u,v) is on the circle if (u u c ) +(v v c ) = R. We formulate the fitting problem as an optimization problem m ((u i u c ) +(v i v c ) R ) i= with three variables u c, v c, R. (a) Show that the problem can be written as a linear least squares problem if we make a change of variables and use as variables Ax b () x = u c, x = v c, x = u c +v c R. (b) Usethenormalequations(optimalityconditions)A T Ax = A T boftheleastsquares problem to show that the optimal solution ˆx of the least squares problem satisfies This is necessary to compute R = ˆx + ˆx ˆx. ˆx + ˆx ˆx from ˆx.

2 (c) Test your formulation on the problem data in the file circlefit.m on the course website. The commands circlefit; plot(u, v, o ); axis equal will create a plot of the m = 5 points (u i,v i ) in the figure. Use the MATLAB command x = A \ b to solve the least squares problem ().. Let X be a symmetric matrix partitioned as X = A B B T C ]. () If A is nonsingular, the matrix S = C B T A B is called the Schur complement of A in X. It can be shown that if A is positive definite, then X (X is positive semidefinite) if and only if S (see page 65 of the textbook). In this exercise we prove the extension of this result to singular A mentioned on page 65 of the textbook. (a) Suppose A = in (). Show that X if and only if B = and C. (b) Let A be a symmetric n n matrix with eigenvalue decomposition A = QΛQ T, where Q is orthogonal (Q T Q = QQ T = I) and Λ = diag(λ,λ,...,λ n ). Assume the first r eigenvalues λ i are nonzero and λ r+ = = λ n =. Partition Q and Λ as Q = ] ] Λ Q Q, Λ = with Q of size n r, Q of size n (n r), and Λ = diag(λ,...,λ r ). The matrix A = Q Λ Q T is called the pseudo-inverse of A. Verify that AA = A A = Q Q T, I AA = I A A = Q Q T. The matrix-vector product AA x = Q Q T x is the orthogonal projection of the vector x on the range of A. The matrix-vector product (I AA )x = Q Q T x is the projection on the nullspace. (c) Show that the block matrix X in () is positive semidefinite if and only if A, (I AA )B =, C B T A B.

3 (The second condition means that the columns of B are in the range of A.) Hint. Let A = QΛQ T be the eigenvalue decomposition of A. The matrix X in () is positive semidefinite if and only if the matrix Q T I ] A B B T C ] Q I ] = Λ B T Q Q T ] B C is positive semidefinite. Use the observation in part(a) and the Schur complement characterization for positive definite block matrices to show the result.. This problem is an introduction to the MATLAB software package CVX that will be used in the course. CVX can be downloaded from We consider the illumination problem of lecture. We take I des = and p max =, so the problem is f (p) = max k=,...,n log(at kp) () subject to p j, j =,...,m, with variable p R m. As mentioned in the lecture, the problem is equivalent to k=,...,n h(at kp) subject to p j, j =,...,m, (4) where h(u) = max{u,/u} for u >. The function h, shown in the figure below, is nonlinear, nondifferentiable, and convex. 5 4 h(u) 4 u To see the equivalence between () and (4), we note that f (p) = max k=,...,n log(at kp)

4 = max k=,...,n max{log(at kp),log(/a T kp)} = log max k=,...,n max{at kp,/a T kp} = log max k=,...,n h(at kp), and since the logarithm is a monotonically increasing function, minimizing f is equivalent to minimizing max k=,...,n h(a T kp). The problem data are given in the file illum_data.m posted on the course website. Executing this file in MATLAB creates the n m-matrix A (which has rows a T k). There are lamps (m = ) and patches (n = ). Use the following methods to compute four approximate solutions and the exact solution, and compare the answers (the vectors p and the corresponding values of f (p)). (a) Equal lamp powers. Take p j = γ for j =,...,m. Plot f (p) versus γ over the interval, ]. Graphically determine the optimal value of γ, and the associated objective value. The objective function f (p) can be evaluated in MATLAB as max(abs(log(a*p))). (b) Least-squares with saturation. Solve the least squares problem n (a T kp ) = Ap. k= If the solution has negative coefficients, set them to zero; if some coefficients are greater than, set them to. Use the MATLAB command x = A \ b to solve a least squares problem ( Ax b ). (c) Regularized least squares. Solve the regularized least squares problem n m (a T kp ) +ρ (p j.5) = Ap +ρ p (/), k= j= where ρ > is a parameter. Increase ρ until all coefficients of p are in the interval,]. (d) Chebyshev approximation. Solve the problem k=,...,n at kp = Ap subject to p j, j =,...,m. We can think of this problem as obtained by approximating the nonlinear function h(u) by a piecewise-linear function u +. As shown in the figure below, this is a good approximation around u =. This problem can be converted to a linear program and solved using the MATLAB function linprog. It can also be solved directly in CVX, using the expression norm(a*p -, inf) to specify the cost function. 4

5 4.5 u (e) Exact solution. Finally, use CVX to solve.5.5 k=,...,n max(at kp,/a T kp) subject to p j, j =,...,m. Use the CVX function inv_pos() to expresss the function f(x) = /x with domain R ++. u 5