Homework Set #7 - Solutions

Transcription

1 EE 5 - Applications of Convex Optimization in Signal Processing and Communications Dr Andre kacenko, JPL hird erm - First note that we can express f(t) as Homework Set #7 - Solutions f(t) = c(t) x, where we have [ c(t) ( ) π () t cos ( ) πkt cos x [ a a b b K ] ( ) π () t sin ( ) ] πkt sin, o compute the L approximation, we can equivalently minimize the L -norm of the error his leads to the following f y = / / ( c(t) x y(t)) dt [ ] [ / / = x c(t) c(t) dt x / } {{ } P c(t) y(t) / ] } {{ } q / + / y (t) dt } {{ } r Note that by the orthogonality of the Fourier series basis functions, we have that [ ] P = / K, () K I K and hence P hus, the optimal choice of x for the L approximation problem is given by x = P q (3) his can be simplified further as follows From our expression for P given in (), it is clear that we have [ ] P = K (4) K I K () Also, note that from (), we have / y(t) dt, k = / / ( ) π (k ) t [q] k = y(t) cos dt, k =,, K + / / ( ) π (k K ) t y(t) sin dt, k = K +,, K + /

2 From this, it can be shown that we have the following for our choice of y(t) 4, ( ) k = [q] k = π (k ) (k ) π sin, k =,, K +, k = K +,, K + his can be further simplified to 4, [ ] k = [q] k = (k ) π ( ) k ( ) k, k =,, K +, k = K +,, K + Combining this result with (3) and (4), we have that, k = [ ] [x ( ) k ( ) k ] k =, k =,, K + (k ) π, k = K +,, K + o compute the L approximation, using the summation approximation to the L -norm advocated in the problem, we get f y = N N m= c(t m ) x y(t m ) Now, the problem of minimizing the L -norm of the error can be posed as the following LP minimize N s s i c(t i ) x y(t i ) s i, i =,, N A plot of the periodic function y(t) along with its optimal L and L Fourier series approximations are shown in Figure From the plot, we can see that the L optimal approximation has the familiar Gibbs phenomenon near the discontinuities in y, but the L optimal approximation has much less pronounced oscillation in this neighborhood Furthermore, the L approximation has much smaller error than the L approximation, except near the discontinuities A plot of the histogram of the residuals for both the L and L optimal solutions is shown in Figure As can be seen, the optimal L approximation exhibits a much larger number of residuals near zero than the L approximation In addition, the L approximation also has a larger number of outlier residuals than the L approximation, as expected Sample MALAB code using cvx that can be used to generate the figures created here is shown below

3 y(t ) L -norm approximation L -norm approximation Figure : Plot of the -periodic function y(t) along with its optimal L and L Fourier series approximations t 35 3 L-norm L-norm Figure : Plot of the histogram of the residuals for both the L and L optimal approximations to y

4 % Initialize problem parameters N = 496; n = :(*N-); = ; t = (-/) + (n*/(*n)); K = ; % Create matrix of Fourier series coefficients for k = :K cos_mat(k,:) = cos(*pi*k*t/); sin_mat(k,:) = sin(*pi*k*t/); end coef_mat = [ repmat(5,,*n) ; cos_mat ; sin_mat]; % Generate samples of the periodic signal y = zeros(*n,); i_live = find(t >= -5 & t < 5); y(i_live) = ; % Calculate the optimal L_ approximation x_l_ = zeros(*k+,); x_l_() = ; for k = :K+ x_l_(k) = (((-)^(k/))*(((-)^(k-)) - ))/((k-)*pi); end % Calculate the optimal L_ approximation variable x_l_(*k+) minimize(norm(coef_mat *x_l_ - y,)) % Alternate method for calculating the optimal L_ approximation variable x_l_(*k+) variable s(*n) minimize((/(*n))*ones(,*n)*s) - coef_mat *x_l_ + y - s <= coef_mat *x_l_ - y - s <= % Compute the optimal function approximation from the Fourier series % coefficients f_l_ = coef_mat *x_l_; f_l_ = coef_mat *x_l_; % Plot the periodic function and its L_ and L_ Fourier series

5 % approximations figure plot(t,y, r ) hold on plot(t,f_l_, g ) plot(t,f_l_, b ) xlabel( $t$, Interpreter, LaeX, FontSize,) %print -depsc Fourier_series_approx_plot % Plot histograms or distributions of the residual magnitudes figure subplot(,,) hist(f_l_ - y,5) ylabel( $L_{}$-norm, Interpreter, LaeX, FontSize,) axis([ ]) subplot(,,) hist(f_l_ - y,5) ylabel( $L_{}$-norm, Interpreter, LaeX, FontSize,) axis([ ]) %print -depsc Fourier_series_hist_plot his problem can be simplified through vectorization of the matrix quantities that appear throughout the problem For example, by defining the horizontal and vertical difference vectors as u x vec(u k,l U k,l ), u y vec(u k,l U k,l ), with some abuse of notation, then the l variation problem can be equivalently expressed as [ ] ux minimize u y, U k,l = U orig k,l while the total variation problem can be expressed as [ ] ux minimize u y U k,l = U orig k,l, In MALAB, with the help of cvx, this can be implemented as follows % Load total variation image interpolation data tv_img_interp % Calculate the optimal l_ variation interpolant variable Ul(m,n); Ux = Ul(:end,:end) - Ul(:end,:end-); % Compute the x % (horizontal) variations

6 O riginal image Obscured image l variation reconstructed image otal variation reconstructed image Figure 3: Plot of the original, obscured, l variation interpolated, and total variation interpolated images Uy = Ul(:end,:end) - Ul(:end-,:end); % Compute the y (vertical) % variations minimize(norm([ux(:);uy(:)],)); % minimize the l_ % roughness measure Ul(Known) == Uorig(Known); % Fix known pixel values % values % Calculate the optimal total variation interpolant variable Utv(m,n); Ux = Utv(:end,:end) - Utv(:end,:end-); % Compute the x % (horizontal) variations Uy = Utv(:end,:end) - Utv(:end-,:end); % Compute the y (vertical) % variations minimize(norm([ux(:);uy(:)],)); % minimize the total % variation measure Utv(Known) == Uorig(Known); % Fix known pixel values A plot of the original and obscured image, along with the l variation and total variation interpolants are shown in Figure 3 As can be seen, for this particular type of sparse original image, the l based total variation reconstruction interpolates the missing data much better than the l variation based approach

7 3 (a) Ideally, the problem we would like to solve is the following one ) ) minimize card(â + card( B ( W / x(t + ) t= ) Âx(t) Bu(t) n ( ) + n ( ) Here, card(x) is the cardinality of matrix X, ie, the number of nonzero entries he issue with trying to solve this problem is that the objective is nonconvex o address this issue, we will use the common heuristic of minimizing the l norm of the entries of  and B Using the standard vectorization operator, this leads to the following convex optimization problem ) minimize vec(â) + vec( B ( W / x(t + ) t= ) Âx(t) Bu(t) n ( ) + n ( ) Intuitively, it can be said that the constraint will always be tight as relaxing the requirement on the) implied errors ) allows for more freedom to reduce the sum of the l -norms of vec(â and vec( B (b) his problem can be easily solved in MALAB with the help of cvx using the following code % Load problem data sparse_lds_data % Set the fit tolerance fit_tol = sqrt(n*(-) + *sqrt(*n*(-))); % Compute the l_ approximation to the sparse dynamical system fit problem variables Ahat(n,n) Bhat(n,m); minimize(sum(norms(ahat,)) + sum(norms(bhat,))); norm(inv(whalf)*(xs(:,:) - Ahat*xs(:,:-) - Bhat*us), fro ) <= fit_tol; disp(cvx_status) % Round near-zero elements to zero Ahat = Ahat * (abs(ahat) >= ); Bhat = Bhat * (abs(bhat) >= ); % Display the number of false positives and negatives disp([ false positives, Ahat: numstr(nnz((ahat ~= ) & (A == )))]) disp([ false negatives, Ahat: numstr(nnz((ahat == ) & (A ~= )))]) disp([ false positives, Bhat: numstr(nnz((bhat ~= ) & (B == )))]) disp([ false negatives, Bhat: numstr(nnz((bhat == ) & (B ~= )))])

8 With the given problem data, we get false positive (at the (6, 5)-th entry) and false negatives (at the (4, )-th and (6, )-th entries) for Â, and no false positives and false negative (at the (7, )-th entry) for B he matrix estimates are and  =, B = Finally, there are a lot of methods that will do better than this, usually be taking the above solutions as a starting point and then polishing the result after that Several of these have been shown to give fairly reliable, if modest, improvements 4 (a) o find qi min, we solve the convex optimization problem minimize q i l Sq u, q, with variable q R n hen, we set qi min = qi convex optimization problem maximize q i l Sq u, q, Similarly, to find qmax i, we solve the and then we set qi max = qi (b) Running spectrum datam yields plots of the spectra of the compounds s (),, s (n), alongside the lower and upper bounds l and u shown in Figure 4(a) and (b), respectively he following MALAB code using cvx can be used to find the range of possible values for each compound quantity q i [ qi min, qi max ] % Load the spectrum compound data spectrum_data

9 4 8 7 lower bound l upper bound u s (i) : i =,,n l, u k : k =,,m (a) k : k =,,m (b) Figure 4: Plots of compound data: (a) spectra of the compounds and (b) lower and upper bounds for the sample % Calculate the quantity bounds for each compound using a for-loop qmin = zeros(n,); qmax = zeros(n,); for i = :n % Compute the minimum bound for the i-th quantity variable q(n) l <= S*q; u >= S*q; q >= ; minimize(q(i)) qmin(i) = q(i); % Compute the maximum bound for the i-th quantity variable q(n) l <= S*q; u >= S*q; q >= ; maximize(q(i)) qmax(i) = q(i); end % Display the quantity bounds [qmin qmax] % Plot the quantity bounds figure; hold on; for i = :n plot([i,i],[qmax(i),qmin(i)], o- ); end

10 9 8 ] qi [ q i min,q i max i : i =,,n Figure 5: Plot of the range of possible values for each quantity q i [ q min i, qi max ] for i =,, n axis([,,,]); A plot of the range of possible values for each quantity q i is shown in Figure 5 From the output of cvx, we have q4 min = and q4 max = 5 *5 (a) First note that we have [ ] Re[G(θ)] G(θ) = Im[G(θ)] hen, note that if we define the following quantities ] x C(θ) [ wre w im R n, where [w re ] k = w re,k, [w im ] k = w im,k, [ cos φ (θ) cos φ n (θ) sin φ (θ) sin φ n (θ) sin φ (θ) sin φ n (θ) cos φ (θ) cos φ n (θ) ] R n, then we have [ ] Re[G(θ)] = C(θ) x Im[G(θ)] o express the design problem as an SOCP, define the following quantities A l C(θ l ), B C ( θ tar), [ ] d

11 3 5 y G(θ) (db) (a) x θ (deg) (b) Figure 6: Plots of (a) the antenna array geometry and (b) antenna array magnitude response G(θ) in db Note that the constraint G ( θ tar) = is equivalent to the following [ [ ( Re G θ tar )] ] [ ] Im [ G ( θ tar)] = Bx = d, which is an affine equality constraint on x hus, the antenna array design problem becomes the following SOCP minimize t A l x t, l =,, N Bx = d (b) his part of the problem can be simplified by noting that we can alternatively express G(θ) as G(θ) = e (θ) w, where e(θ) C n and w C n are given by ( [e(θ)] k = e j πxk λ [w] k = w k, k =,, n cos θ+ πy k λ sin θ ), k =,, n Using this simplification, for the antenna array geometry shown in Figure 6(a), we obtain the antenna array magnitude response G(θ) in db as shown in Figure 6(b) From Figure 6(b), it can be seen that the array response yields unity gain at the target angle of arrival θ tar and over 35 db of attenuation outside of the beamwidth Sample MALAB code using cvx which was used to obtain the results here is shown below % Define design problem parameters n = 4;

12 lambda = *pi; theta_tar_deg = 5; theta_tar = degrad(theta_tar_deg); Delta_deg = 5; Delta = degrad(delta_deg); N = ; % Generate antenna positions rand( state,); x = 3*rand(n,); y = 3*rand(n,); % Construct the discretized angle of arrival values theta_des = linspace(theta_tar+delta,theta_tar+(*pi)-delta); wn = *pi/lambda; e = exp(-j*(((wn*x)*cos(theta_des)) + ((wn*y)*sin(theta_des)))); e_tar = exp(-j*(((wn*x)*cos(theta_tar)) + ((wn*y)*sin(theta_tar)))); % Compute the optimal antenna array using cvx variable w(n) complex minimize(max(abs(e *w))) e_tar *w == ; % Plot the antenna array response theta_plot_deg = linspace(-8,8,89); theta_plot = degrad(theta_plot_deg); e = exp(-j*(((wn*x)*cos(theta_plot)) + ((wn*y)*sin(theta_plot)))); G = e *w; plot(theta_plot_deg,*log(abs(g))) xlim([-8 8]) grid xlabel( $\theta$ (deg), Interpreter, LaeX, FontSize,) ylabel( $\left G\!\left( \theta \right) \right $ (db), Interpreter, LaeX, FontSize,)