EE363 homework 1 solutions

Transcription

1 EE363 Prof. S. Boyd EE363 homework 1 soluions 1. LQR for a riple accumulaor. We consider he sysem x +1 = Ax + Bu, y = Cx, wih 1 1 A = 1 1, B =, C = [ 1 ]. 1 1 This sysem has ransfer funcion H(z) = (z 1) 3, and is called a riple accumulaor, since i consiss of a cascade of hree accumulaors. (An accumulaor is he discreeime analog of an inegraor: is oupu is he running sum of is inpu.) We ll use he LQR cos funcion N 1 N J = u 2 + y 2, = = wih N = 5. (a) Find P (numerically), and verify ha he Riccai recursion converges o a seadysae value in fewer han abou 1 seps. Find he opimal ime-varying sae feedback gain K, and plo is componens (K ) 11, (K ) 12, and (K ) 13, versus. (b) Find he iniial condiion x, wih norm no exceeding one, ha maximizes he opimal value of J. Plo he opimal u and resuling x for his iniial condiion. Soluion: (a) The following Malab scrip solves boh pars, i.e., implemens he Riccai recursion, and finds he wors case iniial condiion. % LQR for a riple accumulaor % daa A = [1 ; 1 1 ; 1 1]; B = [1 ] ; C = [ 1]; Q = C *C; R = 1; m=1; n=3; N=5; % Riccai recursion P = zeros(n,n,n+1); P(:,:,N+1) = Q; K = zeros(m,n,n); %Nrm = zeros(n,1); for i = N:-1:1 1

2 P(:,:,i) = Q+A *P(:,:,i+1)*A-... A *P(:,:,i+1)*B*pinv(R+B *P(:,:,i+1)*B)*B *P(:,:,i+1)*A; K(:,:,i) = -pinv(r+b *P(:,:,i+1)*B)*B *P(:,:,i+1)*A; % Nrm(N+1-i) = norm(p(:,:,i+1)-p(:,:,i))/norm(p(:,:,i+1)); end % wors iniial condiion (max_x() min_u J) [V,D] = eig(p(:,:,1)); x = -V(:,1); % opimal u and resuling x x = zeros(3,n); x(:,1) = x; u = zeros(1,n); u(1) = K(:,:,1)*x(:,1); for = 1:N-1 x(:,+1) = A*x(:,) + B*u(); u(+1) = K(:,:,+1)*x(:,+1); end % plos figure(1); = :49; K = shifdim(k); subplo(3,1,1); plo(,k(1,:)); ylabel( K1() ); subplo(3,1,2); plo(,k(2,:)); ylabel( K2() ); subplo(3,1,3); plo(,k(3,:)); ylabel( K3() ); xlabel( ); %prin -depsc ripleacck figure(2); subplo(4,1,1); plo(,x(1,:)); ylabel( x1() ); subplo(4,1,2); plo(,x(2,:)); ylabel( x2() ); subplo(4,1,3); plo(,x(3,:)); ylabel( x3() ); subplo(4,1,4); plo(,u); ylabel( u() ); xlabel( ); %prin -depsc ripleaccxu %figure(3); %semilogy(nrm); ile( Riccai convergence ); Afer abou 9 ieraions we see ha he marix P +1 P has elemens on he order of 1 3, compared o ypical P values of around 5. The plo below shows he hree elemens of K versus. 2

3 .5 (K) (K) (K) (b) The opimal cos, when sared in sae x, is x T P x. To maximize his we choose x as an eigenvecor of P associaed wih is maximum eigenvalue. The eigenvecor associaed wih he maximum eigenvalue is x max = The opimal rajecories of he hree elemens of x and he opimal inpu u lqr shown below. are 3

4 (x) (x)2 (x)3 u lqr Linear quadraic sae racking. We consider he sysem x +1 = Ax + Bu. In he convenional LQR problem he goal is o make boh he sae and he inpu small. In his problem we sudy a generalizaion in which we wan he sae o follow a desired (possibly nonzero) rajecory as closely as possible. To do his we penalize he deviaions of he sae from he desired rajecory, i.e., x x d, using he following cos funcion: N J = (x τ x d τ) T Q(x τ x d N 1 τ) + u T τ Ru τ, τ= τ= where we assume Q = Q T and R = R T >. (The desired rajecory x d τ is given.) Compared wih he sandard LQR objecive, we have an exra linear erm (in x) and a consan erm. In his problem you will use dynamic programming o show ha he cos-o-go funcion V (z) for his problem has he form z T P z + 2q T z + r, wih P = P T. (i.e., i has quadraic, linear, and consan erms.) (a) Show ha V N (z) has he given form. (b) Assuming V +1 (z) has he given form, show ha he opimal inpu a ime can be wrien as u = K x + g, 4

5 where K = (R + B T P +1 B) 1 B T P +1 A, g = (R + B T P +1 B) 1 B T q +1. In oher words, u is an affine (linear plus consan) funcion of he sae x. (c) Use backward inducion o show ha V (z),...,v N (z) all have he given form. Verify ha Soluion: P = Q + A T P +1 A A T P +1 B(R + B T P +1 B) 1 B T P +1 A, q = (A + BK ) T q +1 Qx d, r = r +1 + x d Qx d + q T +1Bg, for =,...,N 1. (a) We can wrie V N (z) = (z x d N )T Q(z x d N ) = zt Qz 2x d N Qz + xd N Qxd N = zt P N z + 2q T N z + r N, where P N = Q, q N = Q T x d N and r N = x d N Qxd N. (b) Assuming V +1 (z) has he given form, he Bellman equaion can be wrien as V (z) = min w {(z x d )T Q(z x d ) + wt Rw + V +1 (Az + Bw)} = (z x d ) T Q(z x d ) + min w {w T Rw +(Az + Bw) T P +1 (Az + Bw) + 2q T +1(Az + Bw) + r +1 }. To find he w ha minimizes he above expression, we se is derivaive wih respec o w equal o zero. This gives and so 2Rw + 2B T P +1 (Az + Bw) + 2B T q +1 =, u = w = (R + B T P +1 B) 1 (B T q +1 + B T PAz) = K z + g, where K = (R + B T P +1 B) 1 B T P +1 A, and g = (R + B T P +1 B) 1 B T q +1. (c) We can wrie V (z) as V (z) = z T (Q + A T P +1 A)z + 2(A T q +1 x d )T z + x d Qxd + r +1 + min w {w T (R + B T P +1 B)w + 2(B T P +1 Az + B T q +1 ) T w}. 5

6 Subsiuing w ino he above expression gives V (z) = z T (Q + A T P +1 A)z + 2(A T q +1 x d ) T z + x d Qx d + r +1 +(B T q +1 + B T P +1 Az) T (R + B T P +1 B) 1 (B T q +1 + B T P +1 Az) (B T q +1 + B T P +1 Az) T (R + B T P +1 B) 1 (B T q +1 + B T P +1 Az) = z T (Q + A T P +1 A)z + 2(A T q +1 x d ) T z + x d Qx d + r +1 (B T q +1 + B T P +1 Az) T (R + B T P +1 B) 1 (B T q +1 + B T P +1 Az). Afer rearranging and collecing erms we ge where V (z) = z T (Q + A T P +1 A A T P +1 B(R + B T P +1 B) 1 B T P +1 A)z +2(A T q +1 A T P +1 B(R + B T P +1 B) 1 B T q +1 x d ) T z +r +1 + x d Qx d q T +1B(R + B T P +1 B) 1 B T q +1 = z T P z + 2q T z + r, P = Q + A T P +1 A A T P +1 B(R + B T P +1 B) 1 B T P +1 A, q = A T q +1 A T P +1 B(R + B T P +1 B) 1 B T q +1 x d = (A + BK) T q +1 x d, r = r +1 + x d Qxd qt +1 B(R + BT P +1 B) 1 B T q +1 = r +1 + x d Qxd + qt +1 Bg. Thus we have shown ha if V +1 (z) has he given form, hen V (z) also has he given form. Since V N (z) has his form (from par (a)), by inducion, V (z),...,v N (z) all have he given form. 3. The Schur complemen. In his problem you will show ha if we minimize a posiive semidefinie quadraic form over some of is variables, he resul is a posiive semidefinie quadraic form in he remaining variables. Specifically, le J(u, z) = [ u z ] T [ Q11 Q 12 Q T 12 Q 22 ] [ u z be a posiive semidefinie quadraic form in u and z. You may assume Q 11 > and Q 11, Q 22 are symmeric. Define V (z) = min J(u, z). Show ha V (z) = z T Pz, where u P is symmeric posiive semidefinie (find P explicily). The marix P is called he Schur complemen of he marix Q 11 in he big marix above. I comes up in many conexs. Soluion: We can wrie J(u, z) = u T Q 11 u + 2u T Q 12 z + z T Q 22 z. To find he u ha minimizes his expression, we se is derivaive wih respec o u equal o zero. This gives 2Q 11 u + 2Q 12 z = 6 ]

7 and so u = Q 1 11 Q 12 z. Thus we ge V (z) = z T Q T 12 Q 1 11 Q 12z + z T Q 22 z 2z T Q T 12 Q 11Q 12 z = z T (Q 22 Q T 12 Q 1 11 Q 12)z = z T Pz, where P = Q 22 Q T 12 Q 1 11 Q 12. Clearly, P is symmeric; i is also posiive semidefinie, since z T Pz = min u J(u, z), z. 4. A useful deerminan ideniy. Suppose X R n m and Y R m n. (a) Show ha de(i + XY ) = de(i + Y X). Hin: Find a block lower riangular marix L for which [ ] [ ] I X I X = L, Y I I and use his facorizaion o evaluae he deerminan of his marix. Then find a block upper riangular marix U for which [ ] [ ] I X I = U, Y I Y I and repea. (b) Show ha he nonzero eigenvalues of XY and Y X are exacly he same. Soluion: (a) Muliplying on he righ by wo differen marices yields [ ] I X de Y I ([ ] [ ]) I I X = de Y I + Y X I ([ ] [ ]) I + XY X I = de I Y I = de(i + Y X) = de(i + XY ). (b) Recall ha an eigenvalue λ of a marix A saisfies de(λi A) =. Le I r denoe an r r ideniy marix. If λ is a nonzero eigenvalue of XY, = de(λi n XY ) = de (λi n (I n + ( 1 ) λ X)Y ) = de(λi n ) de(i n + ( 1 X)Y ). λ We know de(λi n ) = λ n = λ n m de(λi m ), 7

8 and from par (a) de(i m + Y ( 1 λ X)) = de(i n + ( 1 X)Y ), λ so = λ n m de(λi m ) de(i m + ( 1 )Y X) λ = λ n m de(λi m Y X). Since λ is nonzero, we have ha de(λi m Y X) =. Hence λ is also an eigenvalue of Y X. By similar argumen, if λ is a nonzero eigenvalue of Y X, we ge ha = de(λi n XY ), showing λ is also an eigenvalue of XY. Thus, XY and Y X have exacly he same nonzero eigenvalues. 5. When does a finie-horizon LQR problem have a ime-invarian opimal sae feedback gain? Consider a discree-ime LQR problem wih horizon = N, wih opimal inpu u() = K x(). Is here a choice of Q f (symmeric and posiive semidefinie, of course) for which K is consan, i.e., K = = K N 1? Soluion: K is defined by K = (R+B T P +1 B) 1 B T P +1 A. Therefore, if P 1 = = P N, we have K = = K N 1. Since Q f = P N, and for far from N we have ha P converges o he seady-sae soluion, we see ha o have consan P, we mus have Q f equal o he seady-sae soluion P ss. Wih his choice, we have P = P ss for all, and herefore K = K ss for all. 8