Optimal Control. Quadratic Functions. Single variable quadratic function: Multi-variable quadratic function:

Transcription

1 Optimal Control Control design based on pole-placement has non unique solutions Best locations for eigenvalues are sometimes difficult to determine Linear Quadratic LQ) Optimal control minimizes a quadratic performance index based on time response A state feedback control law results from solving an LQ optimal control problem Quadratic Functions Single variable quadratic function: f x) = q bx c Multi-variable quadratic function: f x) = x T Qx b T x c Where Q is a symmetric Q T =Q) nxn matrix and b is an nx1 vector It can be shown that the Jacobian of f is "f "x = # "f "f L "f & % = 2x T Q b T $ "x 1 " "x n '

2 2-Variable Quadratic Example f x) = x 1 2 2x2 2 " 2x1 " 2x 1 [ ] 1 "1 f x) = x 1 # &# % x 1& # % "2 0 % $ "1 2 ' $ ' $ [ ] x 1 & ' Quadratic Optimization To get x that minimizes fx) {x * } set "f "x = 2xT Q b T = 0 # 2Qx b = 0 or equivalently H = " "x x " =! 1 Q 2 Provided that the Hessian of f, ) " 2 f 2 "x 1 " 2 f L "2 f. "x 1 " "x 1 "x n. T #"f & " 2 f " 2 f % = L "2. f. $ "x ' "x 1 " 2 " " "x n. = 2Q M M O M. " 2 f " 2 f L "2. f. "x 1 "x n " " * n "x n -. is positive definite!1 b,

3 Positive Definite Matrixes Definition: A symmetric matrix H is said to be positive definite denoted by H>0) if x T Hx>0 for any non zero vector x. Positive definiteness Sylvester) test: H is positive definite iff all the principal minors of H are positive: h 11 h 12 L h 1n h 11 > 0, h 11 h 12 > 0,K, h 21 h 22 h 21 h 22 L h 2n M M O M > 0 h n1 h n2 L h nn 2-Variable Quadratic Optimization Example [ ] 1 "1 f x) = x 1 Optimal solution: # H = 2Q = 2 "2 & % > 0 $ "2 4 ' # &# % % $ " ' $ Q x " = # 1 2 Q#1 b = # 1 $ 1 #1' & ) 2 %#1 2 Thus x * minimizes fx) x 1 #1 $ #2 & # "2 0 % ' $ ' & % 0 ) = $ 2 ' & % 1 ) [ ] x 1 & '

4 Continuous-Time Linear Quadratic LQ) Optimal Control Given continuous-time state equation x = Ax Bu Find the control function ut) to minimize t f" Jx,t 0 ) = 1 2 xt t f )Sxt f ) 1 2 S,Q # 0, R > 0 and symmetric [ x T t)qxt) u T t)rut)] t 0 dt Comments on Performance Index PI) Control objective is to make x small by penalizing large inputs and states PI makes a compromise between performance and control action u T Ru x T Qx t

5 1 Principle of Optimality Bellman s Principle of Optimality: At any intermediate state x i in an optimal path from x 0 to x f, the path from x i to goal x f must itself constitute optimal path Optimal Cost-To-Go The optimal cost-to-go at x,t) is the optimal value of the cost function starting the state x at time t given by J " x,t) = 1 2 x"t t f )Sx " t f ) 1 t f$ x "T #)Qx " #) u "T #)Ru " #) 2 [ ] d# t where u * denotes the optimal input and x * t) the resulting optimal state trajectory.

6 LQ Optimization Formulation Let J * x,t) denote optimal cost-to-go, then by principle of optimality [ ] & 1 t$t J " x T #)Qx#) u T * % #)Ru#) d# xt),t) = min' 2 t ut) ) J " xt $t),t $t), where!t is an infinitesimal change in time t. By Taylor series expansion J " xt #t),t #t) = J " % xt),t) $J" ' & $t $J" $x dx dt * #t h.o.t ) Hamilton-Jacobi-Bellman HJB) Equation The HJB Eq. Results by letting!t "0: "J # "t 1 $ & 2 xt 1 t)qxt) min% ut) 2 ut t)rut) "J# '& "x & Axt) But) )) *& = 0 Set derivative w.r.t ut) to 0 to obtain optimal ut) u T t)r "J# "x B = 0 $ u t ) = %R %1 B T & "J # ) ' "x * The solution can be found by noting that cost-to-go is quadratic in x: ) = 1 2 xt Pt)x J " x,t for some symmetric matrix Pt)#0. T

7 Continuous-Time LQ Solution Using optimal u and " "x J# x,t) = x T Pt), " "t J# x,t) = 1 2 xt P t)x in HJB Eq. gives # % $ &% u = R "1 B T Px x T P x x T Qx u T Ru 2x T P Ax Bu) = 0 or x T P Q " PBR "1 B T P 2PA) x = 0 For all x. Note: 2x T PAx= x T PAxx T A T Px Summary of Continuous-Time LQ Solution Control law: u t) = K t)x t), K t) = "R "1 B T P t) Ricatti Equation P PA A T P Q " PBR "1 B T P = 0, Pt f ) = S Cost-to-go: J " x,t) = 1 2 xt Pt)x

8 Comments on Continuous-Time LQ Solution Control law is a time varying state feedback law Matrix Pt) can be found by integrating the Ricatti equation backward in time starting with Pt f )=S. In most cases Pt) and Kt) have steadystate solutions as t f approaches infinity Matlab Example Find the optimal controller that minimizes y J = 5y10) 2 10 " [y 2 t) u 2 t)]dt 0 u M=1 Solution: State Equation x 1 = position, = velocity, u = force " x 1 = # % d & $ x 2 = u dt ' x 1 ) & = 0 1 )& x 1) * ' 0 0* ' * & 0 ) ' 1 u * Performance Index weighting Matrices: J = 1 " 2 xt 10) 10 0 % $ ' x10) 1 10 x T " t) 2 0 % $ ' xt) 2u 2 t) dt # 0 0& 2 # 0 0& S 0 Q R

9 Backward Integration of Ricatti Eq. Ricatti Equation P PA A T P Q " PBR "1 B T P = 0, Pt f ) = S Let P b t)=pt f -t). Then P b 0)=Pt f )=S and P b t f )=P0). Moreover since dp b /dt=-dp/dt P b = P b A A T P p Q " P b BR "1 B T P b, P b 0) = S The above matrix differential equation can be solved numerically using standard ode solvers. Matlab Solution %System Matrices A=[0 1;0 0]; B=[0;1]; tf=10; %Performance Index Matrices S=[10 0;0 0]; Q=[2 0;0 0]; R=2; %Call ode45 to solve for P b T=[0:0.1:10] ; XP0=S:); [T,XPb]=ode45 ricfun,t,xp0,[],a,b,q,r); XP=flipudXPb); for k=1:lengtht) Pk=reshapeXPk,:)',2,2); Kk,:)=-invR)*B'*Pk; end %Call ode45 to simulate the system response [T,X]=ode45 sysfun,t,[1;0],[],a,b,k,t);

10 Functions of State Equations function dx=ricfunt,x,flag,a,b,q,r); %Ricatti Eq. Function to be called by ode45 Pb=reshapex,2,2); dpb=pb*aa'*pbq-pb*b*invr)*b'*pb; dx=dpb:); function dx=sysfunt,x,flag,a,b,k,t) %System Function to be called by ode45 k=maxfindt>=t)); %determine the index k Kk=Kk,:); dx=ab*kk)*x; P Matrix Entries P Values Time sec)

11 K Values 1 K Values K 1 K Time sec) Closed-Loop Response Response X 1 X Time sec)