.. Quadratic Stability of Dynamical Systems Raktim Bhattacharya Aerospace Engineering, Texas A&M University
Quadratic Lyapunov Functions
Quadratic Stability Dynamical system is quadratically stable if ẋ = Ax,. V (x), V. Let V (x) = x T P x, P S n ++ (P = P T > ) Therefore, Therefore V (x) = ẋ T P x + x T P ẋ = x T A T P x + x T P Ax = x T ( A T P + P A ) x V = x T ( A T P + P A ) x = A T P + P A. AERO 632, Instructor: Raktim Bhattacharya 3 / 3
Lyapunov Equation We can write as for Q = Q T. A T P + P A. A T P + P A + Q = Interpretation For linear system ẋ = Ax, if V (x) = x T P x, V (x) = ẋ T P x + x T P ẋ = (Ax) T P x + x T P (Ax) = x T Qx. If V (x) = x T P x is generalized energy, V = x T Qx is generalized dissipation. AERO 632, Instructor: Raktim Bhattacharya 4 / 3
Stability Condition If P >, Q >, then ẋ = Ax is globally asymptotically stable Rλ i (A) <. Note that for P = P T >, eigenvalues are real = λ min (P ) x T x x T P x λ max (P ) x T x = V = x T Qx λ min (Q)x T x λ min(q) λ max (P ) xt P x = αv (x) AERO 632, Instructor: Raktim Bhattacharya 5 / 3
Lyapunov Integral If A is stable, then P =. e tat Qe ta dt, for any Q = Q T >. Proof: Substitute it in LHS of Lyapunov equation to get, ( ) A T P + P A = A T e tat Qe ta + e tat Qe ta A dt, = ( ) d dt etat Qe ta dt, = e tat Qe ta, = Q. AERO 632, Instructor: Raktim Bhattacharya 6 / 3
. Computation of x 2,Q Recall x 2 2 := Define weighted norm as x 2 2,Q := If x(t) is solution of ẋ = Ax, Substituting we get, x 2 2,Q = x T x dt. x(t) := e ta x. x T Qx dt. x T e tat Qe ta dt = x T P x assuming A is stable Cost-to-go interpretation AERO 632, Instructor: Raktim Bhattacharya 7 / 3
LQR Problem
Linear Quadratic Regulator Problem Statement Given system ẋ = Ax + Bu, Determine u (t) that solves Or min J := u(t) = y T y dt. y = Cx + Du. min y 2 with x() = x. u(t) ( x T C T Cx + x T C T Du + u T D T Cx + u T D T Du ) dt = ( x T C T Cx + u T D T Du ) dt. Assume C T D = for simplicity. AERO 632, Instructor: Raktim Bhattacharya 9 / 3
Linear Quadratic Regulator Solution as Optimal Control Problem. min u subject to ( x T Qx + u T Ru ) dt, Q = Q T, R = R T > ẋ = Ax + Bu, x() = x. Euler Lagrange Equations Hamilton-Jacobi-Bellman Equation Dynamic Programming AERO 632, Instructor: Raktim Bhattacharya 1 / 3
Euler Langrange Formulation
Linear Quadratic Regulator Solution as Optimal Control Problem EL Formulation. T min L(x, u)dt + Φ(x(T )), subject to ẋ = f(x, u). u Define H = L + λ T f. Euler-Lagrange Equations Our Problem H u = λt = H x λ(t ) = ϕ x (x(t )) min u T ( x T Qx + u T Ru ) dt, subject to ẋ = Ax + Bu. Define H = x T Qx + u T Ru + λ T (Ax + Bu). AERO 632, Instructor: Raktim Bhattacharya 12 / 3
Linear Quadratic Regulator Solution as Optimal Control Problem EL Formulation Our Problem 1 min u 2 T Define H = 1 2 EL Equations. ( x T Qx + u T Ru ) dt, subject to ẋ = Ax + Bu. ( x T Qx + u T Ru ) + λ T (Ax + Bu). (1) H u = = u T R + λ T B = = u = R 1 B T λ. (2) λt = H x = x T Q λ T A (3) λ(t ) =. AERO 632, Instructor: Raktim Bhattacharya 13 / 3
Linear Quadratic Regulator Solution as Optimal Control Problem EL Formulation Let λ(t) = P (t)x(t) From EL(2) we get λ = Qx A T P x. = λ = P x + P ẋ = P x + P (Ax + Bu), = P x + P (Ax BR 1 B T P x), = ( P + P A P BR 1 B T P )x. = ( P + P A + A T P P BR 1 B T P + Q)x = = P + P A + A T P P BR 1 B T P + Q =. Riccati Differential Equation In the steady-state T, P =, P A + A T P P BR 1 B T P + Q =. Algebraic Riccati Equation AERO 632, Instructor: Raktim Bhattacharya 14 / 3
Linear Quadratic Regulator Solution as Optimal Control Problem EL Formulation. 1 min u 2 is equivalent to ( x T Qx + u T Ru ) dt, subject to ẋ = Ax + Bu. P A + A T P P BR 1 B T P + Q =, u = R 1 B T P. AERO 632, Instructor: Raktim Bhattacharya 15 / 3
Hamilton-Jacobi-Bellman Formulation
. Hamilton-Jacobi-Bellman Approach Let subject to V 1 (x(t)) = min u[t, ) 2 t ẋ = Ax + Bu. (x T Qx + u T Ru)dt AERO 632, Instructor: Raktim Bhattacharya 17 / 3
. Hamilton-Jacobi-Bellman Approach contd. V (x(t)) = min = min u[t,t+ t] 1 2 u[t, ) { t+ t t Let V (x) := x T P x, therefore, t (x T Qx + u T Ru)dt 1 2 (xt Qx + u T Ru)dt + V (x(t + t)) } { 1 V (x(t)) = min u[t,t+ t] 2 (xt Qx + u T Ru) t + V (x(t))+ } (Ax + Bu) T P x t + x T P (Ax + Bu) t + H.O.T AERO 632, Instructor: Raktim Bhattacharya 18 / 3
. Hamilton-Jacobi-Bellman Approach contd. { 1 = min u[t,t+ t] 2 (xt Qx + u T Ru) + (Ax + Bu) T P x+ } x T P (Ax + Bu) + H.O.T =. lim t { 1 = min u 2 (xt Qx + u T Ru)+ } (Ax + Bu) T P x + x T P (Ax + Bu) = Quadratic in u, = u = R 1 B T P x. Optimal controller is state-feedback. AERO 632, Instructor: Raktim Bhattacharya 19 / 3
Variational Approach
A Third Approach Given dynamics ẋ = Ax + Bu,. with controller u = Kx, find K that minimizes J := 1 2 (x T Qx + u T Ru)dt = 1 2 The closed-loop dynamics is The solution is therefore, ẋ = Ax + Bu = (A + BK)x = A c x. x(t) = e ta c x. x T (Q + K T RK)xdt. The cost function is therefore, J := 1 ( ) 2 xt e tat c (Q + K T RK)e ta c dt x = 1 2 xt P x. AERO 632, Instructor: Raktim Bhattacharya 21 / 3
A Third Approach contd. Apply the following trick Or Or. d dt etat c (Q + K T RK)e ta c dt = ( ) A T c e tat c (Q + K T RK)e ta c dt ( ) + e tat c (Q + K T RK)e tac dt [ e tat c (Q + K T RK)e tac ] = AT c P + P A c A T c P + P A c + Q + K T RK =, (A + BK) T P + P (A + BK) + Q + K T RK =. A c AERO 632, Instructor: Raktim Bhattacharya 22 / 3
A Third Approach contd.. The optimal cost is therefore, J = 1 2 xt P x = J P =. P Variation δp from P should result in δj = Let P = P + δp, = J = 1 2 xt P x + 1 2 xt δp x }{{} δj δj = = δp = AERO 632, Instructor: Raktim Bhattacharya 23 / 3
A Third Approach. contd. Substitute P = P + δp, and K = K + δk in the equality constraint to get, or (A + BK) T P + P (A + BK) + Q + K T RK =, (A + B(K + δk)) T (P + δp ) + (P + δp )(A + B(K + δk)) + Q + (K + δk) T R(K + δk) =, (A + BK ) T P + P (A + BK ) + Q + K T RK + δp (A + BK ) + ( ) T + δk T (B T P + RK ) + ( ) T = K = R 1 B T P. + H.O.T =. AERO 632, Instructor: Raktim Bhattacharya 24 / 3
LMI Formulation
Problem Formulation Find gain K such that u = Kx minimizes. subject to dynamics and (x T Qx + u T Ru)dt, ẋ = Ax + Bu, x() = x. AERO 632, Instructor: Raktim Bhattacharya 26 / 3
. An Upper Bound on the Cost-to-go If V (x) > such that Integrating from [, T ], gives us or T dv dt (xt Qx + u T Ru). dv dt dt T V (x(t )) V (x()) Since V (x(t )) for any T = V (x()) (x T Qx + u T Ru)dt, T T (x T Qx + u T Ru)dt. (x T Qx + u T Ru)dt, AERO 632, Instructor: Raktim Bhattacharya 27 / 3
. An Upper Bound on the Cost-to-go Since V (x(t )) for any T or = V (x()) V (x()) T (x T Qx + u T Ru)dt, (x T Qx + u T Ru)dt.. Sufficient condition for upper-bound on cost-to-go.. If V (x) > such that. dv dt (xt Qx + u T Ru). Idea: Minimize upper-bound to get optimal K. AERO 632, Instructor: Raktim Bhattacharya 28 / 3
Optimization Problem. Find P = P T > and K such that with V := x T P x, subject to min P,K V (x()) = x()t P x() Cost Function V x T (Q + K T RK)x Constraint Function. Or equivalently. subject to min trp P,K. (A + BK) T P + P (A + BK) + Q + K T RK. AERO 632, Instructor: Raktim Bhattacharya 29 / 3
Optimization Problem. subject to min trp P,K (A + BK) T P + P (A + BK) + Q + K T RK. Learn about Linear Matrix Inequalities AERO 632, Instructor: Raktim Bhattacharya 3 / 3