State-Feedback Optimal Controllers for Deterministic Nonlinear Systems

Transcription

1 State-Feedback Optimal Controllers for Deterministic Nonlinear Systems Chang-Hee Won*, Abstract A full-state feedback optimal control problem is solved for a general deterministic nonlinear system. The solution method is based on transforming Hamilton-Jacobi equation into an algebraic equation using the pseudo-inverse. Then we interpret the value function in terms of the control Lyapunov function provide the stabilizing controller the stability margins. We also derive an optimal controller for a nonlinear system which requires a solution of the state dependent Riccati equation. Simple examples demonstrate each method. I. INTRODUCTION Linear optimal control theory is a well developed area, however, nonlinear optimal control theory is not as well developed as the linear version. One of the dif culties of the nonlinear optimal control is in solving the Hamilton- Jacobi (HJ) partial differential equation. In this paper, we will convert the HJ partial differential equation problem into nding a pseudo-inverse solving an algebraic equation problem. Recently, a simple inversion method was proposed by Aliyu for a nonlinear H control problem. We will utilize some of his concepts in this paper. Moreover, interest in control Lyapunov function generated renewed interest in nonlinear optimal control. In, Primbs et al. solved a nonlinear control problem using a new class of receding horizon control control Lyapunov function 9. This control scheme relates pointwise min-norm control with optimal control, but it does not solve nonlinear optimal control problem. In, Curtis Beard solved a nonlinear control problem using the satis cing sets control Lyapunov function. This framework completely characterizes all universal formulas, but once again it is not optimal control in the traditional sense. Our paper utilizes some of the concepts introduced in the above papers to solve traditional optimal control problem for a deterministic nonlinear system. In this paper, we derive the optimal full-state-feedback solution of a deterministic af ne nonlinear system. Then we compare the solutions with the inversion method of Aliyu, we relate the value function with the control Lyapunov function. Finally, we derive the state dependent Riccati Equation (SDRE) controller for a nonlinear system. Manuscript received September 5,. This work is supported in part by the National Science Foundation grant, ECS-856. *C. Won is with the Department of Electrical Engineering, University of North Dakota, Gr Forks, ND , Phone (7)777-68, won@und.edu II. HAMILTON-JACOBI EQUATION AND OPTIMAL CONTROLLER Consider the following system, dx(t) = f(x(t), u(t))dt = g(x(t))dt+b(x(t))u(t)dt, () where g : IR n IR n,b : IR n IR n m,x IR n, u IR m. Assume that g B are continuously differentiable in all arguments. Furthermore, consider the following cost function J(x,k) = tf t l(x) + k (x)r(x)k(x)dτ, () where l(x) is continuously differentiable, l(x), R(x) >. We also assume that g(x), l(x) is zerostate detectable. The optimization problem is to design a state-feedback controller, k(x), that will minimize the cost function, V (x) = inf J(x,k). () Now, we optimize the Hamilton-Jacobi equation to nd the optimal controller, { } = inf (g + Bk) V k + l + k Rk, () where we suppressed the argument x. The minimizing controller is obtained as k = R (x)b (x) V (x), (5) where V (x) is a column vector. Second order condition, R(x) >, is satis ed also. Therefore, the minimum is guaranteed, the controller is a cidate for an optimal controller. Instead of nding V, we will determine an explicit expression for V (x). Assuming that there exists a smooth, V : IR n IR +, substituting Eq. (5) into Eq. (), suppressing the argument x, we obtain a Hamilton Jacobi (HJ) equation k V BR B V V g l =. (6) The above HJ equation implies the following HJ inequality for smooth V V BR B V V g l. (7) To nd the optimal controller we introduce the slightly modi ed version of the lemma by Liu Leake 8,.

2 Our version gives two solutions uses pseudo-inverse. Let x IR n be a real n-vector, z(x) y(x) be real r- vector functions, α(x) be a real function de ned on IR n. Lemma.: Let y(x) α(x) be given. Then z(x) satis es the condition z (x)z(x) + z (x)y(x) + α(x) = (8) if only if y(x) α(x). In such a case, the set of all solutions to (8) is represented by z(x) = y(x) ± β(x)a(x) (9) where β(x) = ( y(x) α(x) ) a(x) is an arbitrary unit vector. Proof. The suf ciency follows by direct evaluation. To show the conditions are necessary, note that y < α implies that z + y <, which is a contradiction. Let ±w = z + y, then (8) implies that w w = β. Take a = w, we have w = ±βa. w Consequently, (9) follows. Lemma.: (Liu Leake s Lemma). Let X be a nonnegative de nite symmetric-real matrix. Then z(x) satis es the condition if only if z (x)xz(x) + z (x)y(x) + α(x) = () y (x)x + y(x) α(x), () where X + is the pseudo-inverse of X. In this case, the set of all solutions to () is represented by where z(x) = X + y(x) ± H + a(x)β(x) () β(x) = ( y (x)x + y(x) α(x) ), () H is a non-singular matrix such that X = H H, a(x) is an arbitrary unit vector. Proof. The existence of such an H is well-known. The proof of this lemma follows from Lemma. by a change of variables ẑ = Hz ŷ = (H + ) y. In the next theorem the notation is used for Kronecker product I n is used for n n identity matrix. Now, we state the main theorem of this section. Theorem.: Assume V is smooth. Let P(x) = B(x)R (x)b (x) () ρ(x) = P(x)a(x) g (x)p + (x)g(x) + l(x). (5) For an af ne nonlinear system (), the optimal controller that minimizes the cost function () is given by k (x) = R (x)b (x)p + (x)g(x) ± ρ(x), (6) if the following symmetry non-negative de nite requirements are satis ed. P + ± ρ ) + P + (I n (g ± ρ)) = ± ρ ) ( ) P P + + (I n (g ± ρ)) + (7) P + ± ρ ) + P + (I n (g ± ρ)). (8) Proof. Utilizing the Liu Leake s Lemma on Eq. (6) with z = V,X = (BR B /),y = g/, α = l, we have V = ( BR B ) + (BR B ) + ( g) ± a ( ) + g BR B g + l = (BR B ) + g ± (BR B ) + a g (BR B ) + g + l. (9) From Eq. (5), the optimal controller is given as k = R B (BR B ) + g R B (BR B ) + a g (BR B ) + g + l = R B (BR B ) + g ± BR B a g (BR B ) + g + l, () where the condition () is satis ed because g (BR B ) + g + l. In order to solve the HJ equation (6), we have to solve for ρ with two requirements. The rst requirement is that V is symmetric the second requirement is that V is positive de nite. Using Eqs. () (5), Eq. (9) can be written as V (x) = P + (x)(g(x) ± ρ(x)), () the second partial derivative of the value function is given as ( V (x) g(x) = P + (x) ± ρ(x) ) + P + (x) (I n (g(x) ± ρ(x))).() ( And V ) V = imply that V is a scalar function symmetric. Thus, from Eq. (), the symmetry requirement is given as (7) the non-negative de nite requirement is given as (8). Remark. Note that the value function, V (x) does not have to be explicitly determined.

3 Remark. The symmetry requirement, (7) is satis ed for all scalar cases. Furthermore, if P + (x) is a constant matrix then we obtain a simpler symmetry requirement, P + ± ρ ) = ± ρ ) P +. Remark. Note that the above optimal controller (6) is in the same form as the H optimal controller obtain by Aliyu using the inversion method. Also note that a(x) is a free parameter whose only constraint is that its magnitude is one. Remark. Note that V serves as a Lypunov function, thus with the usual zero-state detectability condition on (g(x), l(x)), guarantees the asymptotic stability of the optimal feedback system. Corollary.: Assume that l(x) R(x) > are bounded for all x. Then an in nite horizon cost function, tf J(x,k) = lim x (τ)q(x)x(τ) + k (x)r(x)k(x)dτ, t F t () converges to the minimal value the optimal controller is determined to be k (x) = R (x)b (x)p + (x)g(x) ± ρ(x), () which is the same controller as in Eq. (6). Proof. Optimal controller can be derived as in the nite time horizon case. We need to show that the cost function converges to a limiting value. Eq. () converges to a nite number as t F because we assumed that l(x) R(x) are bounded. This number is an upper bound for the minimal value of the criterion. Moreover, Eq. () is monotonically nondecreasing function of t F because l(x) R(x) > for all x. This proves that the minimal value of J has a limit as t F because it has a upper bound it is monotonically decreasing. Thus the optimal controller (6) is a solution for the in nite horizon case.. Example. As an example, we will consider a version of Freeman Kototovic s example given in 5, 6. Consider the following system with a cost function dx(t) = x(t) x (t)dt + u(t)dt J = tf t (x + u )dt, Now, nd the solutions of minimal rst cumulant control problem. In this example, we take g(x) = x x,b =,R = /, l(x) = x /. We determine that P = ρ(x) = a(x) x 6 x + x. Substituting these values into the optimal controller equation (), we have k = x + x x 6 x + x, (5) where a(x) = for the scalar case. This is the same optimal controller obtained using pointwise min-norm control law in 6. The rst requirement (7) is satis ed because this is a scalar example. The second requirement (8) is numerically determined to be satis ed for < x <.766. Figure shows that how the state approaches zero exponentially using the optimal control (5). The initial states were chosen as.9,.5,.. The a(x) was chosen to be one. State Time Fig.. The state, x, as a function of time, t. Example. Consider the following second order nonlinear system. ẋ x = + x + u(t), (6) ẋ x with the state weighting matrix, Q = the control weighting matrix, R = /. Using Eqs. () (5), we have P = / P + = / ρ(x) = 7 a (x)x where a(x) = a (x)a (x) is the arbitrary unit vector. Then using Eq. (6) () we nd V ( = ) 7 ± a (x) x k = ( ± ) 7 a (x) x. It is straight forward to verify that the rst requirement, Eq. (7) is satis ed. also the second requirement is found to be ± a. (x) The second requirement is satis ed for an appropriate a (x). Figure is the phase portrait of the system for

4 various initial values of x x. In plotting Figure, we used a (x) = the second requirement was satis ed. To compare with the pointwise min-norm controller, Figure shows the phase portrait of the same system using the pointwise min-norm controller. x Fig.. x Fig x 6 5 The phase portrait of Example using an optimal controller x The phase portrait of Example using a min-norm controller A. Solutions via Inversion Method Now we will nd the solutions to the nonlinear deterministic optimization problem using the inversion method of show that it is equivalent to our solution. Let P(x) = B(x)R (x)b (x). Also, let M N be differential manifolds which are locally Euclidean compact. T M is the tangent bundle of M with dimensions n. Theorem.: For the HJ equation (6), we assume that there exists a vector eld ρ : N TM, such that = ρ (x)p + (x)ρ(x) + g (x)p + (x)g(x) + l(x)(7) where the matrix P + is the pseudo-inverse of P. Then V (x) = P + (x)(g(x) ± ρ(x)), (8) satis es the HJ equation (6). Proof. Direct substitution of Equation (8) into Equation (6) gives the discriminant equation (7). Remark. Equation (7) is called the discriminant equation. This equation is a rst order quasi-linear partial differential equation that can be solved by the classic method of characteristics. This discriminant equation corresponds to the Riccati equation in linear quadratic regulator problem. In order to nd the optimal controller, we have to solve the discriminant equation with two requirements given in Eqs. (7) (8). The optimal controller is given by the following equation, k = R (x)b (x)p + (x)(g(x) ± ρ(x)), which is equivalent to Eq. (6). B. Interpretation via Control Lyapunov Function If the control problem is to nd a feedback control law, k(x) such that the closed loop system ẋ = f(x,k(x)) (9) is globally asymptotically stable at the equilibrium point x =, then we can choose a function V (x) as a Lyapunov function, nd k(x) to guarantee that for all x IR n such that V (x) (g(x) + B(x)k(x)) W(x). () A system for which an appropriate choice of V (x) W(x) exists is said to possess a control Lyapunov function (clf). Following 7 we de ne the control Lyapunov function, V (x), of system (9) to be smooth, positive de nite, radially unbounded if inf u { V f(x,u) } <, () for all x not equal to zero. In order to relate the clf inequality to the HJ equation of the optimal control, we introduce the following lemma by Curtis Beard with a slight modi cation. Lemma.: If S = S > then the set of solutions to the quadratic inequality ζ Sζ + d ζ + c () where ζ IR n is nonempty if only if d S d c () is given by ζ = S d + S / v d S d c () where v B(IR n ) = {ζ IR n : ζ }.

5 Proof. See, modify the inequalities. Now, we state the theorem that gives all stabilizing controller. Theorem.: Assume that V (x) is smooth, positive de nite, radially unbounded function. Assume also W(x) = l(x) + k R(x)k. For the nonlinear system (), all state feedback stabilizing controllers are given by if only if k = R B V + R / v V BR B V V g l (5) V BR B V V g l. (6) Proof. From (), we obtain the control Lyapunov function inequality as V (g + Bk) + l + k Rk. (7) Then we let ζ = k,s = R,d = B V, c = V g +l use Curtis Beard s Lemma. on Eq. (7), to obtain all the stabilizing controller. Remark. Equation (7) states that HJ equation for the optimal value function is less than or equal to zero. Equation (6) states that HJ-type equation for a clf function, V, has to be greater than or equal to zero. Thus, for the optimal value function to be a clf, the HJ equation has to be equal to zero. Remark. Following, we can also show that k in Eq. (5) is inversely optimal for some R(x) l(x). Here we discuss the robustness property of the optimal controller. We follow the de nition of stability margin given in : an asymptotically stabilizing control law, u = k(x), has stability margins (m,m ) where m < m, if for every α (m,m ),u = ( + α)k(x), also asymptotically stabilizes the system. Theorem.: If k(x) is stabilizing control of (5) V B(x)R / (x)v(x) then it has stability margin of ( /, ). Proof. We closely follow the proof given for the robust satis cing control in, Theorem. From (), we obtain the control Lyapunov function inequality as V (g + Bk) l k Rk. Add α V Bk to both sides to obtain V g +(+α) V Bk l k Rk +α V Bk. (8) Non-positiveness of the right h side guarantees asymptotic stability. Eq. (5) is rewritten as k = R B V + R / vσ. Substituting this k into Eq. (8), after some algebraic manipulation we obtain, l k Rk + α V Bk = l σ v v ( + α) V BR B V + ( + α) V BR / vσ. The rst second term on the right side of the equality are nonpositive. The third term on the right side are nonpositive if α ( /, ), The last term on the right side are nonpositive if α ( /, ) V B(x)R / (x)v(x). III. STATE DEPENDENT RICCATI EQUATION Here we will derive the State Dependent Riccati Equation (SDRE) for a value function that is in quadratic form using Kronecker products. We nd the partial differential equations from the necessary conditions of optimality. Theorem.: Assume that l(x) = x Qx V (x) is a symmetric nonnegative de nite matrix, V (x) = x V (x)x. (9) For the nonlinear system given by () with g(x) = A(x)x(t), the optimal controller that minimizes the cost function () with l(x) = x Q(x)x, is given by (suppressing the argument x) k = R B V = ( R B V x + (I n x ) V ) x. Provided that the following conditions are satis ed: () = A V + V A + Q V BR B V, () R B (I n x ) V = R B V x =, () A(x) (I n x ) V (x) =. () Proof. Suppressing arguments for simplicity, we obtain the following HJ equation using Eqs. () (5) = x A V ( ) V BR B V + x Qx. () This is the necessary partial differential equations for the optimal control problem. Because V (x) is assumed to be symmetric nonnegative de nite matrix, we have V = V (x)x + (I n x ) V x, (5)

6 where is the Kronecker product. Substitute the above equations into Equation () to obtain = x Qx x V BR B V x (I n x ) V x BR B V x (I n x ) V x BR B (I n x ) V x +x A V x + x A (I n x ) V x. (6) Rewriting the above equation, we get = x ( Q V BR B V + A ) V x +x V (I n x ) BR B V V (I n x)br B (I n x ) V x. (7) +A (I n x ) V Thus, we obtain the three conditions in Eqs. (), (), (). And the optimal controller is given by Eq. (). Remark. Note that for the special case when A(x) B(x) are in the phase-variable form of the state equation, the condition () is satis ed. Furthermore, both conditions () () are satis ed if V = for i =,,n. i Example. Consider Example in Section II. Just rewriting the system equation we have ẋ x = + ẋ u(t). (8) Substituting the same Q R matrices as in Example, using Eq. () we obtain, V = 7. And from Eq. () the optimal controller is determined to be ( ) 7 k = ± x. This is the same controller as Section II, Example. Because V is a constant in this example, the state dependent Riccati equation () was the only requirement. REFERENCES M. D. S. Aliyu, An Approach for Solving the Hamilton-Jacobi- Isaacs Equation (HJIE) in NonlinearH Control, Automatica, Vol. 9, Issue 5, pp , May. C. I. Byrnes, A. Isidori, J. C. Willems, Passivity, Feedback Equivalence, the Global Stabilization of Minimum Phase Nonlinear Systems, IEEE Transactions on Automatic Control, Vol. 6, No., pp. 8-, November 99. J. R. Cloutier, C. N. D Souza, C. P. Mracek, Nonlinear Regulation Nonlinear H Control Via the State-Dependent Riccati Equation Technique: Part I, Theory; Prat II, Examples Proceedings of the International Conference on Nonlinear Problems in Aviation Aerospace, pp. 7, May 996. J. W. Curtis R. W. Beard, Satis cing: A New Approach to Constructive Nonlinear Control, IEEE Transactions on Automatic Control,, to appear. 5 Freeman, R. A. Kokotovic, P. V., Optimal Nonlinear Controllers for Feedback Linearizable Systems, 99 Workshop on Robust Control via Variable Structure Lyapunov Techniques, Benevento, Italy, September Freeman, R. A. Kokotovic, P. V., Inverse Optimality in Robust Stabilization, SIAM J. on Control Optimization, Vol., No., pp. 65-9, July Miroslav Krstic, Ioannis Kanellakopoulos, Petar Kokotovic, Nonlinear Adaptive Control Design, John Wiley & Sons, Inc R. W. Liu J. Leake, Inverse Lyapunov Problems, Technical Report No. EE-65, Department of Electrical Engineering, University of Notre Dame, August Primbs, J. A. V. Nevistic, J. C. Doyle, A Receding Horizon Generalization of Pointwise Min-Norm Controllers, IEEE Transactions on Automatic Control, Vol. 5, No. 5, pp , May. Sain, M. K. Won, C.-H. Spencer Jr., B. F. Liberty, S. R., Cumulants Risk-Sensitive Control: A Cost Mean Variance Theory with Application to Seismic Protection of Structures, Advances in Dynamic Games Applications, Annals of the International Society of Dynamic Games, Volume 5, pp. 7-59, Jerzy A Filar, Vladimir Gaitsgory, Koichi Mizukami, Editors. Boston: Birkhauser,. IV. CONCLUSIONS For a general nonlinear system, we present a method to determine the optimal controller using pseudo-inverse method. Then we relate this method with the inversion method of Aliyu. Furthermore, the value function is related to the control Lyapunov function this gives the stabilizing controller with good stability margin. State dependent Riccati equation optimal controller is derived for a general nonlinear system. Simple examples illustrate the developed optimal controller determination procedures.