LECTURE NOTES IN CALCULUS OF VARIATIONS AND OPTIMAL CONTROL MSc in Systems and Control. Dr George Halikias

Transcription

1 Ver.1.2 LECTURE NOTES IN CALCULUS OF VARIATIONS AND OPTIMAL CONTROL MSc in Systems and Control Dr George Halikias EEIE, School of Engineering and Mathematical Sciences, City University 4 March 27

2 1. Calculus of variations 1.1 Introduction Calculus of variations in the theory of optimisation of functionals, typically integrals. Perhaps the first problem in the calculus of variations was the brachistochrone problem formulated by J. Bernoulli in 1696: Consider a bead sliding under gravity along a smooth wire joining two fixed points A and B not on the same vertical line. What is the shape of the wire in order that the bead, when released from rest at point A, slides to B in minimum time? A a x g yx b y B The brachistochrone problem The figure shows the choice of axes with A taken to be the origin without loss of generality. Here we required to minimise: B A dt = B A ds v where s is the arc-length along the wire and v is the instantaneous speed of the bead. Here total energy is conserved, so, if m is the mass of the bead, On noting that ds = dx 2 + dy 2 = dx 1 2 mv2 = mgy v = 2gy dy = dx 1 + y dx 2 2 of 58

3 we have to minimise: J[y] = 1 a 1 + y 2 2g y with y = and ya = b. For each curve yx joining A and B, J[y] has a numerical value for the time taken. Thus J acts on a set of functions to produce a corresponding set of numbers. This is unlike what we think as ordinary functions which typically map numbers to numbers. To mark the difference, integrals like J[y] above are called functionals. Calculus of variations deals with optimisation problems of the type described above. We will generalise this class of problems by imposing additional integral constraints e.g. related to the total length of the curve yx or, possibly, relaxing others e.g. those related to the fixed end-points. To describe our problems rigorously we also need to specify clearly the class of the functions over which the optimisation is carried out. 1 2 dx 1.2 The fixed-end-point problem Here we change notation and consider functions xt, where t is the independent variable and x the dependent, so that xt defines the equation of a curve. This is so that we have a smoother notational transition to optimal control problems to be discussed later!. The problem considered here is to find, among all curves in a specified class joining two fixed points t, x and t 1, x 1, the equation of the curve minimising a given functional. This functional is the integral from t to t 1 of a given function ft, xt, ẋt where ẋ = dx/dt. We assume that f is sufficiently smooth, i.e. differentiable with respect to each of its three variables t, x and ẋ as many times as required. Note that the three variables are considered to be independent i.e. the value of x at some time t does not constrain the gradient ẋ at t. Thus the problem is: min J[x] = t1 t ft, x, ẋdt 1 with x t = x and xt 1 = x 1. If x = x t is a minimising curve, then we have that J[y] J[x ] for all other curves yt in a specified class - see later! satisfying the end constraints. 3 of 58

4 Admissible classes of functions that can be considered include C continuous, C 1 once continuously differentiable, C 2 twice continuously differentiable and D 1 continuous piecewise-differentiable, i.e. continuously differentiable except at a finite - or countable - number of points. To minimise technicalities we start by taking the admissible class of functions to be C 2. We first need to define the concept of closeness between functions. Definition - Weak variations: Let x t be the minimising curve and yt an admissible curve. Then, if there exist small numbers ɛ 1 and ɛ 2 such that: x t yt ɛ 1 and ẋ t yt ɛ 2 for all t [t t 1 ] we will say that yt is a weak variation of x t. Note that we require not only the functions but also their derivatives to be close. Definition - Strong variations: Let x t be the minimising curve and yt an admissible curve. Then, if there exist a small numbers ɛ such that: x t yt ɛ for all t [t t 1 ] we will say that yt is a strong variation of x t. x x x*t x*t B B A t A t Weak and strong variations Let x t be a local minimiser of 1 in the class of C 2 functions and define xt = x t + ɛηt where ɛ is a small quantity independent of x, η and t. Then 4 of 58

5 xt will be a weak variation of x t provided ηt is C 2 and ηt = ηt 1 =. The difference J = J[x] J[x ] is known as the variation of J. Hence, t t1 J = t1 t fx + ɛη, ẋ + ɛ η, tdt t1 t fx, ẋ, tdt At each t [t t 1 ] expand the first integrand as a Taylor series in the two variables ɛη and ɛ η around x, x. Then, t1 { J = fx, ẋ, t + ɛη f f + ɛ η x ẋ + 1 } ɛ 2 η 2 2 f 2 x + 2 2ɛ2 η η 2 f x ẋ + ɛ2 η 2 2 f dt ẋ 2 t fx, ẋ, tdt + Oɛ 3 where all partial derivatives are evaluated on the optimal curve and where Oɛ 3 denotes terms of at least order three. This may be written as t J = ɛv 1 + ɛ 2 V 2 + Oɛ 3 where V 1 and V 2 denote the first and second variations: t1 V 1 = ηt fx, ẋ, t + ηt fx, ẋ, t dt x ẋ and V 2 = 1 2 t1 t η 2 t 2 fx, ẋ, t + 2ηt ηt 2 fx, ẋ, t + η 2 t 2 fx, ẋ, t dt x 2 x ẋ ẋ 2 respectively. If x t is minimising then it is necessary that: J = ɛv 1 + ɛ 2 V 2 + Oɛ 3 for all admissible ηt i.e. all C 2 functions in the interval [t t 1 ] such that ηt = ηt 1 =. Now ɛ can be either positive or negative; hence dividing by ɛ gives: V 1 + ɛv 2 + Oɛ 2 for ɛ > and V 1 + ɛv 2 + Oɛ 2 for ɛ < Now taking the limit as ɛ shows that V 1 and V 1 which can only be satisfied if V 1 =. Thus a necessary condition for a minimum is that the first variation V 1 =, i.e. t1 t ηt fx, ẋ, t + ηt fx, ẋ, t dt = x ẋ 5 of 58

6 Integrating the second term by parts gives t1 η f [ ẋ dt = η f ] t1 t1 η d ẋ dt t t t t f t1 dt = η d ẋ t dt f dt ẋ since ηt = ηt 1 =. Thus, the necessary condition becomes: t1 { f ηt x d } f dt = 2 dt ẋ for all admissible η, i.e. all C 2 functions in the interval [t t 1 ] such that ηt = ηt 1 =. Note that the term in the curly brackets is continuous and does not involve the variation ηt. We now make use of the following result: Lemma 1: If gt is continuous in an interval [t t 1 ] and if t1 t ηtgtdt = for every function ηt C 2 t, t 1 such that ηt = ηt 1 =, then gt = for every t [t t 1 ]. Proof: Suppose the function gt is non-zero, say positive, at some point t in the interval [t t 1 ]; then from continuity it must also be positive in some interval [α β] contained in [t t 1 ]. Define: ˆηt = t α 3 β t 3 for t [α β] = for t / [α β] Clearly, the function ˆηt defined above is continuous everywhere in the interval [t t 1 ] and has also continuous first and second partial derivatives at every point in this interval the only points of contention are t = α and t = β at which ˆηt and its first two derivatives may be easily verified to be continuous. Since further ˆηt = ˆηt 1 = we have from the Lemma s hypothesis that: t1 t ˆηtgtdt = β α ˆηtgtdt = which is a contradiction since both ˆηt and gt are strictly positive everywhere in the interval α < t < β. Applying the Lemma to 2 gives a necessary condition for a minimum as: f x d f = dt ẋ 6 of 58

7 which is known as the Euler-Lagrange equation. We summarise this important result as follows: Theorem 1: In order that x = x t should be a solution, in the class of C 2 functions, of Problem 1, it is necessary that at each point of x = x t. Note 1: f x d dt f = ẋ The Euler-Lagrange equation also gives necessary conditions for a local maximum as can be seen by adapting the above derivation!. solutions to the Euler-Lagrange equation are called extremals. For this reason the Note 2: Although here the Euler-Lagrange equation was derived for C 2 curves, it is possible to show that if we enlarge the admissible class to C 1 curves once continuously differentiable the Euler-Lagrange equation still holds along a minimising solution. Example: Find the extremal of the functional J[x] = 2 1 ẋ2 t 3 dt, given that x1 = and x2 = 3. We have t = 1, t 1 = 2, x =, x 1 = 3 and fx, ẋ, t = ẋ 2 t 3. The Euler-Lagrange equation in this case gives: f x d dt giving ẋt 3 = constant. On integrating we find: f = d ẋ dt 2ẋt3 = xt = k t 2 + l for two constants k and l When we apply the end conditions we get x1 = k + l = and x2 = 3 k + l = 3. Solving gives the extremal as: 4 x t = 4 4 t Special form of the Euler-Lagrange equation Suppose that the function f in problem 1 is independent of t. In this case the Euler- Lagrange equation simplifies. To show this let f = fx, ẋ. Then: d f ẋ f = f dt ẋ xẋ + f ẋẍ ẍ f ẋ ẋ d f dt ẋ 7 of 58

8 Hence d f ẋ f f = ẋ dt ẋ x d dt f ẋ On an extremal the RHS of this equation is zero and hence the Euler-Lagrange equation in this case reduces to: f ẋ f ẋ = constant Example: Consider the brachystochrone problem in new x, t notation: This resulted in the minimisation: min J[x] = a 1 + ẋ 2 x 1 2 dt Here the integrand is independent of t and hence the Euler-Lagrange equation reduces to: or fx, ẋ ẋ f ẋ = constant 1 + ẋ2 1 2 x ẋ 1 + ẋ ẋ x 1 2 = constant 1 + ẋ ẋ 2 = constant x1 + ẋ 2 = c constant x 1 2 x ẋ This may be solved in parametric form exercise! by using the substitution ẋ = tanφ as: x = k1+cos 2φ and t = l k2φ+sin 2φ, where k and l are constants determined by the end-conditions. The equation of the extremising curve actually minimising is that of a cycloid. This is the path traced by a point on the edge of a disc as the disc rolls in a straight line A formal setting for calculus of variations In order to formalize the arguments used in the calculus of variations we need to generalise the concepts of distance and continuity to function spaces. An approach for this is outlined in this section. Recall that by a linear space we mean a set R of elements x, y, z,..., for which the operations of addition and multiplication by real numbers more generally elements of a field α, β,..., are defined according to the following axioms: 1. x + y = y + x; 2. x + y + z = x + y + z; 8 of 58

9 3. There exists an element zero element such that x + = x for every x R; 4. For each x R there exists an element x such that x + x = ; 5. 1 x = x; 6. αβx = αβx; 7. α + βx = αx + βx; and 8. αx + y = αx + βy. A linear space R is said to be normed, if each element x R is assigned a non-negative number x the norm of x, such that: 1. x = if and only if x = ; 2. αx = α x ; and 3. x + y x + y. For our purposes we are especially interested in real function spaces. We define: The space C or more precisely Ca, b consisting of all continuous functions xt defined on the closed interval [a, b]. By addition of elements of C and multiplication of elements of C by numbers we mean the normal operations of function additions and number-function multiplication, respectively, while the norm is defined as the maximum absolute value, i.e. x = max a t b xt. The space C 1, or more precisely C 1 a, b, consisting of all functions xt defined in the interval [a b] which are continuous and have continuous first derivatives. The operations of addition and multiplications are the same as in C, but now the norm is defined as: x 1 = max a t b xt + max a t b ẋt where ẋ is the derivative of xt. Thus, two functions in C 1 a, b will be regarded to be close-together say within a distance ɛ if both functions themselves and their derivatives are close, i.e. y z 1 < ɛ implies that yt zt < ɛ and ẏt żt < ɛ for every a t b. 9 of 58

10 The space C n, or more precisely C n a, b, consisting of all functions xt defined in the interval [a b] which are continuous and have continuous n first derivatives. The norm in C n is defined as: x n = n i= max a t b x i t where x i t is the i-th derivative of xt. Note in particular that C n C n 1... C 1 C and that if x C n then x n < ɛ x n 1 < ɛ... x 1 < ɛ x < ɛ. Functionals J can now be defined as maps from a linear space R to the set of real numbers R, i.e. J : R R. R will be normally taken as a real function space e.g. C 1 a, b or C 2 a, b depending on context. In a similar way that continuity is defined for functions f : R R we can define continuity of functionals. Definition: The functional J[x] is said to be continuous at a point ˆx R if for any ɛ > there exists a δ > such that: J[x] J[ˆx] < ɛ provided that x ˆx < δ. Next, we can define formally the concept of variation or differential of a functional, analogously to the concept of differential of a function of n variables. We first give the following definition: Definition: Given a normed linear space R, let each element x R be assigned a real number φ[x], i.e. let φ[x] be a functional defined in R. Then φ[x] is said to be a continuous linear functional if: 1. φ[αx] = αφ[x] for any x R and any real number α; 2. φ[x 1 + x 2 ] = φ[x 1 ] + φ[x 2 ] for any x 1 and x 2 R; and 3. φ[x] is continuous for all x R. Example: We associate with each function xt Ca, b its value at a fixed point t [a, b], i.e. we define the functional φ[x] by the formula φ[x] = xt ; then φ[x] is a linear functional on Ca, b. Example: The integral φ[x] = b xtdt defines a linear functional on Ca, b. a 1 of 58

11 Exercise: Show that the two functionals defined in the above examples are linear by verifying the three properties of the definition. Let J[x] be a functional defined on some normed linear space and let J[h] = J[x + h] J[x] be its increment corresponding to the function h = ht. If x is fixed, J[h] is a functional of h nonlinear in general. Suppose that J[h] = φ[h] + ɛ h and that ɛ as h. Then the functional J[x] is said to be differentiable and the principal linear part of the increment J[h], i.e. the linear functional φ[h] which differs from J[h] by an infinitesimal of order higher than one relative to h is called the variation or differential of J[x] and is denoted by δj[h]. It may be easily shown that the variation of a differentiable functional is unique. We say that the functional J[x] has a relative extremum for ˆx if J[x] J[ˆx] does not change sign in some neighborhood of ˆx. We are normally concerned with functionals defined on a set of continuously differentiable functions and the functions themselves can be regarded as elements either of C or C 1. Correspondingly, we can define two types of extrema, weak and strong. We will say that the functional J[x] has a weak extremum for x = ˆx if there exists an ɛ > such that J[x] J[ˆx] has the same sign for all x in the domain of definition of the functional which satisfy x ˆx 1 < ɛ. On the other hand, we will say that J[x] has a strong extremum for x = ˆx if there exists an ɛ > such that J[x] J[ˆx] has the same sign for all x such that x ˆx < ɛ. Clearly, every strong extremum is also a weak extremum, although the converse is not true in general. It is now possible to show that a necessary condition for a differentiable functional to have an extremum for x = ˆx is that its first variation vanishes at x = ˆx, i.e. that δj[h] = for x = ˆx and all admissible functions h. This leads to the Euler-Lagrange equation as a necessary condition for the functional: J[x] = to have a weak extremum. t1 t fx, ẋ, tdt 1.3 Problems in which the end points are not fixed Here we will consider the modified problem: min J[x] = t1 ft, x, ẋdt 3 t 11 of 58

12 where t, xt is fixed but t 1, xt 1 is required to lie on some differentiable given curve x = ct. We will derive again necessary conditions for a minimum. If x = x t be a minimising curve and suppose it intersects the target curve at t = t 1. Let yt = x t+ɛηt be a weak variation starting at t, xt and reaching the target curve at t = t 1 + τ, where τ is small. For a weak variation τ will be Oɛ. yt=x*t+εηt ct x*t A t t 1 t 1 + τ t A weak variation with only one fixed end-point Now, using Taylor series expansion: yt 1 + τ = x t 1 + τ + ɛηt 1 + τ = x t 1 + τẋ t 1 + ɛηt 1 + Oɛ 2 and Thus, and hence yt 1 + τ = ct 1 + τ = ct 1 + τċt 1 + Oɛ 2 x t 1 + τẋ t 1 + ɛηt 1 = ct 1 + τċt 1 ɛηt 1 = ċt 1 ẋ t 1 τ 4 since x t 1 = ct 1. The variation in J is: J = t1 + τ t ft, x + ɛη, ẋ + ɛ ηdt 12 of 58 t1 t ft, x, ẋ dt

13 Expanding for each t the first integrand around x t, ẋ t gives ft, x + ɛη, ẋ + ɛ η = ft, x, ẋ + ɛη f f + ɛ η x ẋ + Oɛ2 where we keep only terms up to Oɛ since we are only interested in the first variation. Thus J = t 1 t1 t t1 { ft, x, ẋ + ɛη f } f t1 + τ { + ɛ η dt + ft, x, ẋ + ɛη f } f + ɛ η dt x ẋ t 1 x ẋ t ft, x, ẋ dt + Oɛ 2 On noting that the second integral can be written as: t1 + τ { ft, x, ẋ + ɛη f } f + ɛ η dt = ft 1, x t 1, ẋ t 1 τ + Oɛ 2 x ẋ we get t J = t1 t { ɛη f } f + ɛ η dt + ft 1, x t 1, ẋ t 1 τ + Oɛ 2 x ẋ Integrating the second term by parts gives t1 η f [ ẋ dt = η f ] t1 t1 η d f dt = ηt 1 f ẋ dt ẋ ẋ t 1, x t 1, ẋ t 1 t t t t1 t η d dt f dt ẋ since in this case ηt = but the second end-point ηt 1 is not in general zero. Thus: t1 { f J = ɛ ηt x d } f dt+ft 1, x t 1, ẋ t 1 τ+ɛηt 1 f dt ẋ ẋ t 1, x t 1, ẋ t 1 +Oɛ 2 in which all the explicitly-written terms are Oɛ. minimisation problem in which both end-points are fixed, i.e. to pass through points t, xt and t 1, x t 1. Now consider an auxiliary xt is constrained Clearly for this problem x t is still an extremum: To see this clearly, suppose for contradiction that there existed an admissible optimal curve y t in the same class as x t and satisfying the fixed end-point constraints y t = x, y t 1 = x t 1, for which J[y ] < J[x ]; then y t would also be an optimal solution for the original problem end-point lies on ct and that would contradict minimality of x t for the original problem. Thus x t must be a minimising curve for the fixed-end problem, and therefore from section 1.1 it must satisfy the Euler-Lagrange equation at every point of the optimal curve. Thus the term inside the curly brackets in the above equation must be zero, and we get: J = ft 1, x t 1, ẋ t 1 τ + ɛηt 1 f ẋ t 1, x t 1, ẋ t 1 + Oɛ 2 13 of 58

14 Using equation 4 gives: { J = τ ft 1, x t 1, ẋ t 1 + ċt 1 ẋ t 1 f } ẋ t 1, x t 1, ẋ t 1 + Oɛ 2 If this extremal is to minimize J, then using the same argument as in the proof of Theorem 1, the first variation must vanish e.g. consider arbitrary signs in τ and hence we have the necessary condition: ft 1, x t 1, ẋ t 1 + ċt 1 ẋ t 1 f ẋ t 1, x t 1, ẋ t 1 = at the end-points of the extremal, when the end-point is not fixed but constrained to lie of the fixed curve ct. Note that all the terms are evaluated at time t 1, so this is an algebraic equation relating the slope of ct and the slope of the extremal at the point where they meet. Conditions of this type are called transversality conditions. Of course this condition must be satisfied in addition to the Euler-Lagrange equation which is also a necessary condition. Example: Find the extremal of 1 ẋ2 t 3 dt given that x1 =, T > 1 is finite and xt lies on the curve x = 2/t 2 3. Note that this is essentially the same example as in section 1.1 except that now the end-point is not fixed. Using the Euler-Lagrange equation the extremals were found to be: xt = k t 2 + l k and l arbitrary constants Since x1 = we have l = k; thus xt = k k and ẋ = 2k. The target curve is t 2 t 3 ct = 2 3, so ċ = 4. The transversality condition gives: t 2 t 3 ẋt 2 T ċt ẋt ẋt T 3 = Now T is finite and k otherwise the extremal would be xt =, so the tranversality condition gives k = 4. Thus the required extremal is x t = 4 t 2 4, which meets the target curve xt = 2 t 2 3 at T = 2, xt = of 58

15 1.3.1 Special forms of the transversality conditions If the problem is one in which either t 1 or xt 1 is specified while the value of the other is completely free, then the transversality condition can be simplified. Consider each case separately: 1. xt 1 fixed, t 1 is free: Here the target curve is x = ct = constant and hence ċ =. The transversality condition in this case becomes: ft 1, x t 1, ẋ t 1 ẋ t 1 f ẋ t 1, x t 1, ẋ t 1 = x x*t ct=constant A t 1 t 1 + τ t xt 1 fixed 2. t 1 fixed, xt 1 is free: Here the target curve is a straight line perpendicular to the t axis whose slope ċt 1 is infinite. Now, provided ċt, the transversality condition can be written as: 1 ċt 1 ft 1, x t 1, ẋ t 1 ẋ t 1 f ẋ t 1, x t 1, ẋ t 1 which for infinite ċt 1 simply reduces to: f ẋ t 1, x t 1, ẋ t 1 = + f ẋ t 1, x t 1, ẋ t 1 = 15 of 58

16 x x*t A t 1 t t 1 fixed Example: Find the extremal of J = x2 + ẋ 2 dt for each of the following cases: i x = 1, T = 2, and ii x = 1, xt = 2. The Euler-Lagrange equation is f x d dt f ẋ = or 2x 2ẍ = which gives ẍ x =. The general solution is xt = Ae t + Be t. We now distinguish between the two cases: i x = 1 so A + B = 1. Since xt is unspecified and T = 2 the appropriate end condition is fẋt = which gives ẋ2 =, so B = Ae 4. x t = cosht 2/ cosh2 and x2 = 1/ cosh2. The extremal is ii x = 1 so A + B = 1. Since T is unspecified and xt = 2 the appropriate end condition is ft ẋt fẋt = which gives AB =, so that the extremals are either e t or e t. Now xt = 2 so either 2 = e T or 2 = e T. The second of these has no positive solution for T. Thus the extremal is x = e t which cuts x = 2 at T = ln Finding minimising curves Note that the Euler-Lagrange equation derived earlier is only a necessary condition, i.e. it gives necessary conditions for a function to be a local minimiser of a functional J. As we have seen, the Euler-Lagrange equations also give necessary conditions for maximising functions. Think of this equation as the equivalent of the condition that the derivative of a function should vanish at a local minimum or maximum. What we need in order to distinguish between minimising and maximising solutions are sufficient conditions. Although such conditions have been developed field of extremals they 16 of 58

17 are unfortunately outside the scope of this course! complex problems read hardly ever! Fortunately in some cases for we can identify minima and maxima from simple additional arguments or by using geometrical/physical intuition. Consider the following example: Example: Find the extremal of J[x] = 2 1 ẋ2 t 3 dt given that x1 = and x2 = 3. Is this a minimum? The extremal was identified in a previous example as x t = 4 4 t 2. Let us consider the sign of J[y] J[x ] where y = yt is an arbitrary C 1 curve joining the fixed end-points. Then yt = x t + ηt where ηt C 1 and η1 = η2 =. Then J[y] J[x ] = t + η t 3 dt t 3 dt = 16 η + η 2 t 3 dt 3 t 3 1 = [16η] η 2 t 3 dt = η 2 t 3 dt since η1 = η2 =. Note that the integrand is non-negative in the interval t [1 2] and thus J[y] J[x ], i.e. x is actually a global minimiser among all C 1 functions which satisfy the end-point constraints note that nowhere in the above argument is was assumed that ηt is small. In fact we can say more than this: Consider a variation yt = x t + ηt in the class D 1 satisfying the end constraints. Here D 1 is the set of all continuous piece-wise differentiable functions, i.e. all continuous functions with a continuous derivative - except at most at a countable number of points. Then ηt would have a finite number of discontinuities in the interval [1 2], i.e. yt would be continuous with a finite number of corners where the derivative jumps. In this case we could split [1 2] into a number of sub-intervals in which ηt is continuous and repeat the calculation for J[y] J[x ]; we would still get that J[y] J[x ], and hence x is actually a global minimiser among the wider class of D 1 functions. This is not accidental as shown by the next Theorem: Theorem 2: If x = x t is a minimising curve in the class of C 1 functions, then it is also a minimising curve in the wider class of D 1 functions. Proof: See [9]. The following theorem is also stated without proof: Theorem 3: In order that a D 1 function is a minimiser for the fixed end-point problem it is necessary that: 17 of 58 1

18 1. The Euler Lagrange equation is minimised between corners and between a corner and an end-point. 2. f ẋ is continuous at a corner. 3. f ẋ f ẋ is continuous at a corner. Proof: See [9]. 1.5 Isoperimetric problems Here we consider the problem of finding an extremum to a functional subject to an equality constraint involving a second functional. Historically, the first problems of this type involved finding an optimal curve whose total length perimeter was fixed - hence the name. It turns out that the standard method used in the optimisation of functions in R n under equality constraints - Lagrange multipliers - also applies here: Problem IP: Minimise the functional J[x] = t1 t ft, x, ẋdt with xt = x, xt 1 = x 1, subject to the integral constraint: where c is a constant. I = t1 t gt, x, ẋdt = c Theorem 4: In order that x = x t is a solution of Problem IP iso-perimetric problem it is necessary that it should be an extremal of: t1 t ft, x, ẋ + λgt, x, ẋdt for a certain constant λ Lagrange multiplier. Proof: See [9]. Example: Minimise J = 1 ẋ2 dt with x = 2, x1 = 4 subject to the constraint 1 xdt = 1. Theorem 4 says that we should find the extremals of 1 ẋ2 + λxdt. The Euler- Lagrange equation is λ d2ẋ/dt =, or ẍ = λ/2. Integrating gives the solution: 18 of 58

19 xt = λt 2 /4 + kt + l The end conditions give l = 2, k = 2 λ/4. We find λ by using the constraint: 1 { λ 4 t2 + 2 λ } t + 2 dt = 1 4 which gives λ = 48 after some algebra. Hence the required extremal is xt = 12t 2 1t of 58

20 2. Optimal control 2.1 The general optimal control problem The theory of optimal control allows for the solution of a large class of non-linear control problems subject to complex state and control-signal constraints. The theory is an extension of classical calculus of variations since it does not rely of the smoothness assumptions made so far; indeed in most cases the optimal control is highly discontinuous bang-bang control, control along switching curves, slidingcontrol. The formulation of the problem involves the minimization of a costfunction subject to initial and terminal constraints which is reminiscent of calculus of variation problems. The optimal control signal is typically obtained either as a function of time u t or, more interestingly for control applications, in feedback form, i.e. as a function of the state u x. The most important result in this area is Pontryagin s maximum principle which gives necessary conditions for optimality under very general assumptions. The general optimal control problem can be formulated as follows: Suppose the plant is described by the non-linear time-varying dynamical equation ẋt = fx, u, t where the state xt R n and the control ut U R m, where U is some compact region of R m. With this system we associate the performance index: Jx = MxT, T + where [t, T ] is the time-interval of interest. t Lxt, ut, tdt Note that the terminal cost M, is a function of the terminal state and time, whereas the weighting function L,, penalizes time and the state and control variables at intermediate times. The problem is to find the optimal control u t U which drives the state-vector along an optimal trajectory x t, such that Jx is minimised, subject to a constraint on the final state of the form ψxt, T = for a given function ψ,. 2.2 A simplified optimal control problem The optimal control problem which will be considered here is simplified as follows: 1 The system dynamics are assumed to be time-invariant no explicit dependence of f on 2 of 58

21 t; 2 The terminal time T will be assumed fixed; 3 The constraint ut U on the control signal is removed, along with the terminal state constraint ψxt, T = ; this avoids some intricate questions on the existence of an optimal ut controllability. Note that removal of the direct constraint ut U does not mean that ut is allowed to be unrestrained - constraints on the size or energy, etc of the control input can be imposed indirectly through the penalty term L, under the integral sign, which is also taken not to depend explicitly on time. To summarize, we consider the following problem: Time interval of interest: [, T ], T fixed. System dynamics: ẋt = fxt, ut, xt R n, ut R m. ut is assumed piece-wise continuous in [, T ]; the solution of ẋt = fxt, ut is assumed to exist and to be unique for all t. Initial conditions: x = x fixed. Performance index: J[u] = MxT + Lxt, utdt. Functions L, and M are assumed to be differentiable in all their variables. Problem: Find u t, t T which minimizes J[u]. We develop necessary conditions for optimality using informal variational arguments. First suppose that u t, t T is the optimal solution and let x t be the corresponding optimal state trajectory. Next, consider the variation u + δut of u t, resulting in a state-trajectory x t+δxt, where δx = O δu. Now consider variation J: J = Mx T + δxt + Expanding using Taylor series: J = Mx T + δxt T MT + x Mx T Lx + δx, u + δudt Mx T Lx, u dt + O δu 2 Lx, u dt { Lx, u + δx T L } L + δut dt x u where all partial derivatives are evaluated on the optimal trajectory x t, u t. Ignoring terms O δu 2 gives the first variation of performance index as: δj = δx T M { x + δx T L } L + δut dt T x u 21 of 58

22 which may be written out in full as: δj = n j=1 MT x j δx j T + Note that as u t is optimal, δj =. { n j=1 L x j δx j + m j=1 } L δu j dt u j We now face a problem: The δx i and δu j are not independent increments, since they are linked via the dynamic constraints ẋ = fx, u. The standard procedure when we face constraints is to augment the cost function by introducing Lagrange multipliers. Then necessary conditions for a minimum of the constrained problem here the dynamics ẋ = fx, u act as constraints are transformed to necessary conditions for a minimum of the augmented unconstrained problem. The same technique is followed here, but since the constraints are continuous, the multipliers must now be timedependent. We need n scalar Lagrange variables, p i t, one for each dynamic constraint ẋ i t = f i x, u. Consider the n integrals: Φ i = p i tf i x, u ẋ i tdt i = 1, 2,..., n evaluated along the optimal trajectory x t, u t, and note that Φ i = for any ut not just the optimal. Since Φ i =, so is its first variation, i.e δφ i =. Now, [ Φ i = p i t f i x + δx, u + δu ẋ i + d ] dt δx i dt p i tf i x, u ẋ i dt Expanding, or Φ i = p i t δφ i = [ f i x, u + n j=1 f i x j δx j + m j=1 p i t [f i x, u ẋ i t] dt + O δu 2 p i t [ n j=1 Integrating the last term by parts, p i t d dt δx idt = [p i tδx i t] T f i x j δx j + since δx i =, and hence [ T n ] f i m f i δφ i = p i t δx j + δu j x j u j j=1 j=1 m j=1 f i u j δu j ẋ i t d dt δx i f i u j δu j d dt δx i ṗ i tδx i tdt = p i T δx i T ṗ i tδx i tdt dt + 22 of 58 ] dt ] dt ṗ i tδx i tdt p i T δx i T

23 Thus, by changing the order in which summation is carried out under the integral sign, n T n n δφ i = δx j p i t f T m n i dt + δu j p i t f i dt i=1 x j=1 i=1 j u j=1 i=1 j + δx T ṗtdt δxt T pt where the summation in the last integral is written as a vector product. Thus: n δj + δφ i = δx T M T n L n x + δx j + p i t f i dt i=1 T x j=1 j x i=1 j T m L n + δu j + p i t f i dt + δx T ṗtdt δxt T pt u j u j j=1 i=1 A necessary condition for u t to be optimal is that δj + n δφ i = i=1 since each δφ i =. The expression for the variation can be simplified considerably by introducing the Hamiltonean function: Note that: and Hx, u = Lx, u + p T fx, u = Lx, u + Hx, u x j Hx, u u j With this substitution we get that δx T M x pt T + = L x j + = L u j + n i=1 n i=1 n p i tf i x, u i=1 p i t f ix, u x j p i t f ix, u u j { H δx T x + ṗt + δu T H } dt = u is a necessary condition for optimality. Now, we have the Lagrange variables at our disposal and we make the choice: ṗt = H x pt = MT x which form the co-state of the system. With this choice we need δj = δu T H u dt = 23 of 58

24 for a minimum, or equivalently since δu is arbitrary, we have that H u = along the optimal trajectory, i.e. the Hamiltonean is an extremum at every point of the optimal trajectory. To summarize, to find the optimal solution to the problem: We define the Hamiltonean Hx, u, p = Lx, u + p T fx, u and set H u =. To find the optimal control we need to solve this condition simultaneously with: ẋ = H p = fx, u, x = x state equations and ṗ = H x pt = MT x co-state equations Note that the state equations need to be integrated forward in time from initial condition x = x and the co-state equations backwards in time from terminal condition pt = MT x. 24 of 58

25 2.3 The finite-horizon linear/quadratic regulator Here we consider the specific problem of determining the optimal control of a linear time-invariant LTI system with a quadratic performance index. The plant dynamics are described by the standard state-space equations: ẋt = Axt + But x = x where A R n n and B R n m. The performance index to be minimised is: J[u] = 1 2 xt tsxt x T Qx + u T Rudt where T is fixed, and S, Q, R > are all symmetric constant matrices. First, we form the Hamiltonean: Hx, u = 1 2 xt Qx + u T Ru + p T Ax + Bu To obtain the minimising solution we need to solve: H u = Ru + BT p = u = R 1 B T p ṗ = H x = Qx AT p pt = MT = SxT x together with the dynamic equations. These can be assembled together in matrix form as: ẋ A BR 1 B T x = ṗ Q A T p subject to the initial conditions: x = x and pt = SxT. This is a twopoint boundary value problem with part of the boundary conditions specified at the initial time t = x = x and the other part at the terminal time t = T pt = SxT. The matrix defining the dynamic map has a special structure and is called a Hamiltonean matrix; the overall dynamic map defines a Hamiltonean system. To solve the problem inspired from the boundary condition pt = SxT, assume that we have pt = P txt for some unknown matrix function P t, which satisfies 25 of 58

26 P T = S. If we can find such a P t, then the assumption is valid. Now differentiating the co-state equations we get: ṗ = P x + P ẋ = P x + P Ax + Bu = P x + P Ax BR 1 B T p = P x + P Ax P BR 1 B T p = P x + P Ax P BR 1 B T P x Using the co-state equation ṗ = Qx A T p = Qx A T P x gives P x = A T P + P A P BR 1 B T P + Qx for t [ T ]. Since this must hold for all state-trajectories given any x = x, it is necessary that: P = A T P + P A P BR 1 B T P + Q for t [ T ]. This is a differential matrix equation Riccati equation and if P t is its solution with final condition P T = S, then pt = P txt for all t [ T ], so that our assumption is justified. Note that this can be solved backwards in time e.g. by numerical integration from the terminal condition P T = S. Note also that if the differential Riccati equation is transposed it remains the same except that P t is replaced by P T t. Since the terminal condition is symmetric by assumption, i.e. P T = S = S T, this implies that P T = P for all t T. It can further be shown that under the assumptions that S = S T, Q = Q T and R = R T >, the solution P t to the differential Riccati equation is unique. Note that the solution to the problem is obtained in state-feedback form, i.e. u t = Ktxt where Kt denotes the Kalman gain Kt = R 1 B T P t. Note further that the Kalman gain is time-varying. Of course we have not yet shown that u t obtained above is actually the minimizing solution, i.e. that it results in the smallest performance index among all admissible controls ut defined on the interval [ T ]. This follows by considering by the following identities: x T T P T xt x T P x = = = d dt xt P xdt ẋ T P x + x T P x + x T P ẋdt { x T P } + A T P + P Ax + u T B T P x + x T P Bu dt 26 of 58

27 Using the fact that P T = S, the performance index can be written as: J[u] = 1 2 xt T SxT = 1 2 xt P x = 1 2 xt P x = 1 2 xt P x xt P x x T Qx + u T Rudt with equality if and only if ut = R 1 B T P txt. { x T P } + A T P + P A + Qx + u T B T P x + x T P Bu + u T Ru dt { x T P BR 1 B T P x + u T B T P x + x T P Bu + u T Ru } dt u + R 1 B T P x T Ru + R 1 B T P xdt B + +. xt 1/s xt u*t A -R -1 B T Pt Finite-horizon LQR We summarize the results of this section with a Theorem: 27 of 58

28 Theorem 1: Given: A system model: ẋt = Axt + But, x = x and, A performance index J[u] = 1 2 xt T SxT + xt tqxt+u T trutdt with S = S T, Q = Q T and R = R T >, then: The optimal feedback control in [ T ] is given as u t = Ktxt where Kt = R 1 B T P t and P t is the unique symmetric solution of the differential matrix Riccati equation P = A T P + P A P BR 1 B T P + Q with terminal condition P T = S. The corresponding optimal performance index is J[u ] = 1 2 xt P x. 2.3 The infinite horizon linear/quadratic regulator In this section we consider the infinite-horizon LQ optimal control problem. This is obtained from the finite-horizon problem by setting the terminal cost matrix S to zero and taking the limit T. In particular, we seek an optimal control signal ut, t which minimizes the performance index: J[u] = 1 2 x T Qx + u T Rudt where Q = Q T and R = R T >, subject to the plant dynamics constraints ẋt = Axt + But, x = x. It will be shown that in this case there exists an optimal control which solves the problem, provided we make certain controllability and observability assumptions which can be relaxed!. The optimal control is obtained in a time-invariant state-feedback form and depends on the solution of an algebraic matrix Riccati equation. Let us denote by Πt, T for t [; T ] the solution of the Riccati differential equation with terminal condition ΠT, T = P T =. We also write the cost function in the more explicit form Ju, x, T to emphasize the initial condition and time-horizon of the problem. The following Theorem establishes an important link between the solutions of the differential and algebraic Riccati equations: Theorem 2: If Q = Q T, R = R T > and A, B is controllable, then the following limit exists: lim Π, T = P T 28 of 58

29 Further P = P T and satisfies the algebraic Riccati equation: A T P + P A P BR 1 B T P + Q = If A, Q is observable, then P >. Proof: The proof is broken to the following steps for clarity: 1. x T Π, T x is a monotone non-decreasing function in T. 2. x T Π, T x is bounded above for all T. 3. Π, T tends to a limit P. 4. P satisfies the algebraic Riccati equation A T P + P A P BR 1 B T P + Q =. 5. P = P T and P > if A, Q is observable. 1. x T Π, T x is a monotone non-decreasing function in T : Consider two terminal times T 1 < T 2. Denote by ūt the optimal control for the finite-horizon problem on [ T 2 ] i.e. ūt = B T R 1 Πt, T 2 xt and by ū r t its restriction in [ T 1 ] i.e. ū r t = ūt for t T 1, ū r t = for T 1 t T 2. Then: 1 2 xt Π, T 1 x Jū r t, x, T 1 LHS is optimal cost on [ T 1 ] Jūt, x, T 2 extra state-penalty cost on [T 1 T 2 ] = 1 2 xt Π, T 2 x ūt assumed optimal on [ T 2 ] Thus x T Π, T 1 x x T Π, T 2 x if T 1 < T x T Π, T x is bounded above for all T : Because of the controllability assumption we can find a finite control which drives the state to the origin in time 1, say. Clearly, for this control J = xt Qx + u T Rudt is finite. By taken the input to be zero for t 1, the state of the system stays at the origin after t = 1 since ẋ = for t 1 and hence J 1 2 xt Π, T x for every T > 1 since the LHS still represents the cost over the extended horizon - note that additional state and control cost over [1 T ] is zero - while the RHS of the inequality is the minimum cost over [, T ], and hence for any T >. Hence x T Π, T x is bounded above for all T. 3. Π, T tends to a limit P : According to a classical theorem of analysis, since x T Π, T x is monotonically non-decreasing and bounded from above, it must 29 of 58

30 u* r t u*t T 1 T 2 t Step 1: Optimal and truncated control xt x here ut= and xt= x1= t 1 T Step 2: Optimal cost is bounded converge as T. Taking x T = [ ] with the 1 in the i- th position shows that the diagonal entries of Π, T tend to a limit. Taking x T = [ ] with the 1 s in the i-th and j-th positions i j shows that 2Π, T ij = x T Π, T x Π, T ii Π, T jj also converges as T. This shows that Π, T converges and we denote the limit by P. 4. P satisfies the algebraic Riccati equation A T P +P A P BR 1 B T P +Q = : Consider 3 of 58

31 the matrix A T Π, T + Π, T A Π, T BR 1 B T Π, T + Q 5 which from part 3 must tend to the limit A T P + P A P BR 1 B T P + Q 6 as T. However equation 5 is equal to d dt t= Π, T 7 which tends to the limit 6 as T. However, since the Riccati differential equation is time invariant, 7 is equal to d dt t= T Πt, 8 where Πt,, t is the solution of the Riccati differential equation with the terminal solution shifted to time. Thus Π T, = Π, T and its derivative 8 both tend to a limit as T, and hence 8 must tend to the zero limit. Thus the matrix expression 6 is zero and P := lim T Π, T satisfies the algebraic Riccati equation A T P + P A P BR 1 B T P + Q =. 5. P = P T and P > if A, Q is observable: Since Π, T is symmetric for all T, so is its limit P = lim T Π, T. Also P is non-negative definite since x T P x x T Π, T x 9 for all x R n. If x T Π, T x = for some x, then J[u ] = see last part of Theorem 1 and hence { x T tqxt + u T tru t } dt = note that we have taken P T = S = here. Since R = R T u t = identically in the interval [ T ] and thus > this implies that x T tqxtdt = 1 Since u t = identically, the optimal state trajectory is xt = e At x, t T. Thus 1 can be written as: x T e AT t Qe At dt x = 31 of 58

32 which implies that x = if A, Q is observable. Thus in this case 9 implies that P is positive definite. We can now prove the main result of this section: Theorem 3: The minimising control for the performance index J[u] = 1 2 x T Qx + u T Rudt where Q = Q T, R = R T > subject to the dynamic equations ẋ = Ax + Bu; x = x where A, B is assumed controllable and A, Q is assumed observable is given by: ut = R 1 B T P xt := Kxt where P is the unique positive definite solution of the algebraic Riccati equation A T P + P A P BR 1 B T P + Q = The value of the performance index corresponding to the optimal control is given by: J[u ] = 1 2 xt P x Proof: We simply outline the first part of the proof which is too technical: It can be shown that under the stated assumptions there is a unique positive-definite solution P to the algebraic Riccati equation; further for this P all eigenvalues of the matrix A BR 1 B T P have negative real parts such a solution is called stabilising. It can next be shown that under the assumptions of the Theorem, the only candidate optimal controls are those for which xt ; for all other control signals the performance index does not converge i.e. is infinite. Assuming that ut is chosen so that x. Controls of this type do exist: For example, setting ut = R 1 B T P xt results in asymptotically stable closed-loop dynamics ẋ = A BR 1 B T P xt. Thus xt = exp{a BR 1 B T P t}x and hence ut = R 1 B T P exp{a BR 1 B T P t}x 32 of 58

33 which result in finite cost actually optimal since J[u] = 1 2 x T Qx + u T Rudt 1 2 = 1 { λq xt λr ut 2} 2 < where xt 2 denotes the total energy of xt in [ ], i.e. { λq xt 2 + λr ut 2} dt xt 2 2 = n i=1 x 2 i tdt which is finite for all signals of the form xt = expf tx with F an exponentially stable matrix. So next consider all control signals ut that result in state-trajectories xt x = x such that xt as t. Then, similarly to the derivation in section 2.3, J[u] = 1 2 = 1 2 = 1 2 = 1 2 xt P x xt P x x T Qx + u T Rudt [x T A T P P A + P BR 1 B T P x + u T Ru]dt [ d ] dt xt P x + u T B T P x + x T P Bu + x T P BR 1 B T P x + u T Ru dt u + R 1 B T P x T Ru + R 1 B T P xdt with equality if and only if ut = R 1 B T P xt. Note that in order to minimize technicalities in the proofs, the limiting behaviour of the finite horizon LQR problem was derived in this section under an unnecessarily strong assumption, i.e. that A, B is controllable. Moreover, it was shown that A, Q observable is sufficient for P to be positive definite which is not necessary. These may be strengthened as follows: The ARE equation has a unique, stabilising positive definite solution which is the steady-state solution of the differential Riccati equation if and only if A, B is stabilizable and A, Q is detectable. The relaxed conditions are clarified in the following section. 2.4 The algebraic Riccati equation and its solution 33 of 58

34 The solution of the infinite-horizon LQR problem reduces to the solution of the algebraic Riccati equation: A T P + P A P BR 1 B T P + Q = where R = R T > and Q = Q T. Here we will investigate all solutions of this equation. To simplify the presentation we will factor R = R 1/2 R 1/2 with R 1/2 symmetric and positive definite and will redefine BR 1/2 B. We will also factor Q = C T C which is always possible since Q is symmetric and positive-semidefinite. Thus the Riccati equation simplifies to: A T P + P A P BB T P + C T C = Apart from developing a method to generate all solutions of the algebraic Riccati equation, we will be especially interested in the stabilising solution. This is the the solution for which the matrix A P BB T is an asymptotically stable matrix all eigenvalues have negative real parts. This is important because A P BB T or A P BR 1 B T in our earlier set-up is the closed-loop A -matrix when we use the optimal state feedback law u t = BR 1 B T xt. Associated with the algebraic Riccati equation in which A is an n n matrix is the 2n 2n Hamiltonean matrix: A BB T H = C T C A T which was considered earlier in conjunction with the finite-horizon problem. We will develop solvability conditions of the ARE in relation to the properties of H. Along the way, we will derive tighter conditions for the solvability of the LQR problem from those derived so far by relaxing the assumptions on controllability and observability made earlier. It is first useful to note that the spectrum of the Hamiltonean σh is symmetric about the imaginary axis. To see this introduce the 2n 2n matrix: O In J = O I n Then it is easily verified that J 2 = I 2n and that J 1 HJ = JHJ = H T. Thus H and H T are similar and hence if λ is an eigenvalue of H, so is λ. 34 of 58

35 Definition: A subspace S C n is called invariant for a linear transformation A or for a matrix A if AS S. Examples of invariant subspaces is the linear span of the eigenvectors corresponding to n distinct eigenvalues, or the span of the generalised eigenvectors corresponding to a multiple eigenvalue, provided that all lower rank generalised eigenvectors are included. For example, suppose that λ in a multiple eigenvalue of A and let {x 1,..., x r } be the corresponding eigenvector and generalised eigenvectors: A λix 1 = A λix 2 = x 1 A λix r = x r 1 Then the subspace S spanned by the {x 1, x 2,..., x t } t r is A-invariant. Conversely, if S is non-trivial and A-invariant, then there exists a x S and λ C such that Ax = λx. An A-invariant subspace S C n is called a stable resp. anti-stable A- invariant subspace if all eigenvalues of A constrained to S have negative resp. positive real parts. The next theorem gives a method for constructing solutions to the ARE: Theorem 4: Let V C 2n be an n-dimensional invariant subspace of H, and let P 1, P 2 C n n be two complex matrices such that V = Im. P1 If P 1 is invertible, then P := P 2 P 1 1 is a solution to the ARE and σa BB T P = σh V. Further, the solution P is independent of a specific choice of bases of V. Proof: Since V is H-invariant, there exists Λ C n n such that A BB T P1 P1 = Λ C T C A T P 2 P 2 P 2 Post-multiplying by P 1 1, A BB T In I C T C A T P = P P 1 ΛP of 58

36 Pre-multiplying by [ P I n ]: = = P I A BB T C T C A T In I P A C T C P BB T A T n P = P A A T P + P BB T P C T C P which shows that P is indeed a solution of the ARE. In addition, A BB T P = P 1 ΛP 1 1 so that σa BB T P = σλ. But by definition Λ is a matrix representation of H V so that σa BB T P = σh V. Finally note that any other basis spanning V can be represented as: P1 P 2 X = P1 X for some non-singular matrix X. Since P 2 XP 1 X 1 = P 2 P 1 1 the solution is independent of the choice of basis for V. P 2 X The following Theorem shows that the converse implication of Theorem 4 also holds, i.e. that every solution of the ARE can be generated in the manner suggested by Theorem 4. Theorem 5: If P C n n is a solution of the ARE, then there are matrices P 1, P 2 C n n, with P 1 invertible, such that P = P 2 P1 1 and the columns of [P1 T P2 T ] T form a basis of an n-invariant subspace of H. Proof: Let Λ = A BB T P. Multiplying this by P gives: P Λ = P A P BB T P = C T C A T P since P solves the ARE. Write these two equations in matrix form as: A BB T I I = Λ C T C A T P P Hence, the columns of [I P T ] T span an n-dimensional invariant subspace of H. Defining P 1 = I n, P 2 = P completes the proof. 36 of 58

37 Theorems 4 and 5 above suggest the following method for finding the stabilising solution of the ARE: Find n linearly independent vectors spanning the stable invariant subspace of H and stack them in a 2n n matrix, partitioned as: P1 P 2 with P 1, P 2 square the eigenvectors/generalised eigenvectors corresponding to the stable eigenvalues of H would do. Then P = P 2 P 1 1 is the required solution. This algorithm works provided that: i H does not have eigenvalues on the imaginaryaxis for otherwise it is impossible to choose an n-dimensional stable invariant subspace of H - recall symmetry property of the spectrum of H, ii P 1 is invertible. Provided these two conditions hold Theorems 4 and 5 guarantee that a stabilizing solution exists and that it is unique. We will see that both conditions are guaranteed by relatively mild assumptions. We will also establish some additional properties of the stabilising solution symmetry, positive semi-definiteness/definiteness. Example: Consider the ARE with matrices: 2 A =, B = 1 1 and C = 1 1 The Hamiltonean matrix is: H = The eigenvalues of H are { j, j, j, j} and the corresponding eigenvectors are the columns of the eigenvector matrix: j j V = j.4.299j j j j j j j j j 37 of 58

38 1.8.6 a+jb a+jb.4 imag.2 O a jb a jb real Eigenvalues of Hamiltonean To generate all solutions of the ARE we need to generate all 2-dimensional H-invariant subspaces. Let V ij be the matrix formed by the i and j eigenvectors. There are 6 combinations: V 12, V 13, V 14, V 23, V 24 and V 34. Here are some of them: 1 {λ 1, λ 2 } = { j, j}. Here: [ ] P1 V 12 = = j.4.299j P j j P = P 2P j j 1 = [ ] which is real, symmetric and positive-definite. This is the stabilising solution. 2 {λ 1, λ 3 } = { j, j}. Here: j [ V 13 = j j j j j, P = j j j j ] 38 of 58