ECSE.6440 MIDTERM EXAM Solution Optimal Control. Assigned: February 26, 2004 Due: 12:00 pm, March 4, 2004

Size: px

Start display at page:

Download "ECSE.6440 MIDTERM EXAM Solution Optimal Control. Assigned: February 26, 2004 Due: 12:00 pm, March 4, 2004"

Alban Logan
5 years ago
Views:

1 ECSE.6440 MIDTERM EXAM Solution Optimal Control Assigned: February 26, 2004 Due: 12:00 pm, March 4, 2004 This is a take home exam. It is essential to SHOW ALL STEPS IN YOUR WORK. In NO circumstance is COLLABORATION allowed. There are FOUR problems with a total of 100 points. Problem Points Score Total 100 1

2 1. (25 points) (a) (10 points) Solve the following constrained optimization problem both analytically and graphically: for max x,y h(x, y) subject to x2 (2 y) 3, y 0 (a) : h(x, y) = x + y (b) : h(x, y) = x + 3y. i. Apply the Lagrange Multiplier method to find the solutions to (a) and (b) ii. On the x-y plane, show the feasible region and h(x, y) = c for several costants c. Show that the optimal solution may be found by using this graphical method and the solution coincides with your analytical solution. (b) (15 points) Let x k, u k, d k be the inventory, production, and demand at time k. Let x 0 be the initial inventory, and x and u be the goal level inventory and production. The inventory dynamics is x k+1 = x k + u k d k. Let h and c be inventory and production costs. The total cost over N-steps (N is fixed) is min u k,k=0,...,n 1 { J = 1 2 N 1 k=0 Find the optimal production schedule. ( h(xk x) 2 + c(u k u) 2)}. 2

3 Solution: 1(a)1.a: min x,y (x + y) subject to x 2 (2 y) 3 0, y 0. First form the Hamiltonian: First order condition: The Hessian is H(x, y) = (x + y) + λ 1 (x 2 (2 y) 3 ) λ 2 y. H x = 1 + 2λ 1x = 0 (1) H y = 1 + 3λ 1 (2 y) 2 λ 2 = 0. (2) 2 [ H x = 2λ λ 1 (2 y) The constraint conditions can be stated as ]. 1. λ 1 (x 2 (2 y) 3 ) = 0 (3) 2. λ 2 y = 0. (4) Now we examine all possible cases (if neither, either or both of the constraints are active). (a) Both constraints inactive (λ 1 = λ 2 = 0). This violates the first order condition (1). (b) Constraint 1 active (x 2 = (2 y) 3 ), constraint 2 inactive (λ 2 = 0, y > 0). Solve jointly, we obtain two solutions: x 2 = (2 y) 3, 1 = 2λ 1 x, 1 = 3λ 1 (2 y) 2, solution 1: x = (2/3) 3 =.2963, y = , andj = x + y = solution 2: x = 0, y = 2, andj = x + y = 2. (c) Constraint 1 inactive, constraint 2 active. This also leads to contradiction (1 = 0). (d) Both constraints active. In this case, we solve x 2 = (2 y) 3, y = 0, 1 = 2λ 1 x, 1 + λ 2 = 3λ 1 (2 y) 2, joint, and the solution is The corresponding cost is x = 2 2, y = 0. J = x + y = 2 2 = Clearly, this is the optimal solution. definite. Note that λ 1 is +. As a check, the Hessian is also positive 3

4 1(a)1.b: min x,y (x + 3y) subject to x 2 (2 y) 3 0, y 0. First form the Hamiltonian: H(x, y) = (x + 3y) + λ 1 (x 2 (2 y) 3 ) λ 2 y. First order condition: H x = 1 + 2λ 1x = 0 (5) H y = 3 + 3λ 1 (2 y) 2 λ 2 = 0. (6) Go through the 4 cases as before, we can immediately determine that the first constraint has to be active (i.e., x 2 = (2 y) 3 ). If constraint 1 is active and constraint is active, there are two solutions: solution 1: x = 2 2 = , y = 0, andj = x + 3y = solution 2: x = 0, y = 2, andj = x + 3y = 6. If both constraints are active, the solution is x = 2 2 = , y = 0, andj = x + 3y = Therefore, the optimal solution is (x = 0, y = 2). 1(a)2.a: Solution script is in midsol1.m. The graphical solution is the same as the analytical solution: 1(a)2.b: Solution script is in midsol1.m. The graphical solution is the same as the analytical solution: 4

5 1(b): Given x k+1 = x k + u k d k, and x 0. Find {u k : k = 0, 1,..., N 1} to minimize Form the Hamiltonian: J = 1 2 N 1 k=0 (h(x k x) 2 + c(u k u) 2 ). H k = 1 2 h(x k x) c(u k u) 2 + λ k+1 (x k + u k d k ). First solve for u k from H k u k = 0: Then solve for the co-state propagation: u k = u λ k+1 c. λ k = H k x k = h(x k x) + λ k+1. Putting the state and co-state together and substitute for u k, we get [ xk+1 λ k+1 Solving for λ N, we get ] [ 1 + h 1 ] = c c h 1 }{{} A [ xk λ k ] [ h + x + u d ] c k. hx }{{} w k λ N = [ 0 1 ] A N [ x0 λ 0 ] + [ 1 0 ] N 1 k=0 A N k 1 w k. Since there is no terminal cost, λ N = 0. Therefore, [ ] [ ] 0 1 A N 0 λ 1 0 = [ 0 1 ] [ 1 A N 0 }{{} Γ 5 ] x 0 [ 0 1 ] N 1 k=0 A N k 1 w k.

6 λ 0 can then be readily solved: λ 0 = Γ 1 ( [ 0 1 ] A N [ 1 0 ] x 0 [ 0 1 ] N 1 k=0 A N k 1 w k ). Once we have λ 0, λ k can be found from the co-state equation, and u k can in turn be solved. 6

7 2. (25 points) (a) (10 points) Consider a continuous time linear time invariant system Suppose that the performance index is J = ẋ = Ax + Bu, x R n, u R m. (x(t) T Qx(t) + 2x(t) T Nu(t) + u T (t)ru(t)), where Q R n n is symmetric positive semi-definitie, R R m m is symmetric positive definite, and N R n m. Find the optimal controller as a constant gain full state feedback. (b) (15%) Consider a continuous time linear time varying system ẋ = A(t)x + B(t)u. Suppose that the goal is to track a specified desired state and control trajectories (x d (t), u d (t)) for 0 t T, T is fixed. Let the optimization index be J = 1 2 (x(t ) x d(t )) T Q(T )(x(t ) x d (T )) + 1 T ( (x(t) xd (t)) T Q(t)(x(t) x d (t)) + (u(t) u d (t)) T R(t)(u(t) u d (t)) ) dt. 2 0 Find the necessary condition for optimality. Apply the sweep method to solve the resulting two-point boundary value problem. 7

8 Solution: (a) First form the Hamiltonian H = 1 2 xt Qx + x T Nu ut Ru + λ T (Ax + Bu). Solve for optimal control from H u = 0: Co-state equation: λ = u = R 1 (B T λ + N T x). ( ) T H = (Qx + Nu + A T λ) = (Qx NR 1 (B T λ + N T x) + A T λ). x State equation: Let λ = P x. Then ẋ = Ax BR 1 (B T λ + N T x). λ = P ẋ = P Ax P BR 1 (B T λ + N T x) = (Qx NR 1 (B T λ + N T x) + A T λ). Rearrange terms, we get P (A BR 1 N T ) + (A BR 1 N T ) T P + (Q NR 1 N T ) P BR 1 B T P = 0. or A T P + P A + Q (P B + N)R 1 (P B + N) T. (b) Let Then x(t) = x(t) x d (t), ũ(t) = u(t) u d (t). x = A x + Bu ẋ d = A x + Bũ + Ax d + Bu d ẋ }{{ d. } w The optimization index becomes J = 1 2 xt (T )Q(T ) x(t ) Form the Hamiltonian: T 0 ( x T (t)q(t) x(t) + ũ T (t)r(t)ũ(t) ) dt. H = 1 2 ( xt Q x + ũ T Rũ) + λ T (A x + Bũ + w). Optimal control: ũ = R 1 B T λ. 8

9 Co-state equation: State equation: λ = Q x A T λ. x = A x BR 1 B T λ + w. Let Then λ = P x + β. λ = P x + P x + β. From the state and co-state equations, we get Boundary condition: P + P A + A T P + Q P BR 1 B T P = 0 β + (A BR 1 B T P ) T β + P w = 0. P (T ) = Q(T ), β(t ) = Q(T )x d. 9

10 3. (25 points) Consider the harmonic oscillator ẍ + ω 2 ox = ω 2 ou. (a) (5 points) Show through time scaling, the problem can be equivalently transformed to the normalized problem z + z = u. (b) (20 points) For the normalized problem, solve the optimal control problem with the optimization index J = T 0 (1 + u(t) 2 ) dt, with (x(0), ẋ(0)) = (0, 0), (x(t ), ẋ(t )) = (2, 0) and T free. Sketch the solution trajectories in the phase space (i.e., ẋ vs. x). 10

11 Solution: (a) Given Define Then Let ẍ + ω 2 0x = ω 2 0u. z(t) = x(at). dz(t) = a dx(τ) dt dτ τ=at d 2 z(t) = a 2 d2 x(τ) dt 2 dτ 2 = a 2 ω 0x(at) 2 + a 2 ω0u(at). 2 τ=at a = 1 ω. Then d 2 z(t) = z(t) + u(t). dt 2 To recover x, just apply the inverse transformation: (b) Let z 1 = z, z 2 = ż. Hamiltonian: Optimal control: x(t) = z(ωt). H = 1 + u 2 + λ 1 z 2 + λ 2 ( z 1 + u). H u = 0 = u = λ 2 2. Co-state equation, λ = H x λ1 = λ2 State equation: λ 2 = λ 1. ż 1 = z 2 ż 2 = z 1 λ 2 2. Since the final time is free, we need the transversality condition: H(t) = 0 t [0, T ]. 11

12 Choose t = 0 to simplify the expression: H(0) = 0 = (1 + u 2 + λ 2 u) = 1 λ2 2(0) = 0 t=0 4 or λ 2 (0) = ±2. From the co-state equation, λ 2 (t) = a sin t + b cos t. Using the transversality condition, we get The optimal control is therefore λ 2 (t) = a sin t ± 2 cos t. u(t) = λ 2(t) = a sin t cos t. 2 2 By using the state equation, the final state (z 1 (T ), z 2 (T )) can be expressed in terms of (a, T ). Since the final state is given to be (2, 0), we can use the two equations to solve for the two unknowns. The MATLAB solution script may be found in midsol3.m or midsol3a.m. It turns out that only λ 2 (0) = 2 has a solution (with T = 2.508sec). Usage: [a,t,xterror,t,y,u]=midsol3;xterror or midsol3a The optimal control, optimal z, and z 1 vs. z 2 are shown below 12

13 4. (25 points) A factory has received an order to produce B units of merchandise in time T. Assume that the unit cost of production is proportional to the rate of production and the unit cost of holding the inventory is constant. The goal is to find a production schedule that minimizes cost. This may be posed by a calculus of variation problem. Let x(t) be the # of units at time t. Then the total cost is J = T 0 (c 1 ẋ(t) 2 + c 2 x(t)) dt, where the first term is the cost of production (cost for producing dx units is c 1 ẋ) and the second term is the inventory cost. The boundary conditions are x(0) = 0 and x(t ) = B. Find the optimal production schedule, i.e., {x(t) : t [0, T ]} that minimizes J. 13

14 Solution: Optimization index: State equation: Boundary conditions: Hamiltonian: Optimal control: Co-state equation: Therefore, J = Substituting into the state equation: T 0 (c 1 u 2 + c 2 x) dt. ẋ = u x(0) = 0, x(t ) = B. H = c 1 u 2 + c 2 x + λu. u = λ 2c 1. λ = H x = c 2. λ = c 2 t + λ 0. ẋ = c 2 2c 1 t λ 0 2c 1. Solution: Set t = T, we get Solve for λ 0 : x = c 2 4c 1 t 2 λ 0 2c 1 t. B = c 2 4c 1 T 2 λ 0 2c 1 T. λ 0 = 2c 1 T ( c 2 4c 1 T 2 B). 14

Homework Solution # 3

ECSE 644 Optimal Control Feb, 4 Due: Feb 17, 4 (Tuesday) Homework Solution # 3 1 (5%) Consider the discrete nonlinear control system in Homework # For the optimal control and trajectory that you have found