Lecture 11 Optimal Control

Transcription

1 Lecture 11 Optimal Control The Maximum Principle Revisited Examples Numerical methods/optimica Examples, Lab 3 Material Lecture slides, including material by J. Åkesson, Automatic Control LTH Glad & Ljung, part of Chapter 18 Giacomo Como Lecture 11, Optimal Control p. 1

2 Goal To be able to solve simple problems using the maximum principle formulate advanced problems for numerical solution Giacomo Como Lecture 11, Optimal Control p. 2

3 Outline The Maximum Principle Revisited Examples Numerical methods/optimica Example Double integrator Example Alfa Laval Plate Reactor Giacomo Como Lecture 11, Optimal Control p. 3

4 Problem Formulation (1) Standard form (1): Minimize tf ẋ(t)=f(x(t),u(t)) 0 Trajectory cost Final cost {}}{{}}{ L(x(t),u(t))dt+ φ(x(t f )) u(t) U, 0 t t f, t f given x(0)=x 0 x(t) R n,u(t) R m U control constraints Here we have a fixed end-timet f. Giacomo Como Lecture 11, Optimal Control p. 4

5 The Maximum Principle (18.2) Introduce the Hamiltonian H(x,u, λ)=l(x,u)+λ T (t)f(x,u). If problem (1) has a solution{u (t),x (t)}, then min u U H(x (t),u, λ(t))=h(x (t),u (t), λ(t)), 0 t t f, where λ(t) solves the adjoint equation dλ(t)/dt= H T x(x (t),u (t), λ(t)), with λ(t f )= φ T x(x (t f )) Giacomo Como Lecture 11, Optimal Control p. 5

6 Remarks The Maximum Principle gives necessary conditions A pair(u ( ),x ( )) is called extremal if the conditions of the Maximum Principle are satisfied. Many extremals can exist. The maximum principle gives all possible candidates. However, there might not exist a minimum! Example Minimizex(1) whenẋ(t)=u(t),x(0)=0andu(t) is free Giacomo Como Lecture 11, Optimal Control p. 6

7 Goddard s Rocket Problem revisited How to send a rocket as high up in the air as possible? d v u D h = m dt v m γu h m (v(0),h(0),m(0))=(0,0,m 0 ),,γ >0 u motor force,d=d(v,h) air resistance Constraints:0 u u max andm(t f )=m 1 (empty) Optimization criterion:max u h(t f ) Giacomo Como Lecture 11, Optimal Control p. 7

8 Problem Formulation (2) tf min t f 0 0 u:[0,t f ] U L(x(t),u(t))dt+ φ(x(t f )) ẋ(t)=f(x(t),u(t)), x(0)=x 0 ψ(x(t f ))=0 Note the differences compared to standard form: End constraints ψ(x(t f ))=0 t f free variable (i.e., not specified a priori) Giacomo Como Lecture 11, Optimal Control p. 8

9 The Maximum Principle General Case (18.4) Introduce the Hamiltonian H(x,u, λ,n 0 )=n 0 L(x,u)+λ T (t)f(x,u) If problem (2) has a solutionu (t),x (t), then there is a vector function λ(t), a numbern 0 0, and a vector µ R r such that [n 0 µ T ] =0and min u U H(x (t),u, λ(t),n 0 )=H(x (t),u (t), λ(t),n 0 ), 0 t t f, where λ(t)= H T x (x (t),u (t), λ(t),n 0 ) λ(t f )=n 0 φ T x (x (t f ))+Ψ T x (x (t f ))µ If the end timet f is free, thenh(x (t f ),u (t f ), λ(t f ),n 0 )=0. Giacomo Como Lecture 11, Optimal Control p. 9

10 Normal/abnormal cases Can scalen 0, µ, λ(t) by the same constant Can reduce to two cases n 0 =1(normal) n 0 =0(abnormal, sincel and φ don t matter) As we saw before (18.2): fixed timet f and no end constraints normal case Giacomo Como Lecture 11, Optimal Control p. 10

11 Hamilton function is constant H is constant along extremals(x,u ) Proof (in the case whenu (t) Int(U)): d dt H=H xẋ+h λ λ+h u u=h x f f T H T x +0=0 Giacomo Como Lecture 11, Optimal Control p. 11

12 Feedback or feed-forward? Example: J min =1is achieved for dx =u, x(0)=1 dt ( minimizej= x 2 +u 2) dt 0 u(t)= e t openloop (1) or u(t) = x(t) closed loop (2) (??)= stable system (??)= asympt. stable system Sensitivity for noise and disturbances differ!! Giacomo Como Lecture 11, Optimal Control p. 12

13 Reference generation using optimal control Note that the optimization problem makes no distinction between open loop controlu (t) and closed loop control u (t,x). Feedback is needed to take care of disturbances and model errors. Idea: Use the optimal open loop solutionu (t),x (t) as reference values to a linear regulator that keeps the system close to the wanted trajectory Efficient for large setpoint changes. x x Planned trajectoryx Giacomo Como Lecture 11, Optimal Control p. 13

14 Recall Linear Quadratic Control minimizex T (t f )Q N x(t f )+ where tf ẋ=ax+bu, y=cx 0 x T Q 11 Q 12 x u Q12 T Q 22 u Optimal solution ift f =,Q N =0, all matrices constant, and x measurable: u= Lx wherel=q 1 22 (Q 12+B T S) ands=s T >0solves SA+A T S+Q 11 (Q 12 +SB)Q 1 22 (Q 12+B T S)=0 Giacomo Como Lecture 11, Optimal Control p. 14

15 Second Variations ApproximatingJ(x,u) around(x,u ) to second order δ 2 J= 1 2 δxt φ xx δx+ 1 2 δẋ=f x δx+f u δu tf t 0 δx T H xx H xu δx dt δu H ux H uu δu wherej=j + δ 2 J+... is a Taylor expansion of the criterion and δ x =x x and δ u =u u. Treat this as a new optimization problem. Linear time-varying system and quadratic criterion. Gives optimal controller u u =L(t)(x x ) Giacomo Como Lecture 11, Optimal Control p. 15

16 Opt. Ref. Gen. x u Lin. Cont. + u Proc. y ˆx Obs. Take care of deviations with linear controller Giacomo Como Lecture 11, Optimal Control p. 16

17 Outline The Maximum Principle Revisited Examples Numerical methods/optimica Example Double integrator Example Alfa Laval Plate Reactor Giacomo Como Lecture 11, Optimal Control p. 17

18 Example: Optimal heating Minimize tf =1 0 P(t) dt when Ṫ=P T 0 P P max T(0)=0, T(1)=1 T temperature P heat effect Giacomo Como Lecture 11, Optimal Control p. 18

19 Solution Hamiltonian H=n 0 P+λP λt Adjoint equation λ T = H T = H T = λ λ(1)=µ At optimality λ(t)=µe t 1 H=(n 0 + µe t 1 ) P λt }{{} σ(t) P (t)= { 0, σ(t)>0 P max, σ(t)<0 Giacomo Como Lecture 11, Optimal Control p. 19

20 Solution µ=0 (n 0, µ)=(0,0) Not allowed! µ =0 ConstantPor just one switch! T(t) approaches one from below, sop =0neart=1. Hence { 0, P 0 t t1 (t)= P max, t 1 <t 1 { 0, 0 t t1 T(t)= 1 t 1 e (t τ) P max dτ= ( e (t 1) e (t t 1) ) P max, t 1 <t 1 Timet 1 is given byt(1)= ( 1 e (1 t 1) ) P max =1 Has solution0 t 1 1ifP max 1 1 e 1 Giacomo Como Lecture 11, Optimal Control p. 20

21 Example The Milk Race Move milk in minimum time without spilling! [M. Grundelius Methods for Control of Liquid Slosh] [movie] Giacomo Como Lecture 11, Optimal Control p. 21

22 Minimal Time Problem NOTE! Common trick to rewrite criterion into standard form"!! minimize t f =minimize tf 0 1dt Control constraints No spilling u(t) u max i Cx(t) h Optimal controller has been found for the milk race Minimal time problem for linear systemẋ=ax+bu,y=cx with control constraints u i (t) u max i. Often bang-bang control as solution Giacomo Como Lecture 11, Optimal Control p. 22

23 Results- milk race Maximum slosh φ max =0.63 Maximum acceleration = 10 m/s 2 Time optimal acceleration profile 15 Acceleration Slosh Optimal time = 375 ms, industrial = 540ms Giacomo Como Lecture 11, Optimal Control p. 23

24 Outline The Maximum Principle Revisited Examples Numerical methods/optimica Example Double integrator Example Alfa Laval Plate Reactor Giacomo Como Lecture 11, Optimal Control p. 24

25 Numerical Methods for Dynamic Optimization Many algorithms Applicability highly model-dependent (ODE, DAE, PDE, hybrid?) Calculus of variations Single/Multiple Shooting Simultaneous methods Simulation-based methods Analogy with different simulation algorithms (but larger diversity) Heavy programming burden to use numerical algorithms Fortran C Engineering need for high-level descriptions Giacomo Como Lecture 11, Optimal Control p. 25

26 Modelica A Modeling Language Modelica is increasingly used in industry Expert knowledge Capital investments Usage so far Simulation (mainly) Other usages emerge Sensitivity analysis Optimization Model reduction System identification Control design Giacomo Como Lecture 11, Optimal Control p. 26

27 Optimica and JModelica A Research Project Shift focus: from encoding to problem formulation Enable dynamic optimization of Modelica models State of the art numerical algorithms Develop a high level description for optimization problems Extension of the Modelica language Develop prototype tools JModelica and The Optimica Compiler Code generation Giacomo Como Lecture 11, Optimal Control p. 27

28 Outline The Maximum Principle Revisited Examples Numerical methods/optimica Example Double integrator Example Alfa Laval Plate Reactor Giacomo Como Lecture 11, Optimal Control p. 28

29 Optimica An Example tf min 1dt u(t) 0 subject to the dynamic constraint ẋ(t)=v(t), x(0)=0 v(t)=u(t), v(0)=0 and x(t f )=1 v(t f )=0 v(t) u(t) 1 Giacomo Como Lecture 11, Optimal Control p. 29

30 A Modelica Model for a Double Integrator A double integrator model model DoubleIntegrator Real x(start=0); Real v(start=0); input Real u; equation der(x)=v; der(v)=u; end DoubleIntegrator; Giacomo Como Lecture 11, Optimal Control p. 30

31 The Optimica Description Minimum time optimization problem optimization DIMinTime (objective=cost(finaltime), starttime=0, finaltime(free=true,initialguess=1)) Real cost; DoubleIntegrator di(u(free=true,initialguess=0.0)); equation der(cost) = 1; constraint finaltime>=0.5; finaltime<=10; di.x(finaltime)=1; di.v(finaltime)=0; di.v<=0.5; di.u>= 1; di.u<=1; end DIMinTime; Giacomo Como Lecture 11, Optimal Control p. 31

32 Optimal Double Integrator Profiles u x v t [s] t [s] Giacomo Como Lecture 11, Optimal Control p. 32

33 Outline The Maximum Principle Revisited Examples Numerical methods/optimica Example Double integrator Example Alfa Laval Plate Reactor Giacomo Como Lecture 11, Optimal Control p. 33

34 Optimal Start-up of a Plate Reactor Reactant B Reactant A HEX T f FC FC q B1 q B2 T T T T T T T T T T Reactor outlet T c HEX Cooling water Achieve safe start-upt r T max Maximize the conversion Minimize the start-up time Giacomo Como Lecture 11, Optimal Control p. 34

35 The Optimization Problem Reduce sensitivity of the nominal start-up trajectory by: - Introducing a constraint on the accumulated concentration of reactant B - Introducing high frequency penalties on the control inputs min u tf 0 α A c 2 A,out + α Bc 2 B,out + α B1q 2 B1,f + α B2q 2 B2,f + α T1 Ṫf 2 + α T 2 Ṫc 2 dt subjectto ẋ=f(x,u) T r,i 155, i=1..n c B,1 600, c B, q B1 0.7, 0 q B Ṫf 2, 1.5 Ṫc T f 80, 20 T c 80 Giacomo Como Lecture 11, Optimal Control p. 35

36 The Optimization Problem Optimica Robust optimization formulation optimization PlateReactorOptimization (objective=cost(finaltime), starttime=0, finaltime=150) PlateReactor pr(u_t_cool_setpoint(free=true), u_tfeeda_setpoint(free=true), u_b1_setpoint(free=true), u_b2_setpoint(free=true)); parameter Real sc_u = 670/50 "Scaling factor"; parameter Real sc_c = 2392/50 "Scaling factor"; Real cost(start=0); equation der(cost) = 0.1*pr.cA[30]^2*sc_c^ *pr.cB[30]^2*sc_c^2 + 1*pr.u_B1_setpoint_f^2 + 1*pr.u_B2_setpoint_f^2 + 1*der(pr.u_T_cool_setpoint)^2*sc_u^2 + 1*der(pr.u_TfeedA_setpoint)^2*sc_u^2; constraint pr.tr/u_sc<=( )*ones(30); pr.cb[1]<=200/sc_c; pr.cb[16]<=400/sc_c; pr.u_b1_setpoint>=0; pr.u_b1_setpoint<=0.7; pr.u_b2_setpoint>=0; pr.u_b2_setpoint<=0.7; pr.u_t_cool_setpoint>=(15+273)/sc_u; pr.u_t_cool_setpoint<=(80+273)/sc_u; pr.u_tfeeda_setpoint>=(30+273)/sc_u; pr.u_tfeeda_setpoint<=(80+273)/sc_u; der(pr.u_t_cool_setpoint)>= 1.5/sc_u; der(pr.u_t_cool_setpoint)<=0.7/sc_u; der(pr.u_tfeeda_setpoint)>= 1.5/sc_u; der(pr.u_tfeeda_setpoint)<=2/sc_u; end PlateReactorOptimization; Giacomo Como Lecture 11, Optimal Control p. 36

37 1500 c B,1 [mol/m 3 ] c B,2 [mol/m 3 ] T 1 [ C] T 2 [ C] q B1 [-] q B2 [-] T f [ C] T c [ C] Time [s] Time [s] Time [s] Almost as fast, but more robust with lowerc B -constraints Time [s] Giacomo Como Lecture 11, Optimal Control p. 37

38 Summary The Maximum Principle Revisited Examples Numerical methods/optimica Example Double integrator Example Alfa Laval Plate Reactor Giacomo Como Lecture 11, Optimal Control p. 38