Zur Konstruktion robust stabilisierender Regler mit Hilfe von mengenorientierten numerischen Verfahren

Transcription

1 Zur Konstruktion robust stabilisierender Regler mit Hilfe von mengenorientierten numerischen Verfahren Oliver Junge Zentrum Mathematik Technische Universität München mit Lars Grüne, Bayreuth

2 Outline optimal stabilization problem set oriented discretization directed graph shortest path algorithm optimal value function optimality principle optimal feedback optimality residual adaptive discretization robustness: game theoretic viewpoint directed hypergraph, adapted shortest path algorithm Oliver Junge Konstruktion robuster Regler p.2

3 A stabilization problem Discrete-time control system cost function g(x, u) 0. Optimal stabilization f (0, 0) = 0, unstable. x k+1 = f (x k, u k ), k = 0, 1,..., Goal: starting at x 0, impose u 0, u 1,... such that x k 0, while minimizing J(x 0, (u k ) k ) = g(x k, u k ). k=0 Oliver Junge Konstruktion robuster Regler p.3

4 The optimal value function Admissable control sequences: U(x) = {u U N : x k (x, u) 0 as k }. (may be empty). Stabilizable set: S = {x X U(x) }. The optimal value function V : S [0, ]: V (x) = inf{j(x, u) u U(x)} Goal Compute S and V, together with (approximate) optimizing sequences u = u(x). Oliver Junge Konstruktion robuster Regler p.4

5 A simple example X = {0, x 1, x 2 } x 1 0 x 2 Oliver Junge Konstruktion robuster Regler p.5

6 A simple example X = {0, x 1, x 2 } x 1 f(x 1, ) = 0 0 f(x 2, ) = 0 x 2 Oliver Junge Konstruktion robuster Regler p.6

7 A simple example X = {0, x 1, x 2 } x x 2 Oliver Junge Konstruktion robuster Regler p.7

8 A simple example X = {0, x 1, x 2 } x 1 f(x 1, u 1 ) = 0 f(x 1, u 2 ) = x 2 0 x 2 Oliver Junge Konstruktion robuster Regler p.8

9 A simple example X = {0, x 1, x 2 } x x 2 3 Oliver Junge Konstruktion robuster Regler p.9

10 A simple example X = {0, x 1, x 2 } x x 2 Oliver Junge Konstruktion robuster Regler p.10

11 A simple example X = {0, x 1, x 2 } x x 2 Oliver Junge Konstruktion robuster Regler p.11

12 Graph formulation Corresponding directed graph: G = (X, E), E = {e = (x, f (x, u)) x X, u U}, with weights w(e) = g(x, u). Path p = (e 1, e 2,...) in G with length w(p) = w(e k ). k=0 Optimal value function: V (x) = inf{w(p) p connects x to 0 }. Oliver Junge Konstruktion robuster Regler p.12

13 Discretization of phase space Let P be a finite partition of X. Finite graph: f(a, U) A A Weights: w(a, B) = min {g(x, u) f (x, u) B}. x A,u U Oliver Junge Konstruktion robuster Regler p.13

14 Computing the optimal value function Approximate optimal value function: V P (x) = min p {w(p) p connects x to 0} Constructing G P : GAIO ( Global Analysis of Invariant Objects ); Root Computing V P : Dijkstra s algorithm. Oliver Junge Konstruktion robuster Regler p.14

15 Dijkstra s algorithm Dijkstra((P, E), g, D) 1 for each P P set V (P) := 2 V (D) := 0 3 Q := P 4 while Q = 5 P := argmin P QV (P ) 6 Q := Q\{P} 7 for each Q P with (Q, P) E 8 if V (Q) > w(q, P) + V (P) then 9 V (Q) := w(q, P) + V (P) Oliver Junge Konstruktion robuster Regler p.15

16 Convergence Proposition For every collection P, V P V. (P (l) ) l N : nested collections with diam(p (l) ) 0 as l. Theorem (J., Osinga, 04) The approximating optimal value functions V P (l) converge pointwise to V as l. Oliver Junge Konstruktion robuster Regler p.16

17 Example: stabilization of an inverted pendulum m φ u M Oliver Junge Konstruktion robuster Regler p.17

18 Inverted pendulum: optimal value function Oliver Junge Konstruktion robuster Regler p.18

19 Example: double inverted pendulum Oliver Junge Konstruktion robuster Regler p.19

20 Double inverted pendulum: optimal pseudo-trajectory d/dt 1 2 d/dt 2 u 0 states time Oliver Junge Konstruktion robuster Regler p.20

21 Optimal feedback Optimality principle V (x) = inf {g(x, u) + V (f (x, u))} u U Feedback u P (x) = argmin u U {g(x, u) + V P (f (x, u))} Oliver Junge Konstruktion robuster Regler p.21

22 Example: A hybrid system Switched voltage controller: u { 1, 1} control input. ẋ 1 = 1 C (x 2 I load ) ẋ 2 = 1 L x 1 R L x L uv in, (1) Oliver Junge Konstruktion robuster Regler p.22

23 Switched voltage controller: stabilization 0.6 Voltage Current Switch sign Sample Oliver Junge Konstruktion robuster Regler p.23

24 Performance Theorem [Grüne, J., 05] Let V be continuous on D S, 0 int D and D c = V 1 P(1) ([0, c]) D for c > 0. Then for all sufficiently small ε > 0, l l 0 (ε), x 0 D c and all η (0, 1): k 1 V (x k ) V (x 0 ) (1 η) g(x j, u P(l) (x j )), as long as V (x k ) > δ(ε/η) + ε with δ(α) 0 as α 0. j=0 Oliver Junge Konstruktion robuster Regler p.24

25 Error estimation Goal: construct adaptive partition. For x S 0 = {x X : V (x) < } consider the residual e(x) := min u U {g(x, u) + V P(f (x, u))} V P (x) 0. e(x) V (x) V P (x), x S 0. Oliver Junge Konstruktion robuster Regler p.25

26 Feedback performance Theorem [Grüne, J., 05] Consider a partition P and D c = V 1 P ([0, c]) for some c > 0. Assume that the error function e satisfies e(x) max{η min g(x, u), ε} u U for all x D c, some ε > 0 and some η (0, 1). Then the feeback-trajectory for each x 0 D c satisfies k 1 V P (x k ) V P (x 0 ) (1 η) g(x j, u P (x j )), as long as V P (x k ) > δ(ε/η) + ε with δ(α) 0 as α 0. j=0 Oliver Junge Konstruktion robuster Regler p.26

27 Inverted pendulum: value function (adaptive partition) Oliver Junge Konstruktion robuster Regler p.27

28 Inverted pendulum: value function (adaptive partition) Oliver Junge Konstruktion robuster Regler p.28

29 Inverted pendulum: residual (adaptive partition) Oliver Junge Konstruktion robuster Regler p.29

30 Modelling perturbations Perturbed Control System x k+1 = f (x k, u k, w k ), k = 0, 1,..., Game Theoretic Viewpoint 1. control u k, tries to minimize accumulated cost J, 2. perturbation w k, wants to maximize J. Goal construct feedback law u(x) that stabilizes the system to a neighborhood O of the origin, regardless of how the perturbation acts. Oliver Junge Konstruktion robuster Regler p.30

31 Feedback construction Upper value function V (x) = inf u:x U sup J(x, u( ), w) w W N Optimality principle { } V (x) = inf g(x, u) + sup V (f (x, u, w)) u U w W and, as before, feedback } u(x) = argmin u U {g(x, u) + sup V (f (x, u, w)) w W Oliver Junge Konstruktion robuster Regler p.31

32 Discretization: theoretical framework Multivalued game F : X U W X, F (x, u, w), compact, cost function G : X X U W [0, ). Trajectory: any sequence x = (x k ) k such that x 0 = x and x k+1 F (x k, u k, w k ), k = 0, 1, 2,.... Set of all trajectories associated to x, u and w: X F (x, u, w) Oliver Junge Konstruktion robuster Regler p.32

33 Theoretical framework Accumulated cost J (F,G) (x, u, w) = Upper value function Optimality Principle inf (x k ) k X F (x,u,w) G(x k, x k+1, u k, w k ). k=0 V (F,G) (x) = inf sup J (F,G) (x, u( ), w) u( ) w W N V (F,G) (x) = inf u U sup inf w W x 1 F (x,u,w) { G(x, x1, u, w) + V (F,G) (x 1 ) } Oliver Junge Konstruktion robuster Regler p.33

34 Discretization... for the unperturbed case: Dynamic programming operator { } L[v](x) = inf g(x, u) + inf v(x 1) u U x 1 F (x,u) Optimality principle i.e. V (F,G) is a fixed point of L. V (F,G) = L[V (F,G) ], Oliver Junge Konstruktion robuster Regler p.34

35 Discretization Unperturbed case multivalued projection ρ : X X, ρ(x) = P with x P. Multivalued game F (x, u, w) = ρ(f (x, u)), G(x, x 1, u, w) = g(x, u) Projection onto piecewise constant functions: ϕ[v](x) = inf v(x ), x ρ(x) Discretized dynamic programming operator L P = ϕ L. Oliver Junge Konstruktion robuster Regler p.35

36 Discretization Unperturbed case Explicitely, for v piecewise constant: { { }} L P [v](x) = inf inf x ρ(x) u U g(x, u) + inf 1) x 1 F (x,u) Oliver Junge Konstruktion robuster Regler p.36

37 Discretization Unperturbed case Explicitely, for v piecewise constant: { { }} L P [v](x) = inf inf x ρ(x) u U g(x, u) + inf 1) x 1 F (x,u) = inf x ρ(x),u U {g(x, u) + v(f (x, u))}, Oliver Junge Konstruktion robuster Regler p.37

38 Discretization Unperturbed case Explicitely, for v piecewise constant: { { }} L P [v](x) = inf inf x ρ(x) u U g(x, u) + inf 1) x 1 F (x,u) = inf x ρ(x),u U {g(x, u) + v(f (x, u))}, = min P inf {g(x, u) + v(p)} x,u:f (x,u) P Oliver Junge Konstruktion robuster Regler p.38

39 Discretization Unperturbed case Explicitely, for v piecewise constant: { { }} L P [v](x) = inf inf x ρ(x) u U g(x, u) + inf 1) x 1 F (x,u) = inf x ρ(x),u U {g(x, u) + v(f (x, u))}, = min P inf {g(x, u) + v(p)} x,u:f (x,u) P = min P 1 F (P) {w(p, P 1) + v(p 1 )} Oliver Junge Konstruktion robuster Regler p.39

40 Discretization Unperturbed case Explicitely, for v piecewise constant: { { }} L P [v](x) = inf inf x ρ(x) u U g(x, u) + inf 1) x 1 F (x,u) = inf x ρ(x),u U {g(x, u) + v(f (x, u))}, = min P inf {g(x, u) + v(p)} x,u:f (x,u) P = min P 1 F (P) {w(p, P 1) + v(p 1 )} Dijkstra s algorithm! Oliver Junge Konstruktion robuster Regler p.40

41 Discretization Perturbed case Multivalued game F (x, u, w) = ρ(f (x, u, w)) and G(x, x 1, u, w) = g(x, u), optimality principle, projection,..., yields { } V P (P) = inf N F(P) w(p, N ) + sup V P (N) N N. Oliver Junge Konstruktion robuster Regler p.41

42 Data structure: Directed Weighted Hypergraph π(f(x, u, W )) F (x, u, W ) f(x, u, W ) N 1 P N 2 Oliver Junge Konstruktion robuster Regler p.42

43 Dijkstra s Method for the Perturbed System 7 for each (Q, N ) E with P N 8 if V (Q) > G(Q, N ) + max N N V (N) then 9 V (Q) := G(Q, N ) + max N N V (N) During the while-loop in lines 4-9 it holds that V (P) V (P ) for all P P\Q. 7 for each (Q, N ) E with P N 8 if N P\Q then 9 if V (Q) > G(Q, N ) + V (P) then 10 V (Q) := G(Q, N ) + V (P) Oliver Junge Konstruktion robuster Regler p.43

44 Convergence Theorem (Grüne, J., 05) g continuous, g(x, u) > 0 for x O and V (f,g) continuous on O, then V Pi V (f,g) uniformly on every compact set on which V (f,g) is continuous and which is the union of partition elements. If the set of discontinuities of V (f,g) has zero Lebesgue measure, then V Pi V (f,g) in L 1 on every compact subset of the domain of V (f,g). The corresponding feedback is stabilizing for all perturbation sequences. Oliver Junge Konstruktion robuster Regler p.44

45 Conclusion optimal stabilization problem set oriented discretization directed graph, shortest path algorithm yields the solution robustness: game theoretic viewpoint directed hypergraph, adapted shortest path algorithm method is naturally suited for problems with discontinuous value function (resulting from, e.g., state space constraints) discrete state or control spaces (hybrid systems) Oliver Junge Konstruktion robuster Regler p.45

46 Outlook Towards high-dimensional systems: relaxed dynamic programming (Rantzer et al.) neuro-dynamic programming (a la reinforcement learning) model reduction techniques (idea: truncation as a perturbation of the reduced system) Oliver Junge Konstruktion robuster Regler p.46