Feedback Refinement Relations for the Synthesis of Symbolic Controllers

Transcription

1 Feedback Refinement Relations for the Synthesis of Symbolic Controllers Gunther Reissig 1, Alexander Weber 1 and Matthias Rungger 2 1: Chair of Control Engineering Universität der Bundeswehr, München 2: Hybrid Control Systems Group Technische Universität München (TUM)

2 2/33 Controller Synthesis A conceptual synthesis problem Given a system S and a desired behavior Σ, find a controller, i.e., a system C so that the behavior of the closed loop satisfies B(C S) Σ. Σ, B(C S) are subsets of (U Y ) := (U Y ) (U Y ) ω U are the inputs Y are the outputs Examples Reference tracking Optimal control (u, y) Σ Linear temporal logic specifications lim y(t) y reference (t) = 0 t (u, y) Σ (u, y) inf J(u, y) ε (u, y) Σ (u, y) satisfies ϕ

3 3/33 Abstraction and Refinement for Controller Synthesis Usually, direct synthesis of C from (S, Σ) is impossible and a finite substitute (Ŝ, ˆΣ) is used to determine C: 1. Compute a finite substitute (Ŝ, ˆΣ) of (S, Σ) Ŝ is an abstraction or a symbolic model of S 2. Synthesize controller Ĉ to enforce ˆΣ on Ŝ 3. Refine solution Ĉ to C finite (Ŝ, ˆΣ) 2. Ĉ 1. abstract concrete 3. (S, Σ) C infinite

4 4/33 Correctness Reasoning Given that Ĉ enforces ˆΣ on Ŝ. How to ensure that C enforces Σ an S? For verification simulation relations are used: S S Ŝ and B(Ŝ) ˆΣ = B(S) Σ Recall: Q X ˆX is a simulation relation from S to Ŝ if (x, ˆx) Q implies x x ˆx ˆx (x, ˆx ) Q For synthesis alternating simulation relations are used: Ŝ AS S and B(Ĉ Ŝ) ˆΣ = C : B(C S) Σ Recall: Q X ˆX is an alternating simulation relation from Ŝ to S if (x, ˆx) Q implies û admissible u admissible u x x û ˆx ˆx (x, ˆx ) Q

5 The Refinement Procedure (= Controller Transfer) ASR conditions 1. (x, ˆx) Q implies 2. û admissible 3. u admissible u 4. x x û 5. ˆx ˆx 6. (x, ˆx ) Q Construction of refined controller C 1. Given x, Q acts as quantizer and picks ˆx Q(x) 2. In the abstract closed loop and admissible û is proposed 3. In the concrete closed loop û is matched by an admissible u 4. The concrete closed loop proceeds with x u x Reissig, Rungger, and Weber 5. The next Feedback ˆx is Refinement determined Relations for bythe ˆx Synthesis Q(x of Symbolic ) and ˆx Controllers ˆx The closed loop C S follows to û input u x system x state input u x ˆx û ˆx input converter quantizer abstraction û abstract controller ˆx refined controller 5/33

6 A Closer Look at the Refinement Reissig, Rungger, and Weber The closed loop C S follows to Feedback Refinement Relations for the Synthesis of Symbolic Controllers input system state input input converter quantizer abstraction abstract controller refined controller Properties: Abstraction is contained in the refined controller The refined controller requires exact state information Both properties are critical! (a) Figure 1. (a) Actual closed loop resulting from the use of approximate alternating simulation relations [?], [?]. (b) of feedback refinement relations, as proposed in this paper. As our examples in Section III show, the structure of thi with the use of approximate alternating (bi-)simulation relations or approximate (bi-)simulation relations. Typically abstractions. abstractions Indeed, consists as demonstrated of 10 6 states in Section and 10 III, 9 transitions the issue ]a; b[, [a; b[, and]a; b] stand Usuallyofonly the complexity perturbed/quantized of refined controllers state information cannot beisresolved available [a, b] Z, [1; 4[ = {1, 2, 3}, A static bycontroller merely revising notrefinement refined tomethods a staticifcontroller! existing relations In R n,therelations<, 6/33

7 In this Talk 1. A new notion ofreissig, system Weber, relation, and Runggertermed Feedback Refinement Relation, for controller refinement, so that the closed loop is given by input system abstract controller Feedback Refinement Relations for the Synthesis of Symbolic Co state quantizer refined controller Feedback refinement Figure 1. relations Closed loopare resulting necessary from the and abstraction sufficient and refinement for the approach illustrated refinement procedure based on feedback refinement relations, proposed in this paper. 2. We show how feedback refinement relations can be used to solve general abstraction control problems is not only and sufficient to ensure the simple structure synthesize robust of the closed controllers loop in Fig. 1, but in fact also necessary. 3. We show how to Our compute work abstractions builds a general for control notionsystems of of with the form setvalued dynamics and possibly non-deterministic quantizers. This is particularly useful ξ f to (ξ, model u) + Wvarious types of disturbances, including plant uncertainties, input disturbances and where W is a bounded state measurement set of disturbances errors. We demonstrate how to account for those perturbations in our framework so that the synthesized covers stateful s including transitio as special cases. useful e.g. to mod time or in the con occurrence of the any linear time p Besides the lim of robustness furt For example, [18 dynamics, while deterministic wh precise, without a The synthesis based on a novel theory in [24], fe metric of the stat the necessity as authors of [24] c effect of these pe 7/33

8 8/33 Feedback Refinement Relations Systems, Solutions, Behavior Serial Composition, Feedback Composition Main Theorem

9 Systems: Informal Introduction We consider dynamical systems of the form where x is the state u is the input y is the output F is the transition function H is the output function x(t + 1) F (x(t), u(t)) y(t) H(x(t), u(t)) In order to define a meaningful serial/feedback composition of this general type of systems we need internal variables u 1 x 1 F 1 H 1 u 2 y 1 x 2 F 2 H 2 y2 F 12 (x, u 1 ) = F 1 (x 1, u 1 ) F 2 (x 2, u 2 ), x = (x 1, x 2 ), u 2 H 1 (x 1, u 1 ) H 12 (x, u 1 ) = H 1 (x 1, u 1 ) H 2 (x 2, u 2 ), x = (x 1, x 2 ), u 2 H 1 (x 1, u 1 ) In the first and second line, we need to pick the same u 2! 9/33

10 10/33 F ˆ= : set-valued map F : X V 2 can be identified with relation F X V X, Systems but F depends on input only indirectly, via H, whereas X U X. Intuitive interpretation of functioning of a system: x(t + 1) F (x(t), v(t)) (y(t), v(t)) H(x(t), u(t)) x(t + x(0) 1) F(x(t), X 0 v(t)) x(t) v(t) F H y(t) x(t + 1) S (y(t), v(t)) H(x(t), u(t)) u: input; can be prescribed A system S is a tuple S = (X, X 0, U, V, Y, F y:, H) output; where can be measured X, U, V and Y are nonempty sets x, v: state, internal variable; no direct acces : delay X is the state alphabet X 0 X is the initial state alphabet U is the input alphabet V is the internal input alphabet Y is the output alphabet unther Reißig Lecture 12: Abstraction and Synthesis I Course: Modeling and Verification of Embedded Systems (TUM, M. Zamani) January 21, F : X V X is the transition function u(t) H : X U Y V is the output function and is assumed to be strict, i.e., (x,u) X U : H(x, u) Notation We use F : X Y to denote set-valued function from X to Y

11 11/33 Systems We call a system S = (X, X 0, U, V, Y, F, H) 1. static if X is a singleton 2. Moore if the output does not depend on the input, i.e., (y, v) H(x, u) u U = v (y, v ) H(x, u ); 3. basic if U = V and (y, v) H(x, u) = v = u; 4. Moore with state output if X = Y and (y, v) H(x, u) = y = x. For a basic Moore system with state output and X 0 = X we simply use S = (X, U, F ) We define the set U S (x) of admissible inputs at the state x X by U S (x) = {u U F (x, u) }

12 12/33 Solutions and Behavior Let S = (X, X 0, U, V, Y, F, H) be given. A quadruple (u, v, x, y) (U V X Y ) [0;T [ with T N { } is a solution of the system S on [0; T [ if x(0) X 0 x(t + 1) F (x(t), v(t)) holds for all t [0; T 1[ (y(t), v(t)) H(x(t), u(t)) holds for all t [0; T [ Behavior B(S) of S An element of the behavior (u, y) B(S) is an input/output sequence which is generated by a solution of the system S The generating solution either is infinite or ends in a blocking input/state pair (u, y) B(S) iff v,x,t (u, v, x, y) is a solution of S on [0; T [ T < implies that F (x(t 1), u(t 1)) =

13 13/ Serial Composition F 12(x, v) =F 1(x 1, v 1) F 2(x 2, v 2) H 12(x, u 1)={(y 2, v) Y 2 V 12 y1 (y 1, v 1) H 1(x 1, u 1) (y 2, v 2) H 2(x 2, y 1)} Let S i = (X i, X i,0, U i, V i, Y i, F i, H i ), i {1, 2} be two systems. S 2 S 1 is a system x 1 v F 1 H 1 y 1 1 F v 2 H y u 1 S 1 S 2 x 2 S 2 S 1 Gunther Reißig Lecture 12: Abstraction and Synthesis I Course: Modeling and Verification of Embedded Systems (TUM, M. Zamani) January 21, S 1 is serial composable with S 2 if Y 1 U 2 The serial composition of S 1 and S 2 is denoted by S 2 S 1

14 14/33 Serial Composition: Example u x ˆx ZOH ξ(t) = Aξ(t) + Bu Q τ Sample-and-hold system S = (R n, R m, F ) with { ( τ ) } F (x, u) = e Aτ x + e A(τ s) ds u Quantizer Q : R n Z n Q(x) = {ˆx Z n x ˆx 1} is identified with the static system Q = ({q}, {q}, R n, R n, Z n, F q, H q) F q(q, x) = {q} for all x R n H q(q, x) = Q(x) {x} 0 S is serial composable with Q and Q S = (R n, R n, R m, R m, Z n, F, H) is basic with H(x, u) = Q(x) {u}

15 15/33 H 12 : X 12 2 Y 12 V 12 Feedback Composition x 2, v 2 ), 2 (y Let 1, v S i = 1 ) H (X i, X 1 (x i,0, 1, y U i, 2 ) (y V i, Y i, F 2, v i, H 2 ) H i ), i 2 (x {1, 2, y 2} 1 )}. be two systems. v 2 F 2 x 2 H 2 y 2 S 2 y 1 H 1 x 1 F 1 v 1 S 1 I Course: Modeling and Verification of Embedded Systems (TUM, M. Zamani) January 21, S 1 is feedback composable with S 2 if Y 2 U 1 and Y 1 U 2 S 2 is Moore The feedback composition of S 1 and S 2 is denoted by S 1 S 2

16 16/33 Feedback Refinement Relations Let S i = (X i, U i, F i ), i {1, 2} be two systems and assume U 2 U 1 A strict relation Q X 1 X 2 is a feedback refinement relation from S 1 to S 2, denoted by S 1 Q S 2 if the following holds for all (x 1, x 2) Q: 1. U S2 (x 2) U S1 (x 1) 2. u U S2 (x 2) = Q(F 1(x 1, u)) F 2(x 2, u) In words 1. every adimissible input of S 2 at x 2 is an admissible input of S 1 at x 1 2. every sucessor x 1 F 1(x 1, u) when mapped to X 2 via Q is contained in F 2(x 2, u)

17 Comparison with Alternating Simulation Relations ASR VS FRR (x 1, x 2) Q implies u 2 U S2 (x 2) u 1 U S1 (x 1) x 1 F 1(x 1, u 1) we have (x 1, x 2) Q implies u U S2 (x 2) u U S1 (x 1) x 1 F 1(x 1, u) we have (asr) Q(x 1) F 2(x 2, u 2) Q(x 1) F 2(x 2, u) (frr) Difference 1. ASR: inputs U 1 and U 2 can be different 2. (asr) vs (frr) What is the problem with ASR? In the refinement with ASR 1. for (x 1, x 2 ) Q and u 2 to pick the matching u 1 we need to know concrete state x 1 2. and to pick the right x 2 Q(x 1 ) we need to know F 2(x 2, u 2 ) so that x 2 Q(x 1 ) and x 2 F 2(x 2, u 2 ) In the refinement with FRR 1. u can directly be applied to S 1 2. we can pick any x 2 Q(x 1 ) and use (frr) to ensure x 2 F 2(x 2, u) 17/33

18 Feedback Refinement Relations: Necessary Conditions Consider S i = (X i, U i, F i ), i {1, 2} C = (X c, X c,0, U c, V c, Y c, F c, H c) S 2 :abstraction S 1 :system x 1 strict Q X 1 X 2 and the statements 1. C is feedback composable with S 2 2. C is feedback composable with Q S 1 u C:controller x 2 u C:controller Q x 2 3. B(C (Q S 1)) B(C S 2) Theorem: If 1. implies 2. and 3. then S 1 Q S 2. We have to show U 2 U 1 (x 1, x 2) Q implies U S2 (x 2) U S1 (x 1) (x 1, x 2) Q and u U S2 (x 2) implies Q(F 1(x 1, u)) F 2(x 2, u) U 2 U 1 1. implies Y c U 2 2. implies Y c U 1 Since 1. implies 2. we have: Y c U 2 implies Y c U 1. Hence U 2 U 1 We have to show 18/33

19 19/33 Feedback Refinement Relations: Sufficient Conditions Consider S i = (X i, U i, F i ), i {1, 2} C = (X c, X c,0, U c, V c, Y c, F c, H c) S 2 :abstraction S 1 :system x 1 strict Q X 1 X 2 and the statements u x 2 u Q 1. C is feedback composable with S 2 2. C is feedback composable with Q S 1 C:controller C:controller x 2 3. B(C (Q S 1)) B(C S 2) Theorem: If S 1 Q S 2, then (1. and ( )) imply (2. and 3.) where (y c, v c) H c(x c, x 2) F 2(x 2, y c) = = F c(x c, v c) = ( ) ( ): C non-blocking = S 2 non-blocking In the proof we need C (Q S 1) non-blocking = C S 2 non-blocking

20 20/33 Feedback Refinement Relations: Sufficient Conditions Consider S i = (X i, U i, F i ), i {1, 2} C = (X c, X c,0, U c, V c, Y c, F c, H c) strict Q X 1 X 2 Corollary: If S 1 Q S 2, then 1. and ( ) imply C Q is feedback composable with S 1 for every (u, x 1) B((C Q) S 1) exists (u, x 2) B(C S 2) so that t dom(u,x1 ) : (x 1(t), x 2(t)) Q S 2 :abstraction S 1 :system S 1 :system x 1 x 1 u x 2 u Q u Q C:controller C:controller x 2 C:controller x 2

21 21/33 Applications Symbolic Synthesis Robust Synthesis

22 to denote the fact that S 1 Q S 2 and Σ 2 is an abstract specification associated with S 2, S 1, Σ 1 and Q 22/33 Specifications Consider a system S = (X, X 0, U, V, Y, F, H): any subset Σ (U Y ) is a specification for S S satisfies Σ if B(C S) Σ a system C solves the control problem (S, Σ) if C is feedback composable with S S and C satisfy ( ) (non-blocking requirement) C S satisfies Σ Consider two systems S i = (X i, U i, F i ), i {1, 2} with U 2 U 1 a specification Σ 1 for S 1 a strict relation Q X 1 X 2 A specification Σ 2 (X 2 X 2) for S 2 is an abstract specification associated with S 2, S 1, Σ 1 and Q if for any (u, x 1, x 2) (U X 1 X 2) [0;T [, T N { } with We use (u, x 2) Σ 2 and t [0;T [ : (x 1(t), x 2(t)) Q implies (u, x 1) Σ 1 (S 1, Σ 1) Q (S 2, Σ 2)

23 23/33 Symbolic Synthesis As a direct consequence of the behavioral inclusion B(C (Q S 1)) B(C S 2) for any system C and the defintion of abstract specification we get: Theorem: Let (S 1, Σ 1) Q (S 2, Σ 2). If C solves (S 2, Σ 2) then C Q solves (S 1, Σ 1). S 2:abstraction S 1:system x 1 u x 2 u Q C:controller C:controller x 2

24 Robust Synthesis (Teaser: Details are in the Paper) We consider system S 1 = (X 1, U 1, F 1) u S 1 x P 4 y 1 specification Σ 1 for S 1 quantizer Q perturbations P 1 accounts for actuator imprecisions P 2 accounts for measurement noise P 3 and P 4 are useful to robustify the spec y 2 P 3 P 1 P 2 y c C Q u c We show how to construct Figure: Perturbed System auxiliary system Ŝ 1 auxiliary quantizer ˆQ auxiliary specification ˆΣ 1 so that if C ˆQ solves (Ŝ 1, ˆΣ 1) then the behavior of the Perturbed System satisfies Σ 1. C ˆQ can be computed by an abstract problem (S 2, Σ 2) that satisfies (Ŝ1, ˆΣ 1) ˆQ (S2, Σ2) 24/33

25 Computation of Abstractions of Perturbed Control Systems 25/33

26 Computation of Abstractions We focus on X 2 is a cover of X 1, i.e., X 2 is a set of non-empty subsets of X 1 and X 1 x2 X 2 x 2 x 1 every cell x 2 X 2 is a subset of X 1 the quantizer Q is the set membership relation for a given x 1 X 1 the quantizer Q picks x 2 X 2 in a non-deterministic fashion so that x 1 x 2 Let S i = (X i, U i, F i ) be two systems i {1, 2} X 2 be a cover by non-empty sets of X 1 Theorem: S 1 S 2 if and only if 1. U 2 U 1 and x 1 x 2 X 2 implies U S2 (x 2) U S1 (x 1) 2. x 2, x 2 X 2, u U S2 (x 2) and x 2 F 1(x 2, u) implies x 2 F 2(x 2, u) Computation of abstraction reduces to computation (overapproximation) of reachable sets! 26/33

27 27/33 Abstractions of Perturbed Control Systems We consider a differential inclusion ξ f (ξ, u) + W We cast the sample-and-hold behavior of ( ) with τ R >0 as system ( ) S 1 = (R n, R m, F 1) The abstraction S 2 = (X 2, U 2, F 2) is given by U 2 R m X 2 is a cover of R n x 1 F 1(x 1, u) all driven by constant u cells are hyper-rectangles with center c R n and radius r R n >0 x 2 = c + r, r = [c 1 r 1, c 1 + r 1]... [c n r n, c n + r n] we overapproximate the reachable set by another hyper-rectangle c + r, r F 1(c + r, r, u) c + r, r and then determine the cells c + r, r X 2 with c + r, r c + r, r

28 How to compute c and r? Theorem: consider ( ) and S 1 = (R n, R m, F 1 ) for τ R >0 u U and f (, u) is continuously differentiable disturbances W = w, w, w R n 0 a cell c + r, r K, with K R n is convex ξ sol. of ( ) and ξ(0) K = ξ(t) K for all t [0, τ] dom ξ let L R n n satisfy for all x K { D L i,j (u) j f i (x, u) if i = j D j f i (x, u) if i = j Then D j f i : partial derivative of f i w.r.t. jth component of 1st argument F 1 (c + r, r, u) c + r, r where c and r follow by the solution at time τ of c : ẏ = f (y, u) y(0) = c r : ż = Lz + w z(0) = r Overapproximation of reachable set is reduced to estimation of partial derivatives of f solution of two unperturbed initial value problems 28/33

29 29/33 Numerical Examples A planning problem for a mobile robot An aircraft landing maneuver

30 30/33 Reach-Avoid Specifications The desired behavior Σ in both exampels is defined in terms of three sets A init : the set of initial states A avoid : the set of obstacles A reach : the target set Given S = (X, U, F ), we define a reach-avoid specification Σ (U X ) for S by (u, x) Σ x(0) A init = T x(t ) A reach t [0;T ] x(t) A avoid Given Ŝ = ( ˆX, Û, ˆF ) with S Ŝ the abstract specificiation is defined by the reach-void specification induced by the sets  init = {ˆx ˆX ˆx A init }  avoid = {ˆx ˆX ˆx A avoid }  reach = {ˆx ˆX ˆx A reach }

31 Robot Path Planning unicylce/segway dynamics robot ξ 1 = u 1 cos(α + ξ 3) cos(α) 1 ξ 2 = u 1 sin(α + ξ 3) cos(α) 1 ξ 3 = u 1 tan(u 2) ξ 2 ξ 3 where α = arctan(tan(u 2)/2) ξ 1 and ξ 2 coordinates in the plane ξ 3 orientation u 1 forward velocity u 2 steering angle sampling time τ = 0.3 sec computation of S 2 = (X 2, U 2, F 2) number of states X number of inputs U 2 = 49 number of transitions computation times 2.33 sec to compute the abstract problem 0.22 sec to solve the abstract problem specification A init ξ 1 A avoid closed loop simulation A target 31/33

32 y x Aircraft Landing Maneuver aircraft dynamics aircraft ξ 1 = 1 m (u 1 cos u 2 D(u 2, ξ 1 ) mg sin ξ 2 ) ξ 2 = 1 mx 1 (u 1 sin u 2 + L(u 2, ξ 1 ) mg cos ξ 2 ) ξ 3 = ξ 1 sin ξ 2 ξ 1 forward velocity ξ 2 flight path angle ξ 3 altitude u 1 [ 0, ] engine thrust u 2 [0, 10 ] angle of attack actuator and measurement errors [ P 1(u) = (u , ] [ 0.25, 0.25 ] ) U P 2(x) = x + 1 [ 1 20 [ 0.25, 0.25] , 0.05 ] 1 20 [ 1, 1] sampling time τ = 0.25 sec computation of S 2 = (X 2, U 2, F 2) number of states X number of inputs U 2 = 20 number of transitions min to compute (S 2, Σ 2 ) 26 sec to solve (S 2, Σ 2 ) specification: steer the aircraft from 55 m close to the ground with a given total and horizontal touchdown velocity landing maneuver /33

33 33/33 Summary and Related Work We have seen Feedback refinement relations Solve general control problems via abstraction and refinement Account for various perturbations Computation of abstractions Related and ongoing work Abstraction-based solution of optimal control problems Combination of the computation of abstractions within existing toolboxes to solve synthesis problems with more complex specifications Various work to reduce the complexity of the computation of the abstraction compositional construction of abstractions merge the computation of the abstraction with the solution of the abstract control problem pick a cell radius so as to minimize number of transitions Gunther: local refinement of the abstract state space Gunther: Validated numerics: account for approximation errors in the solution of ODE and other computations...