Dynamic and Adversarial Reachavoid Symbolic Planning Laya Shamgah Advisor: Dr. Karimoddini July 21 st 2017 Thrust 1: Modeling, Analysis and Control of Large-scale Autonomous Vehicles (MACLAV) Sub-trust 1-2: Cooperative Localization, Navigation and Control of LSASVs 1
Motivation Reach-avoid Problem: Traveling from an initial point to a desired location while avoiding obstacles Static Environment Dynamic Environment Dynamic Adversarial Environment Challenge: Autonomous Coordination of autonomous vehicles to achieve their sophisticated goals in an dynamic and adversarial environment Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 2
Objective Objective of research: To develop a computationally effective reactive planning method for autonomous vehicles in a dynamic adversarial environment. Dynamic Adversarial Reach-avoid scenario: attacker: tries to reach the target while avoiding of capture. defender: tries to capture the attacker before reaching the defending area. Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 3 3
Challenges and Gaps Existing methods Pursuit-evasion games [Bhadauria et al. 2012] Probabilistic approaches [Vitus et sl. 2011] Differential games [Tomlin et al.2011,2015] Challenges Solving only the avoidance problem Assuming limitations on the vehicle s movements Requiring information about the opponent vehicle High computational cost Lack of Reactiveness Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 4 4
Proposed approach To reduce the complexity: 1- Using Symbolic Control Techniques for abstraction of the (infinitely) large original problem to a (finite) small abstracted environment, 2- Designing a DES supervisor to achieve a complex task over an abstract environment 3- Projecting back the solution to the original domain. Remark: This is the first result in the literature that uses symbolic control techniques for the reach-avoid problem. attacker Target Abstraction? defender P Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 5 5
Proposed Hybrid Structure DES supervisor Abstraction of Vehicle Dynamics Discrete Signal Bisumulation-based abstraction Interface Continuous Signals Vehicle Dynamics Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 6 6
Proposed Implementation Approach Hierarchical Control Supervisor Supervisor, operator, Temporal Logic Symbolic Planning Planner high-level Controller Real-time low-level controller Low-level Controller Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 7 7
Reach-avoid Problem Description attacker? defender Target P Assumptions: Defender vehicle dynamics: x t = f(x t, u(t)) Environment (P) is a bounded convex set Target is in a fixed position The initial position of the attacker and the defender are within P Defender vehicle has full observability over the position of the attacker other Problem: Design a controller to obtain trajectory x t P = j=1,,m i=1,,n P ij, which satisfies the objective of the defender. Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 8 8
Proposed Framework Design Steps: 1.Extracting decision-making strategies 2.Construction of LTL Specification φ = φ a φ d 3.Checking realizability of φ 4.Synthesizing the supervisor automaton G which satisfies φ 5.Designing the hybrid controller Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 9 9
Step1: Optimal Decision-making Strategies Modeled as a finite two-player zero-sum game in matrix form Attacker is the maximizer player and Defender is the minimizer Objective Function P 11 P 12 P 13 P 21 a P 22 P 23 * P 31 P 32 P 33 d 0 if x a, x d P ij L x a, x d = ifx a, x t P ij α x a x β d + x a x t + γ x d x t otherwise Distance between the vehicles Distance between the attacker and the target Distance between the defender and the target Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 10 10
Step1: Decision-making Example: P 11 P 12 P 13 P 21 a P 22 P 23 * P 31 P 32 P 33 d Optimization Parameters: α = 1 β = 1 γ = 0.5 defender attacker P 22 P 31 P 11 P 32 3.414 4.650 4.650 P 23 2 3.236 3.236 Defender : min max 3.414, 4.650, min 2, 3.236 = 3.236 P 23 Attacker : max min 3.414, 2, 4.650, 3.236 = 3.236 P 11 Nash Equilibrium decision : (a 11, d 23 ) Temporal formula (a 21 d 33 d 23 ) nm(nm 2) games should be solved to calculate all the temporal transition rules. Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 11 11
Step 2: Construction of LTL Specification Classical logic: I am hungry Temporal logic "I am always hungry "I will eventually be hungry "I will be hungry until I eat something" Temporal logic: Linear Temporal Logic (LTL) is a formal high-level language to describe many complex missions and a wide class of properties can be expressed by LTL: Coverage: eventually visit all regions Sequencing: visit P2 before you go to P3 Avoidance: until you go to P2 avoid P1 and P3 Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 12
Step 2: Construction of LTL Specification The LTL formulas (φ) are constructed over (Σ) using Boolean operators and temporal operators. Σ : A finite set of atomic proposition: p Σ (p can be either T or F) Boolean operators: negation ( ), disjunction ( ), conjunction ( ), implication ( ) Modal temporal operators: next (O), until (U), eventually ( ) and always ( ) Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 13
Step 2: Construction of LTL Specification Temporal Operators: Operators Definition Diagram φ φ is true in the next moment of time φ φ is true in all future moments φ φ is true in some future moment φuψ φ is true until ψis true Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 14
Step 2: Construction of LTL Specification Static Environment Reactive to changes in Dynamic Environment Dynamic Environment Vehicle φ vehicle φ = (φ e φ s ) Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 15
Step 2: Construction of LTL Specification Reach-avoid Specification: φ = φ a φ d φ a : all assumptions on the attacker φ d : all assumptions on the defender and its desired behavior v v φ v = φ init φ sing v φ term φ v v rul φ obj attacker? defender Target P v 1 φ init v 2 φ sing Boolean (B) Temporal ( T) Initial position of vehicle v 3 φ term Temporal ( T) Termination of the game v 4 φ rul v 5 φ obj Temporal ( T) Temporal ( B) Singularity constraint: At each time the vehicle can be in only one region Transitions rules over the partitioned area Objective of the vehicle Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 16
Step 3-4: Discrete Design Procedure Step 3: Checking realizability of φ Check if there exists any admissible behavior of the attacker such that no behavior of adapter can satisfy φ d. Step 4: Synthesis of automaton G If φ is realizable then G = Q, q 0, A, D, δ, h Synthesis Process: GS =< V, A, D, Θ, ρ a, ρ d, φ > G φ Every path on G is a behavior of the attacker and the corresponding behavior of the defender, which ends when the defender will win Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 17 17
Step 5: Hybrid Control Design Online implementation: Heading angle(θ) Velocity(u) Attacker s behavior Discrete path Interface Continuous path x(t) a i a i+1 d i d i+1 Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 18 18
Example: Description attacker P 11 P 12 P 13 defender P 21 P 22 P 23 Operation region Initial positions Target P = 3 i,j=1 P ij attacker: P 11 defender: P 31 P 23 P 31 P 32 P 33 Problem: Design a controller to obtain trajectory x(t) which satisfies φ = φ a φ d Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 19 19
Example: Task Specification all assumptions on the attacker φ = φ a φ d all assumptions on the defender and its desired behavior φ d = φ d d init φ sing d φ term d φ init d 13 d 11 d 12 d 33 d φ sing d φ term φ d d rul φ obj [(d 11 d 12 d 13 d 33 ) ] [((a 23 a 11 ) d 11 ) d 11 ) ] d φ rul [((a 11 d 31 a 21 ) d 21 ) ] d φ obj [ a 11 d 11 a 12 d 12 ] a φ init a φ sing a φ term a φ rul a φ obj φ a = φ a a init φ sing a φ term a 11 a 12 a 13 a 33 φ a a rul φ obj [(a 11 a 12 a 13 a 33 ) ] [(a 23 a 23 ) [(a 11 d 11 ) a 11 ) ] [(a 11 ( a 12 a 21 )) ] True Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 20 20
Example: Discrete Results 1. φ = φ a φ d is realizable. 2. Automaton G: Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 21 21
Example: Final Results (1) Discrete path Continuous path a 11 a 12 a 12 a 13 Attacker Defender a 11 a 12 a 13 a 12 d 31 d 32 d 22 d 12 Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 22 22
Example: Final Results (2) Discrete path Continuous path a 11 a 21 a 12 a 11 Attacker Defender a 11 a 12 a 22 a 21 d 13 d 32 d 22 d 21 Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 23 23
Example: Final Results (3) Discrete path Continuous path a 11 a 12 a 12 a 11 Attacker Defender a 11 a 12 a 11 a 12 d 13 d 32 d 22 d 12 Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 24 24
Conclusion Conclusion: A novel, formal hybrid symbolic controller was developed for the vehicles involved in a reach-avoid scenario. Significance of the results: Future Work: To the best of our knowledge, this is the first work in the literature that employs symbolic motion planning for reach-avoid problem. The proposed method is a computationally effective method that can reactively capture the changes in a dynamic and adversarial environment. The developed approach considers less restrictions on the robot motion and requires no knowledge about the model of the opponent. The extension of the proposed framework to more complex scenarios and environments, for example reach-avoid problem with more number of players. Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 25 25
Thank You 26