Receding Horizon Temporal Logic Planning for Dynamical Systems

Transcription

1 Submitted, 2009 Conference on Decision and Control (CDC) Receding Horizon Temporal Logic Planning for Dynamical Systems Tichaorn Wongpiromsarn, Ufu Topcu, and Richard M. Murray Abstract This paper bridges the advances in computer science and control to allow automatic synthesis of complex dynamical systems which are guaranteed, by construction, to satisfy the desired properties even in the presence of adversary. The desired properties are expressed in the language of temporal logic. With its expressive power, a wider class of properties than safety and stability can be specified. The resulting system consists of a discrete planner which plans, in the abstracted discrete domain, a set of transitions of the system to ensure the correct behaviors and a continuous controller which continuously implements the plan. For a system with certain structure, we present an approach, based on a receding horizon scheme, to overcome computational difficulties in the synthesis of a discrete planner and allow more complex problems to be solved. I. INTRODUCTION Recent advances in computer science, such as the development of a polynomial-time algorithm to construct finite state automata from their temporal logic specifications, enable automatic synthesis of digital designs which satisfy a large class of properties including safety, guarantee and response even in the presence of adversary [1]. On the other hand, recent advances in control and abundance of computational resources enable automatic synthesis of continuous controllers which ensure the safety and stability of the systems even in the presence of disturbances and modeling errors [2], [3], [4]. In many applications, systems need to perform complex tass and interact with (potentially adversarial) environments. These systems may contain both continuous and discrete components. A big challenge in this paradigm is to integrate the methods from the two communities such that automatic synthesis of such systems is possible. Hybrid system theory has been developed to deal with systems with both discrete and continuous components. Control of hybrid systems has been studied extensively but properties of interest are typically limited to stability and safety [5], [6]. For the system to be able to perform complex tass, a wider class of properties such as guarantee (e.g. eventually perform tas 1 or tas 2 or tas 3) and response (e.g. if the system fails, then eventually perform tas 1 or perform tass 1, 2 and 3 infinitely often in any order) need to be considered. Temporal logics have therefore garnered great interest due to their expressive power. In particular, MILP formulation has been used to incorporate linear temporal logic in control [7]. The interaction with potentially adversarial environments, however, is not taen into consideration. The development of language equivalence and bisimulation notions allows abstraction of the continuous component This wor is partially supported by AFOSR and the Boeing Corporation. T. Wongpiromsarn, U. Topcu and R. M. Murray are with the Division of Engineering and Applied Science, California Institute of Technology, Pasadena, CA 91125, USA. no@caltech.edu, utopcu@cds.caltech.edu, murray@cds.caltech.edu of the system to a purely discrete model while preserving all the desired properties [8]. This subsequently provides a hierarchical approach to system design. In particular, a two-layer design is common and widely used in the area of planning and control. In the first layer, a discrete planner plans, in the abstracted discrete domain, a set of transitions of the system to ensure the satisfaction of the desired properties. This abstract plan is then continuously implemented by a continuous controller in the second layer. Bisimulations provide the proof that the continuous execution preserves the desired properties. This two-layer approach has been applied to robot motion planning for a special case of fully actuated (ẋ u) [9], [10] and inematic (ẋ A x u) [11] models. In [10], [11], the discrete planner is constructed using a digital design synthesis tool [1]. The main limitation of this tool is the state explosion problem which restricts its applications to small problems. The contribution of this paper is twofold. First, we extend the wor of [10], [11] to a more complicated system with linear and piecewise affine (PWA) dynamics. A notion of reachability is defined that is sufficient to ensure that the continuous execution preserves the correctness of the discrete plan. The requirement of a bisimulation abstraction is then relaxed to a simulation abstraction which is enforced by restricting the set of discrete plans to those satisfying the reachability relation which can be established by solving a multiparametric programming problem. The Multi-Parametric Toolbox [4] provides an off-the-shelf computational machinery that enables the multiparametric programming problem to be solved in an automated fashion. Together with the digital design synthesis tool [1], this allows automatic design of dynamical systems which satisfy a wide range of properties expressed in temporal logic, taen into account the interaction with potentially adversarial environments. Second, to address the state explosion problem in the digital design synthesis, we present a receding horizon scheme for executing finite state automata while ensuring system correctness. This allows the synthesis to be performed on a smaller domain and thus potentially substantially reduce the size of the space of the synthesis problem. As a result, more complex problems can be handled. II. PRELIMINARIES We use linear temporal logic (LTL) to describe the desired properties of the system. Given an LTL formula, we want to construct a finite state automaton, which can be thought of as a graph with a finite number of nodes (representing the states of the system) and edges (representing the transitions between states), such that the state transitions in the automa-

2 ton ensure the correctness of the system. In this section, we briefly describe the definition of LTL and the synthesis of a finite state automaton which satisfies a given LTL formula. A. Terminology and Notations Definition 1: A system consists of a set of variables V. The domain of V, denoted by dom V, is the set of valuations of V. A state of the system is an element v dom V. Definition 2: A finite transition system is a tuple D, where is a finite set of states, and is a transition relation. Given states i, j, we write i j if there is a transition from i to j. Definition 3: An atomic proposition is a statement on system variables υ which has a unique truth value (True or False) for a given value of υ. Let v dom V be a state of the system and π be an atomic proposition. We write v π if π is True at the state v. Otherwise, we write v π. Definition 4: An execution of a discrete-time system is an infinite sequence of the states of the system over a particular run, i.e. an execution σ can be written as σ v 0 v 1 v 2... where for each t 0, v t dom V is the state of the system at time t. B. Linear Temporal Logic The use of linear temporal logic (LTL) as a specification language was introduced by Pnueli [12], [13]. LTL is built up from a set of atomic propositions, the logic connectives (,,, ), and the temporal modal operators (,,, ) 1. An LTL formula is defined inductively as follows. 1) Any atomic proposition π is an LTL formula. 2) Given an LTL formula ϕ and ψ, the following are also LTL formulas: ϕ, ϕ ψ, ϕ and ϕ ψ. Other operators can be defined as follows: ϕ ψ ϕ ψ, ϕ ψ ϕ ψ, ϕ True ϕ, and ϕ ϕ. A propositional formula is one that does not include temporal operators. Given a set of LTL formulas ϕ 1,..., ϕ n, their boolean combination is an LTL formula formed by joining ϕ 1,..., ϕ n with logic connectives. Semantics of LTL: An LTL formula is interpreted over an infinite sequence of states. Given an execution σ v 0 v 1 v 2... and an LTL formula ϕ, we say that ϕ holds at position i 0 of σ, written v i σ, if and only if ϕ holds for the remainder of the execution σ starting at position i. The semantics of the temporal logic formula is defined as follows: 1) For an atomic proposition π, v i π iff v i π 2) v i ϕ iff v i ϕ 3) v i ϕ ψ iff v i ϕ or v i ψ 4) v i ϕ iff v i 1 ϕ 5) v i ϕ ψ iff j i, v j ψ and i, j,v ϕ Based on the above definition, ϕ holds at position i iff ϕ holds at every position in σ starting at position i, and ϕ holds at position i iff ϕ holds at some position j i in σ. Definition 5: An execution σ v 0 v 1 v 2... satisfies ϕ, denoted by σ ϕ, if v 0 ϕ. Let Σ be the set of all executions 1,, and are read as next, always, eventually, and until, respectively. Some authors also use release ( ) which can be defined as ψ ϕ ψ ϕ for any LTL formulas ψ and ϕ. of a system. The system is said to be correct with respect to its specification ϕ, written Σ ϕ, if all its executions satisfy ϕ, that is, Σ ϕ σ, σ Σ σ ϕ. (1) C. Synthesis of Finite State Automata In many applications, systems need to interact with the environments and whether they satisfy the desired properties depends on what the environments do. In this section, we informally describe the wor of Piterman, et al. on Synthesis of Reactive(1) Designs [1]. We refer the reader to [1] and references therein for the detailed discussion of automatic synthesis of a finite state automaton from its specification. From Definition 5, for a system to be correct, its specification ϕ must be satisfied regardless of what the environment does. Thus, the environment can be treated as adversary and the synthesis problem can be viewed as a two-player game between the system and the environment: the environment attempts to falsify ϕ while the system attempts to satisfy ϕ. We say that ϕ is realizable if the system can satisfy ϕ no matter what the environment does. For a specification of the form ψ j, (2) i I ϕ i nown as Generalized Reactivity(1), Piterman et al. shows that checing its realizability and synthesizing the corresponding automaton can be performed in polynomial time. In particular, we are interested in a specification of the form j J ϕ ϕ e ϕ s (3) where roughly speaing, ϕ e is an LTL formula that characterizes the initial states of the system and the assumptions of the environment and ϕ s is an LTL formula that describes the correct behavior of the system, including the valid transitions the system can mae. We refer the reader to [1] for precise definitions of ϕ e and ϕ s. Note that since ϕ e ϕ s is satisfied whenever ϕ e is False, if the assumptions of the environment or initial state of the system violate ϕ e, the correct behavior ϕ s of the system is not ensured, even though the specification ϕ is satisfied. If the specification is realizable, the synthesis algorithm generates a finite state automaton that represents a set of transitions the system should follow in order to satisfy ϕ. Assuming that the environment and the initial state of the system satisfy ϕ e, at any instance of time, there exists a node in the automaton that represents the current state of the system, and the system can follow the transition from this node to the next based on the current nowledge about the environment. However, if ϕ e is violated, the automaton is no longer valid, meaning that there may not exist a node in the automaton which represents the current state of the system, or even though such a node exists and the system follows the transitions in the automaton, the correct behavior ϕ s is not guaranteed. If the specification is not realizable, the synthesis algorithm provides an initial state of the system for which there

3 exists a set of moves of the environment such that the system cannot satisfy ϕ. The nowledge of the realizability of the specification is useful since it provides information about under what ind of conditions the system will fail to satisfy its desired properties. The main limitation of the synthesis of finite state automata is the state explosion problem. In the worst case, the resulting automaton may contain all the possible states of the system. For example, if the system has 10 variables, each can tae any value in 1,..., 10, then there may be as many as nodes in the automaton. III. PROBLEM FORMULATION Consider a system S with a set of variables V S E where S and E represent, respectively, the set of variables controlled by the system and the set of variables controlled by the environment. The domain of V is therefore given by dom V dom S dom E and a state of the system can be written as v s, e where s dom S and e dom E. Throughout the paper, we call s the controlled state and e the environment state. Assume that the controlled state evolves according to the following discrete-time piecewise-affine (PWA) dynamics 2 : s t 1 A s t B u t C if s t,u t Ω u t U where 1,..., N PWA, N PWA is the number of regions in the PWA partition, Ω 1,...,Ω NPWA is a polyhedral partition of dom S U, for any natural number t, s t dom S is the controlled state at time t, u is the control signal and U is the set of admissible control inputs. Let Π be a finite set of atomic propositions of variables from V and ϕ be an LTL specification built from Π. Assume that ϕ can be expressed without the next ( ) operator. 3 We are interested in designing a controller for the system which ensures that any execution σ v 0 v 1... satisfies ϕ where for each natural number t, v t dom V is the state of the system at time t. IV. HIERARCHICAL APPROACH In general, constructing a controller which ensures that any execution of the system satisfies the specification ϕ while respecting the dynamics (4) is hard since both the adversarial nature of the environment and the dynamics of the system need to be taen into account. To separate the concern of the environment from the concern of the dynamics, we apply the hierarchical approach, similar to [9], [10], [11], to solve the problem defined in Section III. That is, we decompose the problem into (a) designing a discrete planner that computes a discrete plan satisfying the specification ϕ regardless of what the environment does and (b) designing a continuous controller that implements the discrete plan while ensuring that the evolution of the system satisfies the dynamics (4). 2 We restrict ourselves to PWA dynamics for computational reasons. Our framewor applies to nonlinear dynamics but no computational scheme to date is able to handle them efficiently. 3 This assumption is sufficient to ensure that ϕ is stutter invariant. See, for example, [14] for more detail. (4) The discrete planner can be automatically synthesized using a digital design synthesis tool [1] as described in Section II-C. However, since the synthesis algorithm requires a finite domain, the system S must be abstracted to a finite transition system. To construct a finite transition system D from S, we first partition dom S and dom E, as in [9], [10], [11], into a finite number of equivalence classes or cells and, respectively, such that the partition is proposition preserving [8]. Roughly speaing, this means that for any atomic proposition π Π and any states v 1 and v 2 that belong to the same cell in the partition, if v 1 satisfies π, then v 2 also satisfies π. We denote the resulting discrete domain of the system by. Throughout the paper, we call v dom V a continuous state and ν a discrete state of the system. For a discrete state ν, we say that ν satisfies an atomic proposition π Π, denoted by ν d π, if and only if there exists a continuous state v contained in the cell labeled by ν such that v π. (Due to the proposition preserving property of the partition, this implies that for any continuous state v contained in the cell labeled by ν, v π.) Given an infinite sequence of discrete states σ d ν 0 ν 1 ν 2... and an LTL formula ϕ built from Π, we say that ϕ holds at position i 0 of σ d, written ν i d ϕ, if and only if ϕ holds for the remainder of σ d starting at position i. With these definitions, the semantics of LTL for a sequence of discrete states can be derived from the general semantics of LTL defined in Section II-B. Next, we need to determine the transition relations of D. As opposed to the special case of fully actuated dynamics considered in [9], [10], in our case, the dynamics (4), in general, constrain the evolution of the continuous state of the system which subsequently constrains a set of valid transitions of D. As a result, the transition relations of D cannot be simply defined based on the topological adjacencies of the cells as in [9], [10]. Furthermore, constructing a bisimulation partition as in [9], [10] for a general system with PWA dynamics is hard. In this section, we relax the requirement that the partition is bisimulation and define the notion of reachability that is sufficient (but not necessary) to guarantee that if a discrete controlled state j is reachable from i, the transition from i to j can be continuously implemented or simulated by a continuous controller. (See, for example, [15] for the exact definition.) A computational scheme that provides sufficient condition for reachability between two discrete controlled states is also presented in Section IV-A. A. Reachability Let 1, 2,..., n be a set of discrete controlled states. We define a map T s dom S that sends a continuous controlled state to a discrete controlled state of its equivalence class. That is, Ts 1 i dom S is a set of all the continuous controlled states contained in the cell labeled by i and Ts 1 i,..., Ts 1 n is the partition of dom S. We define the reachability relation, denoted by, as follows: a discrete state j is reachable from a discrete state i, written i j, only if starting from any point s 0 Ts 1 i, there exists a control law u U that taes the system (4) to a point s T Ts 1 j satisfying the constraint

4 s t Ts 1 i Ts 1 j, t 0,..., T. Note that this is stronger than the usual definition of reachability [16]. We write i j if j is not reachable from i. Clearly, if Ts 1 i Ts 1 j (i.e. Ts 1 i and Ts 1 j are not topologically adjacent), then i j. In general, for two discrete states i and j, verifying the reachability relation i j is hard. Therefore, we resort to a heuristic based on the following optimal control problem: Given discrete controlled states i, j, the set of admissible control inputs U, the matrices A and B as in (4), a horizon length N 0 and the cost matrices P N,Q 0 and R 0, solve min u 0,...,u N 1 P N s N 2 N 1 t 0 Qs t 2 Ru t 2 s.t. s N Ts 1 j, s 0 dom S s t 1 A s t B u t if s t,u t Ω u t U s t Ts 1 i Ts 1 j t 0,..., N 1. (5) Note that (5) is a finite horizon optimal control problem. Furthermore, one can consider the problem in (5) as a family of problems parametrized by s 0 and it can be regarded as a multiparametric programming problem [3]. For certain choices of Ts 1 i, Ts 1 j, and U (for example polytopic sets, i.e., sets defined by affine inequalities), the Multi- Parametric Toolbox [4] computes the explicit solution for this multiparametric programming problem, i.e., computes a partition i,j of some subset of Ts 1 i Ts 1 j such that for any s 0 i,j, the problem in (5) is feasible. An example of a set i,j along with i and j is shown in Figure 1. Fig. 1. An example of a set i,j represented by the unshaded region. For any s 0 in the shaded region, the optimal control problem (5) is infeasible. Different unshaded regions have different associated controllers. For more detail, see [4]. B. State Space Discretization In general, given the previous partition of dom S and any i, j 1,..., n, the reachability relation between i and j may not be established through the solution of the multiparametric programming problem (5) since T s 1 i is not necessarily covered by i,j (due to the constraints on u and a specific choice of the finite horizon N). This section describes a state space discretization scheme based on the reachability relation defined earlier to increase the number of valid discrete state transitions of D. The underlying idea is that for each i and j, we determine i,j such that for any s 0 i,j, the problem in (5) is feasible. Then, we partition Ts 1 i into Ts 1 i i,j, labeled by i,1 and Ts 1 i i,j, labeled by Ts 1 S i,2 and obtain the following reachability relations: i,1 j and i,2 j. TABLE I DISCRETIZATION ALGORITHM Discretization Algorithm input: The lower bound on cell volume (Vol min ), the parameters of the multiparametric problem (A, B, Ω, U, N, P N, Q, R), and the original partition ( Ts 1 i i 1,..., n ) output: The new partition sol sol Ts 1 i i 1,..., n ; IJ i, j i, j 1,..., n ; while (size IJ 0) Pic an i, j IJ ; Solve the multiparametric problem for i,j; if (volume sol i i,j Vol min and volume sol i i,j Vol min ) then Replace sol i with sol i i,j and add sol i i,j to sol; For each 1,..., size sol, add i,,, i, size sol,, and, size sol to IJ ; else Remove i, j from IJ ; endif endwhile Discretization Algorithm: Pic a natural number N and the cost matrices P N,Qand R. Define a lower bound Vol min on the volume of each cell in the new partition. Starting with a pair i, j where i, j 1,..., n, determine the set i,j such that for any s 0 volumes of both Ts 1 i i,j and T 1 i,j, the problem in (5) is feasible. If the s i i,j are greater than Vol min, then partition T 1 s i into T 1 T 1 s i i,j and s i i,j. Keep iterating this process until none of the cells can be partitioned. Table I shows the pseudo-code of the algorithm. Remar 1: Vol min only provides a terminating criterion for the proposed algorithm. Other criteria such as the maximum number of iterations can be used as well. Remar 2: The proposed discretization algorithm terminates when no cell can be partitioned such that the volumes of the two resulting new cells are both greater than Vol min. Larger Vol min causes the algorithm to terminate sooner. Remar 3: The point at which the algorithm terminates affects the reachability between discrete controlled states of the new partition and as a result, affects the realizability of the specification. Generally, a coarse partition maes the specification unrealizable but a fine partition causes state space explosion. A way to decide when to terminate the algorithm is to start with a coarse partition and eep refining it until the specification is realizable. C. Correctness of the System Let 1, 2,..., m be the set of all the discrete controlled states corresponding to the resulting partition of dom S after applying the discretization algorithm proposed in Section IV-B. Since the partition obtained from the proposed algorithm is a subpartition of Ts 1 1,..., Ts 1 n and is proposition preserving, it is trivial to show that is also proposition preserving. We define the finite transition system D which serves as the abstract model of S as follows: is the set of states of D and for any two states i is, ie and j js, je, i j (i.e. there exists a transition from i to j) only if is js. Using the abstract model D, a discrete planner that guarantees the satisfaction of ϕ while ensuring

5 that the discrete plans are restricted to those satisfying the reachability relations can be automatically synthesized using a digital design synthesis tool as described in Section II-C. From the stutter invariant property of ϕ, the formulation of the optimal control problem (5) and the proposition preserving property of, it is straightforward to show the following proposition. Proposition 1: Let σ d ν 0 ν 1... be an infinite sequence of discrete states of D where for each natural number, ν ν 1, ν ρ,ɛ, ρ is the discrete controlled state and ɛ is the discrete environment state. If σ d d ϕ, then by applying a sequence of control laws, each corresponding to the solution of (5) with i ρ and j ρ 1, the infinite sequence of continuous states σ v 0 v 1 v 2... satisfies ϕ. V. RECEDING HORIZON STRATEGY As discussed in Section II-C, automatic synthesis of finite state automata from their LTL specifications [1] suffers from the state explosion problem. In many applications, however, it is not necessary to plan for the whole execution, taing into account all the possible behaviors of the environment since the state that is very far from the current state of the system typically does not affect the near future plan. For example, consider a robot motion planning problem where the robot has to travel 100 ilometers. Under certain conditions, it may be sufficient to only plan out an execution for 500 meters and implement it in a receding horizon fashion, i.e., re-compute the plan as the robot moves. In this section, we present a sufficient condition and a receding horizon scheme that allows the synthesis to be performed on a smaller domain; thus, substantially reduces the number of states (or nodes) of the automaton while still ensuring the system correctness. We consider a subclass of Generalized Reactivity(1) formulas (2): (a) Let ϕ init be a propositional formula of variables from V which characterizes the initial state of the system; (b) let ϕ e be a boolean combination of propositional formulas of variables from V and expressions of the form ψ e where ψ e is a propositional formula of variables from E which describes the assumptions on the transitions of environment states; (c) for each j J where J is a finite set, let ϕ j be a propositional formula of variables from V which describes the assumption that must be satisfied infinitely often; (d) let ϕ s be a boolean combination of propositional formulas of variables from V and expressions of the form ψ s where ψ s is a propositional formula of variables from V which describes the constraints on the transitions of system states; and (e) let ϕ g be a propositional formula of variables from V. We consider a specification of the finite transition system of the form ϕ init ϕ e j J ϕ j ϕ s ϕ g (6) where ϕ s and ϕ g express the safety and the progress properties of the system. Let be the discrete domain of the system after applying the discretization algorithm presented in Section IV. Similar to the map T s for the controlled states defined in Section IV, we let T dom V be a map which sends a continuous state to a discrete state of its equivalence class, i.e. for each i, T 1 i dom V is a set of all the continuous state contained in the cell labeled by i and T 1 i i is a partition of dom V. For the progress property ϕ g, suppose there exists a collection of disjoint subsets 1,..., p of such that (a) p, (b) ϕ g is satisfied for any v T 1 i p i, and (c) 1,..., p, ϕ g is a partially ordered set where for any i and j, i ϕ g j if and only if any execution starting from v T 1 i and satisfying the property ϕ g contains a state v T 1 j. By the abuse of notation, for each i 1,..., p, we let T 1 i T 1 i. Suppose there exists a propositional formula Φ of variables from V and for each i 1,..., p, there exist g i 1,..., p and a subset i of dom S satisfying the following conditions: (1) ϕ init Φ is a tautology, i.e., any state that satisfies ϕ init also satisfies Φ, (2) T 1 i, T 1 g i i dom E, and (3) i ϕ g g i and for each i p, i ϕ g g i such that Ψ i v T 1 i Φ ϕ e j J ϕ j ϕ s v T 1 g i Φ is realizable with the domain of S restricted to i. For i 1,..., p, let i be an automaton that satisfies Ψ i. Since in the synthesis of i, the domain of S is restricted to i, this can substantially reduce the number of states in the automaton, especially when the size of i is much smaller than the size of dom S. Receding Horizon Strategy: Starting from the state v 0, pic an automaton i such that v 0 T 1 i and execute i until the system reaches the state v T 1 j where j ϕ g i, at which point, switch to the automaton j. Keep iterating this process until p is executed. Theorem 1: Suppose for each i 1,..., p, Ψ i is realizable. Then the proposed receding horizon strategy ensures the correctness of the system. Proof: Consider an arbitrary execution σ of the system that satisfies the formula to the left of in (6). From the tautology of ϕ init Φ, it is easy to show that if σ starts from v T 1 i, then σ satisfies the formula to the left of in (7). Let v 0 dom V be the initial state of the system. First, suppose v 0 T 1 p. Then, the system always executes p; thus, Ψ p ensures that σ satisfies (6). Next, suppose v 0 T 1 i where i p. Then, the system executes i and Ψ i ensures that the safety property ϕ s holds at every position of σ up to and including position p i at which the system switches the automaton and Φ holds at position p i. In addition, since Ψ i satisfies the progress property v T 1 g i where g i ϕ g i, Ψ i ensures that eventually the system reaches the state v j T 1 j where i ϕ g j. According to the receding horizon scheme, the system switch an automaton at this state, i.e., v j is the state of the system at position p i of σ. Since v j T 1 j and v j satisfies Φ, σ satisfies the formula to the left of in (7). Using the previous (7)

6 argument, we get that Ψ j ensures that the safety property ϕ s holds at every position of σ starting from position p i up to and including position p j at which the system switches the automaton and Φ holds at position p j. Keep iterating this proof, we get that ϕ s holds at every position of σ and due to the finiteness of the set 1,..., p and its partial order structure, eventually the automaton p is executed which ensures that σ satisfies the progress property ϕ g. Remar 4: The propositional formula Φ (which can be viewed as an invariant of the system) adds a constraint on the initial state of the system assumed by each of the automata so that Ψ i is realizable. One way to determine Φ is to start with Φ True and chec the realizability of the resulting Ψ i. If for any i 1,..., p, Ψ i is realizable, we are done. Otherwise, the synthesis process provides the initial state of the system for which there exists a set of moves of the environment such that the system cannot satisfy Ψ i. This information provides guidelines for constructing Φ. VI. EXAMPLE We consider a point-mass omnidirectional vehicle navigating a straight road while avoiding obstacles and obeying certain traffic laws. It was shown in [17] that the nondimensional equations of motion of the vehicle are given by ẍ ÿ θ ẋ ẏ 2mL 2 J θ q x q y, (8) q θ with the following constraints on the control efforts: t, q 2 x t q2 y t 3 q θ t 2 2 and q θ t 3. (9) Conservatively, we can set q x t 0.5, q y t 0.5 and q θ t 1 so that the constraints (9) are decoupled. In this section, we are only interested in the translational (x and y) components of the vehicle state. Discretizing the dynamics (8) with time step 0.1, we obtain the following discrete-time linear time-invariant state space model z t 1 v z t z t v z t q z (10) where z represents either x or y and v z represents the rate of change in z. Let C z be the domain of the vehicle state projected onto the z, v z coordinates. We restrict the domain C z to zmin, zmax 1, 1 and partition C z as C z i zmin 1,...,zmax C z,i (11) where C z,i i 1,i 1, 1 as shown in Figure 2. Throughout the section, we call this partition the original partition of the domain C z. We consider the road with 2 lanes, each of width 1, so we set ymin 0 and ymax 2. Since the vehicle dynamics are translationally invariant, without loss of generality we set xmin 0 and xmax L where L is the length of the road. For each i 1,..., L and j 1, 2, we define a Boolean variable O i,j that is assigned the value True if and only if an obstacle is detected at some position Fig. 2. The original partition of the domain C z x o,y o i 1,i j 1,j. The state of the system is therefore a tuple x, v x, y, v y,o 1,1,O 1,2,...,O L,1,O L,2 where x, v x, y, v y 0,L 1, 1 0, 2 1, 1 is the vehicle state or the controlled state and O 1,1,O 1,2,..., O L,2 0, 1 2L is the environment state. A. System Specification We assume that at the initial configuration, the vehicle is at least d obs away from any obstacle and that the vehicle starts in the right lane, with x C x,1. That is, ϕ init in (6) is defined as: for any i 1,..., L, x i d obs i d obs C x, O i,1 O i,2 y C y,1 (12) The following properties are assumed for the environment. 1) An obstacle is always detected before the vehicle gets too close to it. That is, there is a lower bound d popup on the distance from the vehicle for which obstacle is allowed to instantly pop up. An LTL formula corresponding to this assumption is a conjunction of the following formula: for all i 1,..., L and 1, 2, x i d popup j i d popup C x,j O i, O i, (13) 2) Sensing range is limited. That is, the vehicle cannot detect an obstacle that is away from it farther than d sr. An LTL formula corresponding to this assumption is a conjunction of the following formula: for all i 1,..., L, x C x,i j i d sr O j,1 O j,2 (14) 3) The road is not bloced. That is, for any i 1,..., L, O i,1 O i,2 (15) 4) To mae sure that the stay-in-lane requirement (see below) is achievable, we assume that an obstacle on the right lane does not disappear. That is, for any i 1,..., L, O i,1 O i,1, or equivalently, O i,1 O i,1 (16) We define ϕ e in (6) to be the conjunction of formula (13) (16). (Note that for any LTL formulas ϕ and ψ, ϕ ψ is equivalent to ϕ ψ.) Next, we define the desired safety property, ϕ s, as the conjunction of the following properties: 1) No collision. That is, for any i 1,..., L and j 1, 2, O i,j x C x,i y C y,j (17)

7 2) The vehicle stays in the right lane unless there is an obstacle blocing the lane. That is, for any i 1,..., L, O i,1 x C x,i y C y,1 (18) Finally, we define ϕ g x C x,l, i.e., we want to ensure that eventually the vehicle gets to the end of the road. B. State Space Discretization Since the dynamics and the constraints on the control efforts for the x and y components of the vehicle state are decoupled, we apply the discretization algorithm presented in Section IV for the x and y components separately for the sae of computational efficiency. 4 Since the vehicle dynamics (8) are translationally invariant, we can use similar partition for all C z,i. The discretization algorithm with horizon length N 10 and Vol min 0.1 yields a partition with 11 cells Cz,i 1,C2 z,i,...,c11 z,i for each C z,i as shown in Fig. 3. For each i zmin 1,..., zmax and j j 1,..., 11, we let z,i be the state label of cell C j z,i and let 1 z,i z,i,..., 11 z,i. A discrete state is therefore a tuple ν x,ν y,o 1,1,...,O L,2 where ν x,ν y x,i y,i is the discrete controlled state. Using MPT [4], the reachability between discrete controlled states can be determined and a controller associated with each reachable pair of them can be generated such that the resulting continuous execution implements the discrete transition between them. The specification of the resulting finite transition system can then be derived as discussed in Section IV-C. v z i 1 z i Fig. 3. The partition of each cell C z,i in the original partition of the domain C z C. Receding Horizon Formulation Based on the new partition of the vehicle state space, there are the total of 242 L discrete vehicle states and 2 2 L discrete environment states. Thus, in the worst case, the resulting automaton may have as many as 242 L 2 2 L nodes. To avoid the state explosion problem, we apply the receding horizon strategy on the variable x. The partial order structure is defined as i ν x,ν y,o 1,1,..., O L,2 ν x x,i. It can be easily shown that for any i j, i ϕ g j since to get to the cell C x,l starting from C x,i, the system needs to visit cell C x,j for any j i. Next, we need to define an invariant Φ such that the specification (6) is realizable. Similar to z, we let represent either or. We classify each component of the discrete controlled states into one of the following sets. 4 Before performing the discretization, we partition each C z,i into C z,i C z,i where C z,i i 1,i 0, 1 and C z,i i 1,i 1, 0 to allow the possibility of enforcing other traffic laws such as disallowing reverse motion of the vehicle. (a) A set notrans defined as: for any ν z notrans, i j zmin 1,..., zmax and j 1,..., 11, ν z z,i. (b) A set right defined as: for anyν z right such that ν z z,i, there exists 1,..., 11 such that ν z z,i 1. In addition, for any j i 1 and 1,..., 11, ν z z,j. (c) A set left defined as: for any ν z left such that ν z z,i, there exists 1,..., 11 such that ν z z,i 1. In addition, for any j i 1 and 1,..., 11, ν z z,j. (d) A set same defined as: for any ν z same such that ν z z,i, there exists 1,..., 11 such that ν z z,i. In addition, for any j i and 1,..., 11, ν z z,j. (e) A set both defined as: for any ν z both such that ν z z,i, there exists, l 1,..., 11 and j i such l that ν z z,i and ν z z,j. To restrict the initial states of the system assumed by i so that Ψ i is realizable, we mae the following observations. 1) To ensure the progress property ϕ g, we need to assume that ν x notrans and ν y notrans. 2) To ensure no collision, the vehicle cannot collide with an obstacle at the initial state. 3) Suppose ν x x,i. To ensure no collision, if ν y can only transition to ν y y,1, then either O i,1 is False or O i 1,1 is False. Similarly, if ν y can only transition to ν y y,2, then either O i,2 is False or O i 1,2 is False. 4) Suppose ν x x,i such that it can only transition to ν x x,i 1. To ensure no collision, if ν y can only transition to ν y y,1, then O i 1,1 is False. Similarly, if ν y can only transition to ν y y,2, then O i 1,2 is False. Similar observation can be derived for the case where ν x x,i such that it can only transition to ν x x,i. 5) To ensure the stay-in-lane property, the vehicle cannot be in the left lane unless there is an obstacle blocing the right lane at the initial state. In addition, the vehicle is never in the state ν x,ν y x,i y,1 which can only transition to ν x,ν y x,i y,2. 6) Suppose ν x x,i and O i 1,1 is False. To ensure that the vehicle does not go to the left lane when the right lane is not bloced, it is not the case that ν y y,1 which can only transition to ν y C y,2. In addition, it is not the case that ν x can only transition to ν x C x,i 1 and ν y y,2 which can only transition to ν y y,2. We thus define Φ to be the conjunction of all the above statements which can be expressed in LTL. For example, the corresponding LTL formula for observation 3 is a conjunction of the following formulas: y Y same C y,1 y Y left C y,2 x C x,i O i,1 O i 1,1 y Y right C y,1 y Y same C y,2 x C x,i O i,2 O i 1,2 where Y same j i,j s.t. Cj y,i same y,i, Y left j Cj y,i left y,i and Y right j i,j s.t. Cj y,i right y,i. i,j s.t. With d popup 1 and the horizon length 2 (i.e. g i i 2), the specification (7) is realizable. In addition, if we let d obs be greater than 1 and restrict the initial state of the system such that ν x notrans and ν y notrans, we get that ϕ init Φ is a tautology.

8 D. Results The synthesis was performed on a Pentium 4, 3.4 GHz computer with 4 Gb of memory. The computation time was 1230 seconds. The resulting automaton contains 2845 nodes. During the synthesis process, nodes were generated. Based on the authors experience, this particular computer crashes when approximately nodes are generated. Thus, this problem with horizon length 2 is as large as what the computer can handle. This means that without the receding horizon strategy, problems with the road of length greater than 3 cannot be solved. A simulation result with the road length of 30 is shown in Fig. 4. The polygons drawn in red are the obstacles. Notice that when there is no obstacle blocing the lane, the vehicle tries to stay as close to the lane boundary (y 1) as possible. This is expected since to be able to avoid a pop up obstacle, due to the constraint on the admissible control inputs, the vehicle needs to stay close to the lane boundary to be able to change lane. To force the vehicle to stay close to the center of the lane, we need a finer partition of the road and extra LTL formula to ensure this property needs to be added to the system specification. Fig. 4. y x Simulation result. The polygons are the obstacles. VII. CONCLUSIONS AND FUTURE WORK This paper illustrated how off-the-shelf tools from computer science and control can be integrated to allow automatic synthesis of complex dynamical systems which are guaranteed, by construction, to satisfy the desired properties expressed in temporal logic even in the presence of adversary. A receding horizon scheme was described that addresses the main limitation of the synthesis algorithm, the state explosion problem, and allows more complex problems to be solved, assuming that the system has a certain partial order structure. The example showed that without the receding horizon scheme, the synthesis problem can be extremely computationally challenging. Although the adversarial nature of the environment has been incorporated in the synthesis, the effects of disturbances and modelling errors have not yet been studied. To increase the robustness of the system, we plan to impose more conditions on the multiparametric programming problem so that the continuous control law can be executed in a closed loop manner. In addition, the system specification needs to be modified to allow the possibility that the system may deviate from the plan due to disturbances and modelling errors. Automatic or semi-automatic computation of an invariant Φ in the receding horizon scheme based on the information provided by the synthesis tool is also of interest. This direction sounds promising since as described in the paper, Φ can be constructed by iteratively adding, until the specification is realizable, a propositional formula which describes the initial state of the system for which there exists a set of moves of the environment such that the system cannot satisfy Ψ i. VIII. ACKNOWLEDGMENTS The authors gratefully acnowledge Hadas Kress-Gazit for the inspiring discussions on the automatic synthesis of finite state automata, for her suggestions and help with the synthesis tool, and for the code for generating an input to the synthesis tool and extracting its output; K. Mani Chandy and Knot Pipatsrisawat for the discussions and comments regarding the restriction of domain for the synthesis problem; and Vanessa Jönsson for thoughtful comments on the paper. REFERENCES [1] N. Piterman, A. Pnueli, and Y. Sa ar, Synthesis of reactive(1) designs, in 7th International Conference on Verification, Model Checing and Abstract Interpretation, vol of Lecture Notes in Computer Science, pp , Springer-Verlag, [2] R. M. Murray, J. Hauser, A. Jadbabaie, M. B. Milam, N. Petit, W. B. Dunbar, and R. Franz, Online control customization via optimizationbased control, in Software-Enabled Control: Information Technology for Dynamical Systems, pp , Wiley-Interscience, [3] A. Bemporad, M. Morari, V. Dua, and E. N. Pistiopoulos, The explicit linear quadratic regulator for constrained systems, Automatica, vol. 38, no. 1, pp. 3 20, [4] M. Kvasnica, P. Grieder, and M. Baotić, Multi-Parametric Toolbox (MPT), Available at mpt/. [5] A. J. der Schaft and J. M. Schumacher, Introduction to Hybrid Dynamical Systems. London, UK: Springer-Verlag, [6] J. Hespanha, Stabilization through hybrid control, in Encyclopedia of Life Support Systems (EOLSS), vol. Control Systems, Robotics, and Automation, [7] S. Karaman, R. G. Sanfelice, and E. Frazzoli, Optimal control of mixed logical dynamical systems with linear temporal logic specifications, Decision and Control, CDC th IEEE Conference on, pp , Dec [8] R. Alur, T. A. Henzinger, G. Lafferriere, and G. J. Pappas, Discrete abstractions of hybrid systems, in Proceedings of the IEEE, pp , [9] G. Faineos, H. Kress-Gazit, and G. Pappas, Temporal logic motion planning for mobile robots, Robotics and Automation, ICRA Proceedings of the 2005 IEEE International Conference on, pp , April [10] H. Kress-Gazit, G. Faineos, and G. Pappas, Where s waldo? sensorbased temporal logic motion planning, Robotics and Automation, 2007 IEEE International Conference on, pp , April [11] D. Conner, H. Kress-Gazit, H. Choset, A. Rizzi, and G. Pappas, Valet paring without a valet, Intelligent Robots and Systems, IROS IEEE/RSJ International Conference on, pp , [12] A. Pnueli, The temporal logic of programs, in Proceedings of the 18th Annual Symposium on the Foundations of Computer Science, pp , IEEE, [13] M. Huth and M. Ryan, Logic in Computer Science: Modelling and Reasoning about Systems. Cambridge University Press, 2 ed., [14] D. Peled and T. Wile, Stutter-invariant temporal properties are expressible without the next-time operator, Inf. Process. Lett., vol. 63, no. 5, pp , [15] H. Tanner and G. J. Pappas, Simulation relations for discrete-time linear systems, in Proceedings of the 15th IFAC World Congress on Automatic Control, pp , [16] S. Prajna, Optimization-based methods for nonlinear and hybrid systems verification. PhD thesis, California Institute of Technology, [17] T. Kalmr-Nagy, R. D Andrea, and P. Ganguly, Near-optimal dynamic trajectory generation and control of an omnidirectional vehicle, Robotics and Autonomous Systems, vol. 46, no. 1, pp , 2004.