Optimal Control of Hybrid Systems Using Statistical Learning

Transcription

1 Proceedings of the 44th IEEE onference on Decision and ontrol, and the European ontrol onference Seville, Spain, December 2-, Mo. Optimal ontrol of Hbrid Sstems Using Statistical Learning. ollaro,.t. bdallah,. Tornambè, and U. Dole bstract In this paper we discuss how statistical learning methods ma be used to obtain solutions to various optimal control strategies for hbrid sstems. The results are illustrated on the problem of autonomous navigation for mobile robots in thepresenceofobstacles. I. INTRODUTION When dnamical sstems are composed of the interaction of continuous and discrete-dnamics, the become hbrid sstems. While such sstems have been around for a long time (temperature control, process control, telecommunications sstems, manufacturing sstems, are eamples of hbrid sstems), and more realisticall model various phsical phenomena, it was recentl that their stud has become a mainsta of control theor [], []. The modeling and control of such sstems relies on concepts from control theor, computer science and simulation, as well as mathematics. t one end of the spectrum are the usual continuous-time or discrete-time dnamics well understood b control theorists, while at the other end are the event-driven dnamics (utomata) favored b computer scientists. The combination of these points of views leads to various hbrid models such as hbrid automata, hbrid Petri nets, piecewise-affine models, etc. It turns out however, that man questions related to the modeling and control of hbrid sstems are hard, and more specificall man decision problems (is a sstem stabilizable?, is it reachable?) are NP-hard [2]. In earlier work b one of the authors [] and others [4], [2], it was recommended that statistical learning techniques ma offer useful, albeit approimate answers to some NP-hard problems. It was also shown in [], [4], [2] and others that some difficult optimization and robust control problems ma be approimatel solved using statistical learning methods. It is eactl this track that has led us to appl such methods in various optimal control scenarios of hbrid sstems. The remainder of this paper is organized as follows: Section II provides a quick overview of statistical learning concepts. In sectioniii we present a list of optimal control problems for hbrid sstems that might benefit from the statistical learning methods. In section IV the autonomous control problem for mobile robots, in the presence of obstacles and uncertainties is studied and statistical learning The work of.t. bdallah is partiall supported b NSF-233 U. Dole and.t. bdallah are with the Electrical and omputer Engineering Department, MS, The Universit of New Meico, lbuquerque, NM 873-, US. {umesh,chaouki}@ece.unm.edu. ollaro and. Tornambé are with DISP, Universit of Roma, Tor Vergata, Via del Politecnico 33, Roma - ITLY. {collaro,tornambe}@disp.uniroma2.it methods are proposed for its solution. Section V presents our results and our conclusions are summarized in section VI. II. STTISTIL LERNING OVERVIEW In order to introduce statistical learning concepts, let (S, ) be a measurable space and let {X n } n be a sequence of independent identicall distributed (i.i.d) observations in this space with common distribution P. We assume that this sequence is defined on a probabilit space (Ω, Σ, P). Denote b P(S) := P(S, ) the set of all probabilit measures on (S, ). Suppose P P(S) is a class of probabilit distributions such that P P. In particular, if one has no prior knowledge about P, then P = P(S). In this case, we are in the setting of distribution free learning. One of the central problems of statistical learning theor is the risk minimization problem. It is crucial in all cases of learning (standard concept or function learning, regression problems, pattern recognition, etc.). It also plas an important role in randomized (Monte arlo) algorithms for robust control problems, as has been shown b Vidasagar [4]. Given aclassf of -measurable functions f from S into [, ] (e.g., decision rules in a pattern recognition problem or performance indices in control problems), the risk functional is defined as R P (f) :=P (f) := fdp := Ef(X), f F. S The goal is to find a function f P that minimizes R P on F. Tpicall, the distribution P is unknown (or, as it occurs in man control problems, the integral of f with respect to P is too hard to compute) and the solution of the risk minimization problem is to be based on a sample (X,...,X n ) of independent observations from P. In this case, the goal of statistical learning is more modest: given ε>,δ (, ), find an estimate ˆf n F of f P, based on the data (X,...,X n ), such that sup P{R P ( ˆf n ) inf R P (f)+ε} δ. () P P f F In other words, one can write that with probabilit δ, R P ( ˆf n ) is within ε of inf f F R P (f) = R. Denote b ÑF,P L (ε; δ) the minimal number n such that for some estimate ˆf n the bound () holds, and let ÑF,P U (ε; δ) be the minimal number N such that for some sequence of estimates { ˆf n } and for all n N the bound () holds. Let us call the quantit Ñ F,P L (ε; δ) the lower sample compleit and the quantit Ñ F,P U (ε; δ) the upper sample compleit of learning. These quantities show how much data we need in //$. IEEE 4

2 order to guarantee certain accurac ε of learning with certain confidence level δ. learl, ÑF,P L (ε; δ) Ñ F,P U (ε; δ), and it is eas to show that the inequalit can be strict. The upper sample compleit is used rather frequentl in statistical learning theor and is usuall referred to simpl as the sample compleit. method of empirical risk minimization is widel used in learning theor. Namel, the unknown distribution P is replaced b the empirical measure P n, defined as P n () := n I (X k ), n k= where I () =for and I () =for. The risk functional R P is replaced b the empirical risk R Pn, defined b R Pn (f) :=P n (f) := fdp n := n f(x k ), f F. n S k= The problem is now to minimize the empirical risk R Pn on F, andweletf Pn F be a function that minimizes R Pn on F. In what follows, f Pn is used as our learning algorithm, i.e. ˆfn := f Pn. Determining the sample compleit of the empirical risk minimization method is definitel one of the central and most challenging problems of statistical learning theor (see, e.g., [], or Vidasagar [4] for the relevant discussion in the contet of robust control problems). reasonable upper bound for the sample compleit can be obtained b finding the minimal value of n for which the epected value Ef(X) is approimated uniforml over the class F b the empirical means with given accurac ε and confidence level δ. More precisel, denote N(ε, δ) := NF,P(ε, L δ) (2) { := min n : sup P { P n P F ε } } δ P P where F is the sup-norm in the space l (F) of all uniforml bounded functions on F. Let us call the quantit N(ε; δ) the (lower) sample compleit of empirical approimation on the class F. Unfortunatel, the quantit NF,P L (ε, δ) is itself unknown for most of the nontrivial eamples of function classes, and onl rather conservative upper bounds for this quantit are available. These bounds are epressed in terms of various entrop characteristics and combinatorial quantities, such as V-dimensions, which themselves are not alwas known precisel and are replaced b their upper bounds. Recent work however [2], [4], [] has shown the applicabilit of randomized algorithms in order to find bounds on the number of samples to come within ɛ from an optimum point with high confidence. It is eactl these techniques we intend to appl on the optimal control problems discussed net. n algorithm that was proved in [] provides a general solution to the optimization problems encountered so far, and gives a bound on the number of samples n that will guarantee some accurac and confidence. It is this algorithm that is used later to solve our optimization problem for a hbrid sstem. Note that the algorithm does not require the function f to be optimized to be differentiable, or even continuous. It is onl required to be measurable. III. OPTIML ONTROL OF HYRID SYSTEMS In order to illustrate the optimal control problems that might benefit from such algorithms, let us first focus on the class of hbrid sstems described in [3]. Note that the robots used in the eperimental set-up in section IV can actuall fit this model, when the coordination goal is to take each robot through a path in a minimum time while satisfing a qualit performance objective (tracking errors, collision avoidance, etc). onsider N sstems each evolving according to the state equation ż i = g i (z i,u i,t); z i (τ i )=ζ i ; i =,,N (3) The control signal u i is used to attain a desired final state belonging to the target qualit set Γ i (u i ).Ifs i (u i ) is the service time (the time during which sstem i is active), we have the following: s i (u i )=min{t ; z i (τ i + t) Γ i (u i )} (4) Meanwhile, the temporal state i which represents the time when state z i is active, evolves according to the so-called Lindle equation i =ma( i,a i )+s i (u i ) () The optimal control problem for such a hbrid sstem is then stated as follows, min u,,u N J = N L i ( i,u i ) (6) i= subject to the dnamic equations (3) and () and where the function L i ( i,u i ) is the cost associated with sstem i. s detailed in the paper [3], one of the main difficulties is that the ma function is non-differentiable eactl at the points i = a i. Recall that differentiabilit was not needed when appling algorithm??. In fact, if one defines the augmented cost J (, λ, u) = (7) N i= [L i ( i,u i )+λ i (ma( i,a i )+s i (u i ) i )] where L i and s i are differentiable functions, and λ is the costate, immediate difficulties arise in appling the first-order necessar conditions for optimalit because of the presence of ma. One has to resort then to using generalized gradients and non-smooth optimization techniques [4] in order to proceed further [3]. On the other hand, a randomized algorithm rooted in statistical learning, will provide an efficient albeit approimate solution to the minimization problem. 46

3 IV. UTONOMOUS ONTROL OF MOILE ROOTS In order to eperimentall validate our algorithms, we focus in the remainder of the paper specificall on the problem of autonomous navigation of mobile robots. It is important for the autonom of such robots that the interact with their environment. Tpicall, mobile robots collect sensor information, process them to etract features or detect obstacles, and eecute appropriate control laws for optimal, obstacle-free navigation. However, different controllers activate different behaviors such as obstacle avoidance, approach target, etc. and the need to be coordinated for a given navigation task. It is this coordination that when implemented with a hbrid automaton, helps activate an event-specific behavior. path following navigation task is considered here. It involves the subtasks of obstacle avoidance and approach target []. These two behaviors define the nodes of the hbrid automaton and there is a minimum safet distance, d O governing the discrete transitions between them. However, the real problem is how to coordinate these behaviors from the point of view of safet and optimalit. If we fuse them such that the are simultaneousl active and the sstem control vector is the sum of their control vectors, then in spite of achieving smooth performance, we undermine our goals of reducing compleit and dependenc. If, on the other hand, we use hard switches [] then the sstem might eperience chattering, which would definitel degrade the overall sstem performance. The solution to this problem lies in removing the Zeno properties of the sstem b adding nodes to the hbrid automaton in order to regularize the overall sstem. Let us suppose the sstem dnamics described in the plane b [ẋ ẏ] =F (, ). IfF O and F G represent the continuous dnamics of the obstacle avoidance and goal attraction behaviors, respectivel, then the sliding solution that will regularize the sstem is chosen such that f S F O []. This is due to the fact that the repulsive potential field under the tpical reactive obstacle avoidance behavior is orthogonal to the surface on which the behavior becomes active []. The problem now is to design a controller such that the control variable drives the mobile robot around the obstacles at the preferred safet margin and avoids high curvatures in the path which cross the phsical limits of the signals that the actuators can track. The main idea is to generate near-optimal trajectories around the obstacles to the goal, and then convert this navigation problem into a trajector tracking one. With the given tracking algorithm, the stead-state tracking error can be made ver small as the robot s steering angle φ eponentiall converges to its desired value φ d [6]. Let us suppose that the environment is characterized b the presence of three obstacles of different dimensions as seen in Figure 2. Since the robot has to approach the final target along a minimum path and safel, it has to avoid the obstacles ever time the are detected. We then need to define optimal coordination between two different behaviors: ) Goal ttraction or attractive behavior, and 2) Obstacle voidance or reactive behavior. We assume that the reactive behavior is valid onl in a circular safet region around the obstacle (represented b the outer circles in Figure ), while the attractive one is true elsewhere. ased on our assumptions: (X t,y t ) Final Destination; (X o,y o ) Initial Position; (X obi,y obi ) Obstacle Position in the plane; (Gap i,gap i Upper and lower safet Gap ounds; i = {,, } Three Obstacles. Thus, we represent each behavior (continuous dnamics) of the sstem as a specific node of a ontrol Graph, while the Guard onditions, associated with each edge, define discrete transitions between these different dnamics. Robot Trajector Fig.. Navigation Environment In order to design a Hbrid utomaton of the Mobile Robot Navigation sstem, we need to define the three different behaviors that characterize the robot during its navigation through the obstacles: ) Goal ttraction behavior: F G =[ F G F G ], where F G = Mẍ = K d ẋ K p ( X t ), F G = Mÿ = K d2 ẏ K p2 ( Y t ). 2) Obstacle voidance behavior: where 3) Sliding behavior: F O =[ F O F O ], F O = Mẍ = F O = Mÿ = ( X ) n ( Y ) n F S =[ F O F O ] The Sliding behavior is used to regularize the sstem b adding nodes to the control graph that characterized the Hbrid utomaton model. The sliding dnamics are orthogonal to the repulsive potential field (obstacle avoidance) which is true inside the Safet rea asshowninfigure2. 47

4 d SupGap d < InfGap ) different possible dimensions Figure 4. These dimensions are given b a combination of four different set of input values (one set for each circle Gap, and Gap, ). fter Goal ttraction Sliding Obstacle voidance Robot Trajector d > SupGap d InfGap Fig. 2. Hbrid utomaton V. SIMULTION The aim of the simulation is to show how statistical learning methods ma be used in order to obtain optimal solutions to autonomous navigation of a mobile robot in the presence of obstacles. Given the model described before, let s suppose the dimension of the safet areas (both the Safet Gap and the Safet Region) changing according to some environment conditions, for eample the shape of the obstacles or the plant of the area where the mobile robots are presumed to move. Since the coordination goal is to take the robot through a minimum path avoiding undesired obstacle collisions and satisfing some other qualities performance conditions (as minimizing the number of commutations of the robot), let s define the objective function of the mobile robot problem as a linear combination of two different parameters: f o = 3 D + where Distance=D is the length of the path of the robot from an initial position to a final destination, and ommutation= is the number of changes of direction requested to the robot to reach the final target safel. While the safet areas are circumferences, the idea is tring to change their dimension (b modifing the radius of each circle) for two of the three obstacles considered in the model ( and ), and to find the optimal condition for the navigation robot mobile problem (3). The optimal control problem for such a Hbrid Sstem is then stated as: Min N i= ( 3 Distance i + ommutaion i ). Let us suppose that environment conditions Robot Trajector Fig. 3. Obstacle voidance allow one to consider for each circle (that identifies the Safet Region and the Safet Gap around the obstacles and Fig. 4. Safet area combinations defining an upper bound and a lower bound for each set Gap [3, 7], Gap [2, 6], Gap [.,.], and Gap [., 4.], we define nine possible values for each interval. It turns out that there are N = n m k l =9 4 Table : Input values for Robot Mobile Navigation Gap Gap Gap Gap possible combinations that characterize a sample of independent observations (Table ). Each observation is obtained b a simulation of the robot mobile s model, according to a particular combination of the input values. The output of each simulation provides different values of the main parameters: length of the path and number of commutation, so that we can evaluate the optimal condition from a set of possible solutions. We did however appl our learning algorithm with 36 samples (obtained from a prior eperiment), and used the following values: K di =,K pi =to navigate from (X ),Y ) = (, ) to (X t,y t ) = (24, 24) while avoiding obstacles at (X o,y o ) = (6, 6),(X o,y o ) = (, 6), and (X o,y o ) = (8, 8). In order to reach our goal, we used the interaction properties between Simulink and Stateflow (fig. ). We note each combination of the input parameters Gap,Gap },andgap, Gap } define a particular solution to the safe navigation problem. The general trend is that the more the Safet Gap and the Safet rea are decreased, the larger the number of commutations, and the longer the length of the path. This is wh the Sliding behavior, used to regularize the sstem, is valid in a smaller area (see Figures fig:dessin8,fig:dessin9,fig:dessin). It turns out that most of the combinations provide a number of commutation between values four to si (see Figure 9), and a length of the path between 4 to 46 (Figure ). The robot never collides with the obstacles, and it trajectories are fairl efficient. choosing the parameters that minimize the objective 48

5 Switching conditions Robot Trajector continuous variable Fig. 8. Set [7,4,,3] s 7 6 Distance Histogram q q 2 Number of simulation 4 3 q 4 q 3 Fig.. Stateflow modeling with Simulink and Stateflow 4 4 Distances Fig. 9. length s path Distribution Robot Trajector Number of simulation Number of ommuation Histogram ommutations Fig. 6. Set [7,4,4,3] Fig.. ommutation Distribution Robot Trajector Fig. 7. Set [,4,4,2] function, we can see that the optimal conditions for the navigation of mobile robot problem are given b more than a combination (Table 2). In particular, we can see how the dimension of the Safet Region, relative to the Obstacle, has to be less than 3., otherwise the robot must avoid also this obstacle, increasing the number of commutation and the path. In the other hand, the dimension of Obstacle, has to be fied to the values Gap =6,Gap =). Suppose these conditions are true, then the robot never collides an obstacle, and it reaches the target in a minimum path (fig.3). However, b choosing (Sup Gap = 6,Inf Gap =)and (Sup Gap =4.,Inf Gap =3.) as input values, we can see how the trajector of the mobile 49

6 Table 2: Optimal combinations Gap Gap Gap Gap D Fo Fig.. Robot Trajector Optimal condition for the Robot Mobile Navigation Problem robot is no more optimal (fig.4). Fig. 2. Robot Trajector Optimal condition for the Robot Mobile Navigation Problem VI. ONLUSIONS In this paper, we have proposed to appl statistical learning algorithms to various optimal control problems for hbrid sstems. Such problems are particularl suited for randomized algorithms due to the presence of non-differentiable cost functions, along with continuous and discrete dnamics. We have focused our attention on the problem of autonomous navigation of mobile robots. In the behavior-based method above, the robot model is assumed certain. However, in practice, the wheel radi r and r 2 of the left and right wheels change, and as a result the robot model usuall has parametric uncertaint. Such an imprecise model cannot have a unified controller that ields optimal performance as described in [8]. If the uncertain parameters take values over a finite set P then we can design a famil of estimators and controllers b discretizing the parameter space [9]. The two approaches mentioned above address the problem of hbrid sstems based mobile robot navigation from two different perspectives. The first approach includes a behavior-based framework and shows how different behaviors can be integrated and implemented as one hbrid sstem. The second approach considers a single robotic task, that of parking, and uses the hbrid sstems approach to make the overall sstem robust to parametric uncertaint. possible scenario where the two approaches are combined will lead into a situation where the statistical learning approaches ma be applied. REFERENES [] ntsaklis, P. Editor, Hbrid Sstems: Theore and pplications, Special Issue of Proceedings of the IEEE, Vol. 8, No.7,. [2] londel,v., Tsitsiklis, J., ompleit of Stabilit and ontrollabilit of Elementar Hbrid Sstems, utomatica, Vol. 3, No. 3, pp , 999. [3] assandras,.g., Pepne, D.L., Wardi, Y., Optimal ontrol of a lass of Hbrid Sstems, EEE Transactions on utomatic ontrol, Vol. 46, No. 3, March., pp [4] larke, F.H., Optimization and Nonsmooth nalsis, Wile- Interscience, New York, 983. [] Egerstedt, M., ehavior ased Robotics Using Hbrid utomata in Lecture Notes in omputer Science: Hbrid Sstems III: omputation and ontrol, pp. 3-6, Pittsburgh, P, Springer-Verlag, March. [6] Egerstedt, M., Hu, X., Stotsk,., ontrol of Mobile Platforms Using a Virtual Vehicle pproach, IEEE Transactions on utomatic ontrol, Vol. 46, No., Nov., pp [7] Gare, M.R., Johnson, D.S., omputers and Intractabilit: Guide to the Theor of NP-ompleteness Freeman, New York, 979. [8] Hespanha, J. P., Morse,. S., ertaint Equivalence Implies Detectabilit, Sstems & ontrols Letters, 999, Vol. 36, pp. -3. [9] Hespanha, J. P., Liberzon, D., Morse,. S., Logic-based Switching ontrol of a Nonholonomic Sstem with Parametric Modeling Uncertaint, Sstems & ontrols Letters, Nov. 999,Vol. 38, pp [] Koltchinskii, V., bdallah,.t., riola, M., Dorato, P., Panchenko, D., Improved Sample ompleit Estimates for Statistical Learning ontrol of Uncertain Sstems, IEEE Transactions on utomatic ontrol, Vol. 46, pp ,. [] Liberzon, D., Switching in Sstems and ontrol, irkhaüser, oston, 3. [2] Tempo, R., alafiore, G., Dabbane, F., Randomized lgorithms for nalsis and ontrol of Uncertain sstems, Springer-Verlag, London,. [3] Tomlin,.J., Lgeros, J., Sastr, S.S., Game Theoretic pproach to ontroller Design for Hbrid Sstems, Proceedings of the IEEE, Vol. 88, No. 7, pp ,. [4] Vidasagar, M., Learning and Generalization: With pplications to Neural networks, Springer-Verlag, New York 2. [] Vidasagar, M., londel, V., Probablilistic Solutions to some NP-hard Matri Problems, utomatica, Vol. 37,, pp