Quantitative Local Analysis of Nonlinear Systems. Ufuk Topcu. B.S. (Bogazici University, Istanbul) 2003 M.S. (University of California, Irvine) 2005

Transcription

1 Quantitative Local Analysis of Nonlinear Systems by Ufuk Topcu B.S. (Bogazici University, Istanbul) 2003 M.S. (University of California, Irvine) 2005 A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Engineering - Mechanical Engineering in the GRADUATE DIVISION of the UNIVERSITY OF CALIFORNIA, BERKELEY Committee in charge: Professor Andrew K. Packard, Chair Professor Kameshwar Poolla Professor Laurent El Ghaoui Fall 2008

2 The dissertation of Ufuk Topcu is approved. Chair Date Date Date University of California, Berkeley Fall 2008

3 Quantitative Local Analysis of Nonlinear Systems Copyright c 2008 by Ufuk Topcu

4 Abstract Quantitative Local Analysis of Nonlinear Systems by Ufuk Topcu Doctor of Philosophy in Engineering - Mechanical Engineering University of California, Berkeley Professor Andrew K. Packard, Chair This thesis investigates quantitative methods for local robustness and performance analysis of nonlinear dynamical systems with polynomial vector fields. We propose measures to quantify systems robustness against uncertainties in initial conditions (regions-ofattraction) and external disturbances (local reachability/gain analysis). S-procedure and sum-of-squares relaxations are used to translate Lyapunov-type characterizations to sumof-squares optimization problems. These problems are typically bilinear/nonconvex (due to local analysis rather than global) and their size grows rapidly with state/uncertainty space dimension. Our approach is based on exploiting system theoretic interpretations of these optimization problems to reduce their complexity. We propose a methodology incorporating simulation data in formal proof construction enabling more reliable and efficient search for robustness and performance certificates compared to the direct use of general purpose solvers. This 1

5 technique is adapted both to region-of-attraction and reachability analysis. We extend the analysis to uncertain systems by taking an intentionally simplistic and potentially conservative route, namely employing parameter-independent, rather than parameter-dependent, certificates. The conservatism is simply reduced by a branch-and-bound type refinement procedure. The main thrust of these methods is their suitability for parallel computing achieved by decomposing otherwise challenging problems into relatively tractable smaller ones. We demonstrate proposed methods on several small/medium size examples in each chapter and apply each method to a benchmark example with an uncertain short period pitch axis model of an aircraft. Additional practical issues leading to a more rigorous basis for the proposed methodology as well as promising further research topics are also addressed. We show that stability of linearized dynamics is not only necessary but also sufficient for the feasibility of the formulations in region-of-attraction analysis. Furthermore, we generalize an upper bound refinement procedure in local reachability/gain analysis which effectively generates nonpolynomial certificates from polynomial ones. Finally, broader applicability of optimizationbased tools stringently depends on the availability of scalable/hierarchial algorithms. As an initial step toward this direction, we propose a local small-gain theorem and apply to stability region analysis in the presence of unmodeled dynamics. Professor Andrew K. Packard Dissertation Committee Chair 2

6 To my parents Züleyha and Fevzi for their support and patience, and to my wife Zeynep for sharing the joy of life with me. i

7 Contents Contents ii List of Figures v List of Tables ix 1 Introduction Thesis Overview and Contributions Summary of Examples Background Semidefinite and Linear Programming Sum-of-Squares Polynomials and Sum-of-Squares Programming Generalized S-procedure and Positivstellensatz Preliminary Remarks Chapter Summary ii

8 3 Simulation-Aided Region-of-Attraction Analysis Characterization of Invariant Subsets of ROA and Bilinear SOS Problem Relaxation of the Bilinear SOS Problem Using Simulation Data Examples Critique Sanity Check: Does Linear Stability Imply Existence of SOS Certificates? Appendix Chapter Summary Local Stability Analysis for Uncertain Nonlinear Systems Setup and Motivation Computation of Robustly Invariant Sets Implementation Issues Sanity Check: Does Robust Stability Imply Existence of SOS Certificates? Examples Chapter Summary Appendix Extensions of the Robust Region-of-Attraction Analysis: Refinements and Non-affine Uncertainty Dependence 77 iii

9 5.1 Setup and Estimation of the Robust ROA of Systems with Affine Parametric Uncertainty Polynomial Parametric Uncertainty Branch-and-Bound Type Refinement in the Parameter Space Examples Chapter Summary Reachability and Local Gain Analysis for Nonlinear Dynamical Systems Upper and Lower Bounds for the Reachable Set and Local Input-Output Gains Upper Bound Refinement for Reachability and L 2 L 2 Gain Analysis Simulation-Based Relaxation for the Bilinear SOS Problem in Reachability Analysis Examples Region-of-Attraction Analysis for Systems with Unmodeled Dynamics using a Local Small-Gain Theorem Chapter Summary Conclusions 126 Bibliography 129 iv

10 List of Figures 3.1 Sets Y and B and points generated by H&R algorithm. Φ j and b j denote the j th column of Φ and j th component of b Histograms of β L before CW Opt (black bars) and β L after CW Opt (white bars) for (V ) = 2 (top), 4 (middle), and 6 (bottom) The invariant subsets of the ROA (dot: (V ) = 2, dash: (V ) = 4, and solid: (V ) = 6 (indistinguishable from the outermost curve for the limit cycle)) Invariant subset of the ROA for (E 6 ) reported in [32] (solid surface) and that computed using the sequential procedure from section (dotted surface) Invariant subset of the ROA for (E 7 ) reported in [75] (thick solid curve) and that computed using the sequential procedure from section (Ω V,γ ) (thin solid curve), and system trajectories (dash-dot curves) Histograms of β L before CW Opt (black bars) and β L after CW Opt (white bars) for (V ) = 2 (top) and 4 (bottom) A slice of the invariant subset of the ROA (solid curve) and initial conditions (with x 2 = 0 and x 4 = 0) for diverging trajectories (dots) v

11 4.1 Invariant subsets of ROA reported in [18] (black curve) and those computed solving the problem in (4.13) with (V ) = 2 (blue curve) and (V ) = 4 (green curve) along with initial conditions (red stars) for some divergent trajectories of the system corresponding to α = Invariant subsets of ROA with (V ) = 4 (green curve) and (V ) = 6 (blue curve) along with the unstable limit cycle (red curves) of the system corresponding to α = 1.0, 0.8,..., 0.8, Invariant subsets of ROA with (V ) = 2 (green) and (V ) = 4 (blue) along with initial conditions (red stars) for divergent trajectories Polytopic cover for {(ζ, ψ) R 2 : ζ [0, 1], ψ = ζ 2 } with 2 cells (red) and 4 cells (yellow). Black curve is the set {(ζ, ψ) R 2 : ζ [0, 1], ψ = ζ 2 } Curves with are for the lower bounds obtained by directly solving (5.6) with D taken as the vertices of the corresponding cell, curves with are for the lower bounds obtained by applying the sequential procedure from section 4.3 by taking sample (in the first step) as the center of the corresponding cell, and curves with (in the top figure only) and show β nc and β lp, respectively Estimates of the robust ROA: from [19] (black), using the branch-and-bound based method for (V ) = 2 (red) and (V ) = 4 (green) vi

12 5.4 Lower bounds for β with (V ) = 2 (green solid curve with marker) and (V ) = 4 (blue solid curve with marker) and β nc (red solid curve with marker) computed at the centers of the cells generated by the B&B Algorithm for the (V ) = 4 run. Dashed curves are for (computed values of) β {δ} where δ is the center of the cell with the smallest lower bound at the corresponding step of the B&B refinement procedure for (V ) = 2 (green curve with marker) and (V ) = 4 (blue curve with marker) Final partition generated by the B&B algorithm for the (V ) = 4 run Controlled short period aircraft dynamics with uncertain first order linear time-invariant dynamics (δ p := (δ 1, δ 2 )) Bounds on reachable sets due to disturbance w with w 2 2 R2 for Example 1 without delay (blue curve with dots: before refinement, green curve with : after refinement, red curve with : lower bound) Bounds on reachable sets due to disturbance w with w 2 2 R2 for Example 2 (blue curve with dots: before refinement, green curve with : after refinement, red curve with : lower bound, and black circles: failed PENBMI runs) Bounds on reachable sets due to disturbance w with w 2 2 R2 for Example 2 (blue curve with dots: before refinement, green curve with : after refinement, red curve with : lower bound, and black circles: failed PENBMI runs) vii

13 6.4 Upper bounds for (V ) = 2 (with ) and (V ) = 4 (with ) before the refinement (blue curves) and after the refinement (green curves) along with the lower bounds (red curve) Feedback interconnection of and M Controlled short period aircraft dynamics with unmodeled dynamics (δ p := (δ 1, δ 2 )) viii

14 List of Tables 3.1 Parameters used in and results of SimLF G and CW Opt algorithms Volume ratios for (E 1 )-(E 7 ) Results of SimLF G and CW Opt algorithms. Upper bounds are established by a separate run of SimLF G algorithm with N conv = The upper bound for (V ) = 4 is by a divergent trajectory whereas as the upper bound is by the infeasibility of (3.6), (3.11), and (3.12) for the given β value. Representative computation times are on 2.0 GHz desktop PC N SDP (left columns) and N decision (right columns) for different values of n and 2d Number of decision variables in (4.13) (top entry in each cell of the table) and the number of decision variables in (4.15) (bottom entry in each cell of the table) for (V ) = 2, (s 2δ ) = 2, and (s 3δ ) = Optimal values of β in the problem (4.13) with different values of µ and (V ) = 2 and ix

15 4.4 Optimal values of β in the problem (4.13) with different values of µ and (V ) = 4 and Optimal value of β in the first step, β sample, and β subopt with µ for (V ) = 2 and Computed (sub)optimal values of β with p(x) = x T x (with (V ) = 2 / (V ) = 4) x

16 List of Symbols R R n R n m Z Z + C 1 M 0 (x 0) The real numbers Real n-vectors Real n-by-m matrices The ring of integers Positive integers The set of continuously differentiable real valued functions on R n M symmetric and is positive semidefinite (for x R n, entries of x are non-negative) M 0 (x 0) M is symmetric and positive definite (for x R n, entries of x are positive) R[x] Σ[x] (π) Ω f,η The set of polynomials (of certain finite degree) in x with real coefficients The set of sum-of-squares polynomials in R[x] The the degree of π R[x] The η-sublevel set of f, i.e., for η R and f : R n R Ω f,γ := {x R n : f(x) η}, xi

17 Acknowledgements It has been a short stay at Berkeley and I have enjoyed every minute of it. I would like to take this opportunity to thank the following people. I would like to express my gratitude to Andy Packard for making my studies at Berkeley possible and being a role model with his passion for research. I want to extend heartfelt thanks to Laurent El Ghaoui for long discussions on optimization and his support and guidance. Special thanks to Kameshwar Poolla for being a great mentor and his interest in my academic progression. I also want to thank Pete Seiler for valuable discussions and for providing me a summer internship opportunity at the Honeywell Labs. Thanks to all at the BCCI for their friendship, support, and creating a pleasant working environment. Finally, I would like to acknowledge the financial support from the Air Force Office of Scientific Research under grant/contract number FA and thank the program managers Sharon Heise and Scott Wells. xii

18 Chapter 1 Introduction 1.1 Thesis Overview and Contributions The objective of this thesis is to develop quantitative robustness and performance analysis tools for nonlinear dynamical systems. Nonlinear systems possess local properties that are not global. For example, an asymptotically stable equilibrium point may not be globally attractive or input-output properties may radically vary for different ranges of disturbance levels. Therefore, we emphasize local analysis rather than global and focus on the following measures of robustness: (i) inner estimates of the region-of-attraction of an equilibrium point; (ii) outer estimates of reachable sets under bounded disturbances; (iii) upper bounds for local input-output gains. In each case, we account for modeling uncertainties and extend the applicability of proposed tools to uncertain systems. Using Lyapunov/storage function type characterizations for these robustness measures and S-procedure type relaxations, analysis questions are translated to verification of global nonnegativity of functions satisfying certain properties. Polynomial optimization (more 1

19 specifically sum-of-squares programming) provides an effective framework for this verification. Therefore, we restrict our attention to systems with polynomial vector fields and polynomial robustness certificates and obtain bilinear sum-of-squares programming problems with two challenging features: nonconvexity and rapid growth of the size with increasing state and/or uncertainty space dimension. Our approach is based on exploiting system theoretic interpretations of these problems to reduce their complexity. For example, we propose a methodology incorporating simulations in formal proof generation by constructing convex outer-bounds on the set of feasible Lyapunov/storage functions. Lyapunov/storage function candidates drawn from this outer-bound set either directly qualify as robustness certificates or can be used as initial seeds for further bilinear search with more efficient and reliable performance. Another example comes from the realization that, in the presence of parametric uncertainties, the dependence of the constraints in the resulting optimization problems on the two groups of indeterminate variables, namely states and uncertain parameters, is not the same. In order to take advantage of this difference, we choose not to reflect the dependence of robustness properties in the form of the Lyapunov/storage function candidates and employ parameter-independent certificates. The conservatism due to this choice is simply reduced by sub-partitioning the uncertainty set by a branch-and-bound type refinement procedure. A common feature of proposed techniques is that most of the computation is suitable for embarrassingly parallel computation [1]. The goal of this work is to lay out a feasible path toward computationally efficient and reliable schemes for analyzing the behavior of nonlinear systems with 15 states, 5 uncertain parameters, and cubic polynomial vector fields. Heading toward this goal, restrictions and choices (on the system description or the form of certificates) we make throughout this 2

20 thesis are motivated by the trade-off between system complexity, available computational resources, and strength of the proofs. The content and the contributions of respective chapters are outlined below. Chapter 2 presents a summary of background material focusing on the aspects needed for the development in the subsequent chapters. Mainly, this material coupled with Lyapunov-type theorems introduced in respective chapters will be used to translate system analysis questions to numerical optimization problems. A brief overview of semidefinite programming centered on distinguishing properties of affine and bilinear semidefinite programs is followed by the introduction of sum-of-squares polynomials and sum-of-squares programming. Finally, a generalization of the S-procedure, a central tool for handling set containment conditions, and its link to the Positivstellensatz are discussed. In Chapter 3, we propose a method for computing invariant subsets of the regionof-attraction for asymptotically stable equilibrium points of dynamical systems with polynomial vector fields. We use polynomial Lyapunov functions as local stability certificates whose certain sublevel sets are invariant subsets of the region-of-attraction. Similar to many local analysis problems, this is a nonconvex problem. Furthermore, its sum-of-squares relaxation leads to a bilinear optimization problem. We develop a method utilizing information from simulations for easily generating Lyapunov function candidates. For a given Lyapunov function candidate, checking its feasibility and assessing the size of the associated invariant subset are affine sum-of-squares optimization problems. Solutions to these problems provide invariant subsets of the region-of-attraction directly and/or they can further be used as seeds for local bilinear search schemes or iterative coordinate-wise affine search schemes for improved performance of these schemes. We report promising results in all these di- 3

21 rections. Finally, it is shown that, for systems with cubic polynomial vector fields, if the linearized dynamics are exponentially stable, then the bilinear sum-of-squares programming problems (formulated to estimate the region-of-attraction) are always feasible. The material presented in this chapter is partly parallel to [73, 70]. Chapter 4 is dedicated to developing a method to compute provably invariant subsets of the region-of-attraction for asymptotically stable equilibrium points of uncertain nonlinear dynamical systems. We consider polynomial dynamics with perturbations that either obey local polynomial bounds or are described by uncertain parameters multiplying polynomial terms in the vector field. This uncertainty description is motivated by both incapabilities in modeling, as well as bilinearity and dimension of the sum-of-squares programming problems whose solutions provide invariant subsets of the robust region-of-attraction. Finally, we discuss a sequential suboptimal solution technique suitable for parallel computing. The technique proposed in this chapter, coupled with the sequential implementation, is conservative. Yet, its computational complexity is lower than other methods with more elegant formulations based on parameter-dependent Lyapunov functions. Therefore, it provides a relatively feasible infrastructure for the extensions in chapter 5. More importantly, lessons learnt from the study reported in this chapter apply to other analysis (and possibly synthesis) questions with parametric uncertainty. The material presented in this chapter is partly parallel to [68, 67]. In Chapter 5, we extend the applicability of the method proposed in Chapter 4 to handle systems with non-affine uncertainty dependence and to reduce the conservatism associated with using a common Lyapunov function for an entire family of uncertain systems. Non-affine appearances of uncertain parameters in the vector field are replaced by artificial parameters and the graph of non-affine functions of uncertain parameters are covered 4

22 by bounded polytopes. Conservatism (due to parameter-independent Lyapunov functions) is simply reduced by partitioning the uncertainty set using a branch-and-bound type refinement procedure. The approach offers the following advantages: (i) The parameterdependent Lyapunov functions achieved by uncertainty-space partitioning do not require an a priori parametrization of the Lyapunov function in the uncertain parameters. (ii) It leads to optimization problems with smaller semidefiniteness constraints since uncertain parameters do not explicitly appear in the constraints. Although the size of the semidefinite programming constraints does not increase with the number of uncertain parameters, their number does and the problem becomes challenging as the number of uncertain parameters increases. (iii) A sequential implementation for computing suboptimal solutions which decouples these constraints into smaller, independent problems arises naturally. This is suitable for trivial parallel computing offering a major advantage over approaches utilizing parameter-dependent Lyapunov functions. Most of the results of this chapter are reported in [71, 72]. In Chapter 6, we analyze reachability properties and local input/output gains of systems with polynomial vector fields. Upper bounds for the reachable set and nonlinear system gains are characterized using Lyapunov/storage functions and computed solving bilinear sum-of-squares programming problems. A procedure to refine the upper bounds by transforming polynomial Lyapunov/storage functions to non-polynomial Lyapunov functions is developed. The simulation-aided analysis methodology is adapted to reachability and local gain analysis. Finally, a local small-gain theorem is proposed and applied to the robust region-of-attraction analysis for systems with unmodeled dynamics. Parts of the material presented in this chapter is reported in [69] 5

23 1.2 Summary of Examples Listed below are the examples in this thesis. For all examples, the problem data and results (mainly Lyapunov functions and corresponding multipliers) are available at 1) Simulation-aided region-of-attraction analysis Van der Pol dynamics Examples from [20, 75, 33, 32] Controlled short period aircraft dynamics Pendubot dynamics Closed-loop dynamics with nonlinear observer based controller ) Region-of-attraction analysis for uncertain nonlinear systems Uncertain Van der Pol dynamics and examples from [18, 78] Uncertain controlled short period aircraft dynamics ) Extensions of robust region-of-attraction analysis An example from [19] Uncertain controlled short period aircraft dynamics Uncertain controlled short period aircraft dynamics with first-order uncertain dynamics ) Local reachability and gain analysis Reachability analysis for an example from the literature 6.4 6

24 Reachability analysis for Pendubot dynamics 6.4 L 2 L 2 gain analysis for a system with adaptive controller 6.4 Robust region-of-attraction analysis of controlled short period aircraft dynamics with unmodeled dynamics

25 Chapter 2 Background The goal in the following chapters is to develop computational tools for estimating certain robustness and performance measures (e.g. regions-of-attraction and input-output gains) for nonlinear systems. The general strategy for computing these measures consists of three main steps: i. Characterize these measures using Lyapunov-type functions that satisfy certain conditions which can be translated to set containment (or set emptiness) questions. ii. Obtain S-procedure type sufficient conditions for the set containment constraints. iii. Search for Lyapunov-type certificates and S-procedure multipliers by solving numerical optimization (or feasibility) problems. In the following sections of this chapter, we present a summary of background material needed to carry out this plan. 8

26 2.1 Semidefinite and Linear Programming Semidefinite Programming A semidefinite program (SDP) 1 is an optimization problem with linear objective and matrix semidefiniteness constraint. Formally, for c R n and the symmetric matrix valued map F : R n R m m, a SDP can be written as We mainly deal with two types of SDPs: min c T x subject to x R n F (x) 0. Linear (affine) SDPs: For symmetric matrices F 0, F 1,..., F n R m m, the map F takes the form F (x) = F 0 + n i=1 x if i. In this case, the constraint F (x) 0 is called a linear matrix inequality (LMI). Bilinear SDPs: For symmetric matrices F 0, F i, and F ij in R m m with i = 1,..., n and j = 1,..., m, the map F takes the form n n m F (x) = F (y, z) = F 0 + y i F i + y i z j F ij. (2.1) i=1 i=1 j=1 In this case, the constraint F (x) 0 is called a bilinear matrix inequality (BMI). Remarks Although the term semidefinite program is generally used for problems with affine objective function and LMI constraints [15], we will use it here to mean optimization problems with affine objective function and general matrix inequality constraints and specify its type if needed. 1 We will use the abbreviation SDP to mean both semidefinite program and semidefinite programming. Similarly, LP will be used for linear program and linear programming. 9

27 Affine SDPs are convex optimization problems, considered to be computationally tractable with polynomial time algorithms [44, 15, 80]. There are several reliable and relatively efficient solvers for affine SDPs including SeDuMi [56], DSDP [10], and SDPT3 [65]. 2 On the other hand, bilinear SDPs are nonconvex and NP-hard in general [66]. Consequently, the state-of-the-art of the solvers for bilinear SDPs is far behind that for the linear ones. We now review several strategies to solve bilinear SDPs Solution Strategies for Problems with Bilinear Matrix Inequalities Optimization problems with BMIs provide an effective framework for many problems in controls, e.g. µ-synthesis [6] and static output feedback [34]. Although there is no general purpose efficient BMI solver, several methods have been proposed. Global optimization schemes based on the branch-and-bound algorithm [2] and generalized Benders decomposition are discussed in [30] and [11], respectively. See also [43] for a discussion of methodological, structural, and computational aspect of problem with BMI constraints. Recently PENBMI, a solver for bilinear SDPs, was introduced [39]. It is a local optimizer and its behavior (speed of convergence, quality of the local optimal point, etc.) depends on the point from which the optimization starts. On the other hand, note that although the function F in (2.1) is not affine in y and z jointly, it is affine in y when z is fixed and the constraint becomes a LMI in y and vice versa. This observation suggests a simple strategy to attack bilinear SDPs: first set z to some candidate solution, say z, and optimize over y by solving ȳ := argmin y c T y z subject to F (y, z) 0, 2 As these solvers have specific formats to describe SDPs, parsers such as YALMIP [41] and CVX [31] are extremely useful in setting up SDPs in these specific formats. 10

28 then set y to ȳ and optimize over z by solving z = argmin z c T ȳ z subject to F (ȳ, z) 0, and alternate between these two problems as long as there is satisfactory improvement in the solution. This two-way iterative search, which we call coordinate-wise affine search, is of course a local search scheme and generated candidate solutions highly depend on the initial point from which the search starts (it may not reach the optimal solution or may require a large number of iterations to tightly approximate the optimal solution) [43]. Nevertheless, it is practically attractive since it only requires an affine SDP solver and it has been widely used by controls community (for example, D K iteration in µ-synthesis [6] is based on alternating between the controller K and the D-scales). Moreover, our experience suggests that, coupled with efficient methods for generating high quality initial points (see chapter 3), coordinate-wise affine search can be efficiently used to compute suboptimal solutions for problems with BMI constraints. Consequently, we implement coordinate-wise affine search schemes throughout this thesis and provide implementation details in the corresponding chapters Linear Programming A linear program (LP) is an optimization problem with linear objective and affine constraints. Formally, for c R n, A R m n, and b R m, a linear program can be written as min c T x subject to x R n Ax b. Several optimization problems in the following chapters will have both SDP constraints 11

29 and LP constraints, namely for c R n, A R m n, b R m, symmetric matrix valued map F : R n R N SDP N SDP, min c T x subject to x R n Ax b (2.2) F (x) 0. Finally, note that constraints in (2.2) can equivalently be written as F (x) a T 1 x b 1. 0,.. a T mx b m which is another (larger) SDP constraint (here a T i and b i denote the i-th row of A and b, respectively). Therefore, the optimization in (2.2) can be solved as a SDP. Both SDP and LP have been extensively studied and there are excellent references on the topic (including but not limited to) [9, 15, 79, 28, 42]. It is worth mentioning that the state-of-the-art of the solvers for LPs are far beyond that for SDPs [80, 9, 3]. 2.2 Sum-of-Squares Polynomials and Sum-of-Squares Programming Restricting our attention to systems with polynomial vectors fields and searching for unknown functions in pre-specified finite-dimensional subspaces of polynomials, we will formulate the search for robustness and performance certificates as optimization problems with polynomial nonnegativity constraints. However, verifying the global nonnegativity of a multivariate polynomial is an hard problem [49]. On the other hand, if the polynomial can 12

30 be represented as a sum of squares of finitely many polynomials, i.e., it is a sum-of-squares (SOS) polynomial, then it trivially follows that the polynomial is globally nonnegative. An appealing property of sum-of-squares polynomials is that checking whether a polynomial is sum-of-squares can be formulated as a SDP feasibility problem. Consequently, in the following sections, the strategy for dealing with global polynomial nonnegativity constraints will be replacing nonnegativity constraints by sum-of-squares conditions. We now review useful facts about SOS polynomials. Formally, a polynomial p in x R n is said to be SOS if, for p 1,..., p M R[x] it can be decomposed in the form p(x) = M p i (x) 2. (2.3) i=1 Obviously, a SOS polynomial is globally nonnegative. Therefore, the set Σ[x] of SOS polynomials in x (of some fixed degree 3 ), defined as { Σ[x] := s R[x] : M <, p 1,..., p M R[x] such that s(x) = } M p i (x) 2, i=1 is a subset of the set of globally nonnegative polynomials. In fact, Σ[x] is a strict subset of the set of globally nonnegative polynomials except for univariate polynomials, quadratic polynomials and quartic polynomials in two variables [53]. Let p be a polynomial in x of degree m and z(x) be a vector of monomials in x up to degree min{m e Z : m/2 m e }. Then, for some symmetric matrix Q, p can be decomposed as p(x) = z(x) T Qz(x). (2.4) Based on this fact, the following theorem provides a characterization of SOS polynomials. 3 We do not specify the degree of polynomials in Σ[x] in notation unless it leads to confusion and use Σ[x] to denote the set of sum-of-squares polynomials in x R n of some fixed degree to be inferred from the context. 13

31 Theorem A polynomial p, in x R n of degree 2d, is SOS if and only if there exists Q such that p(x) = z(x) T Qz(x) where z is as defined above. A proof of Theorem can be found in [22]. Here, we highlight a few useful observations that lead to the proof. Consider that p is SOS, i.e., there exist an integer M > 0 and polynomials p 1,..., p M such that p(x) = M i=1 p i(x) 2. Then, each p i is of degree d and therefore can be represented as α T i z(x) for some vector α i. Consequently, the matrix Q in the theorem statement can be taken as Q = M i=1 α iα T i which is clearly positive semidefinite. On the other hand, if p can be represented as p(x) = z(x) T Qz(x) with Q positive semidefinite, then p(x) can be written as p(x) = M i=1 p i(x) 2, where, for i = 1,..., M, p i (x) = λ i q T i z(x), λ 1,..., λ M are eigenvalues of Q and q i s are corresponding eigenvectors. Another important observation is that for a given polynomial p, the entries of z may not be algebraically independent. Therefore, there may be multiple symmetric Q such that (2.4) holds. This is most easily demonstrated by an example. Example Let x R 2, z(x) = [ x 2 1 x 1x 2 x 2 2] T, Qp R 3 3 be a symmetric matrix, and p(x) = z(x) T Q p z(x). Then, for any λ R 0 0 λ p(x) = z(x) T Q p z(x) = z(x) T Q p z(x) + z(x) T 0 2λ 0 z(x) λ 0 0 } {{ } Q h (λ) since z(x) T Q h (λ)z(x) = 0 for all λ R which follows from the relation x 2 1 x2 2 = (x 1x 2 ) 2. This degree of freedom of varying λ without violating p(x) = z(x) T Q(λ)z(x), where Q(λ) := Q p +Q h (λ), leads to a procedure to search for positive semidefinite Q(λ) by choice of λ. The search for λ such that Q(λ) 0 is an affine SDP: Find λ R such that Q(λ) 0. In fact 14

32 the reverse implication also holds: If there is no λ such that Q(λ) 0, then p is not SOS [49]. Theorem [48, 49] The existence of a SOS decomposition of a polynomial in n variables of degree 2d can be decided by solving a feasibility SDP. A useful corollary of Theorem is that if the polynomial p contains decision variables, checking whether p is SOS for some choice of these decision variables is also a SDP. More precisely, if p is a polynomial in x parameterized by α R m, then the search for α such that p(x, α) Σ[x] is a SDP feasibility problem. If p is affine in α, then this is an affine SDP. By a SOS program (or SOS programming problem), we mean an optimization problem with linear objective and SOS constraints. If the constraints are affine (bilinear) in the decision variables, then the problem in an affine (bilinear) SOS programming problem. Finally, recall that SOS programming problems can be translated to SDPs and there are specialized software packages for this translation, namely SOSTOOLS (only for affine SOS programs) [51] and YALMIP (for both affine and bilinear SOS constraints) [41]. 2.3 Generalized S-procedure and Positivstellensatz We now discuss algebraic sufficient conditions for set containment constraints used throughout this thesis. S-procedure is widely used in robust control theory to obtain linear matrix inequality based sufficient conditions for set containment questions involving quadratic function [13, 26]: for quadratic functions q 0, q 1,..., q m of the form q i (x) = [x 1]Q i [x 1] T, for i = 0,..., m, with symmetric matrices Q i R (n+1) (n+1), does the set 15

33 containment constraint {x R n : q 1 (x) 0,..., q m (x) 0} {x R n : q 0 (x) 0} (2.5) hold? A (possibly conservative) certificate for this containment is the existence of nonnegative real numbers such that Q 0 τ 1 Q 1... τ 2 Q 2 0, which is a LMI. We now state a straightforward generalization of the S-procedure to the case where the quadratic functions are replaced by general scalar valued functions. Lemma Given scalar valued functions g 0, g 1,, g m : R n R, if there exist positive semidefinite functions s 1,, s m such that g 0 (x) then m s i (x)g i (x) 0 for all x R n, (2.6) i=1 {x R n : g 1 (x),..., g m (x) 0} {x R n : g 0 (x) 0}. (2.7) Lemma provides algebraic sufficient conditions (nonnegativity of the multipliers s i and (2.6)) for the set containment constraints (2.7). However, these algebraic conditions require verifying the global nonnegativity of certain scalar valued functions. In order to circumvent this difficulty, we now specialize Lemma to the case where g 0, g 1,, g m polynomials and replace nonnegativity conditions by SOS conditions that are suitable for numerical verification. Lemma (Generalized S-procedure). Given g 0, g 1,, g m R[x], if there exist s 1,, s m Σ[x] such that then (2.7) holds. m g 0 s i g i Σ[x], (2.8) i=1 16

34 The Positivstellensatz, a central theorem from real algebraic geometry, provides generalizations of Lemma For its statement, a few definitions are needed. Definition Given {g 1,..., g t } R[x], the multiplicative monoid generated by g j s is the set of all finite products of g j s, including 1 (i.e. the empty product). It is denoted as M(g 1,..., g t ). For completeness define M( ) := 1. Definition Given {f 1,..., f r } R[x], the cone generated by f i s is { } m P(f 1,..., f r ) := s 0 + s i b i : m Z +, s i Σ[x], b i M(f 1,..., f r ). i=1 Definition Given {h 1,..., h u } R[x], the ideal generated by h k s is { } I(h 1,..., h u ) := hk p k : p k R[x]. With these definitions, we can state the following theorem from [12, Theorem 4.2.2]: Theorem (Positivstellensatz). Given polynomials {f 1,..., f r }, {g 1,..., g t }, and {h 1,..., h u } in R[x], the following are equivalent: i. The set below is empty: x R n : f 1 (x) 0,..., f r (x) 0, g 1 (x) 0,..., g t (x) 0, h 1 (x) = 0,..., h u (x) = 0 ii. There exist polynomials f P(f 1,..., f r ), g M(g 1,..., g t ), h I(h 1,..., h u ) such that f + g 2 + h = 0. 17

35 Example We now give a proof of Lemma to demonstrate the use of the Positivstellensatz. Note that the set containment constraint (2.7) holds if and only if {x g 1 (x) 0,..., g m (x) 0, g 0 (x) 0, g 0 (x) 0} =. (2.9) Theorem 2.3.1, applied to (2.9), gives that (2.9) holds if and only if there exist s ( ) Σ[x] and k Z + such that s + s 0 ( g 0 ) + m s i g i + i=1 m s 0i ( g 0 )g i + i=1 m m m s ij g i g j + + s 0...m ( g 0 ) g i + g0 2k = 0. (2.10) i=1 j=i Setting k = 1 and all s ( ) = 0 except s 0, s 01,..., s 0m, we have a sufficient condition m g 0 s 0 + s 0j g j g 0 = 0. (2.11) j=1 i=1 Since g 0 is not identically zero, (2.8) follows from (2.11) by renaming s 0i as s i. Lemma Let g R[x] be positive definite, h R[x], γ > 0, s 1, s 2 Σ[x], l R[x] be positive definite and satisfy l(0) = 0. Suppose that [(γ g)s 1 + hs 2 + l] Σ[x] (2.12) holds. Then, it follows that {x R n : g(x) γ, x 0} {x R n : h(x) < 0} (2.13). Proof. Note that (2.13) holds if and only if {x R n : γ g(x) 0, l(x) 0, h(x) 0} = 18

36 and the Positivstellensatz, applied to the last condition, yields (2.12) after appropriate simplifications [37]. An independent proof of Lemma can be obtained without using the Positivstellensatz. Let x {x R n : g(x) γ, x 0}. Suppose that s 2 (x) = 0. Noting that g(x) γ and l(x) > 0 and substituting into (2.12) lead to a contradiction. Hence, s 2 (x) > 0 for all x Ω g,γ \{0}. Now, suppose (2.12) holds. Then, h(x)s 2 (x) l(x). By the first part, h(x) l(x)/s 2 (x). Consequently, h(x) < Affine versus Bilinear Sufficient Conditions Let A 1, A 2 : R[x] R[x] be affine maps on R[x], f 1 R[x] and f 2 R[x], and consider the constraint {x R n : A 1 (f 1 (x)) 0} {x R n : A 2 (f 2 (x)) 0}. (2.14) We will use the generalized S-procedure to handle mainly two types of questions: Question 1: Given f 1 R[x] and f 2 R[x], does (2.14) hold? Question 2: Given f 2 R[x], does there exist f 1 R[x] such that (2.14) (possibly along with other constraints on f 1 ) holds? Then, S-procedure based sufficient conditions are Sufficient condition for Question 1: Existence of s 2 Σ[x] such that A 2 (f 2 (x)) A 1 (f 1 (x))s 2 (x) Σ[x]. (2.15) Sufficient condition for Question 2: Existence of s 2 Σ[x] and f 1 R[x] such that (possibly along with other constraints on f 1 ) A 2 (f 2 (x)) A 1 (f 1 (x))s 2 (x) Σ[x]. (2.16) 19

37 The main difference between these conditions is that the constraint (2.16) is bilinear in its decision variables in f 1 and s 2 whereas the constraint (2.15) is affine in its decision variables in s 2 (f 1 is fixed in this case and does not contain any decision variables). Consequently, S-procedure based sufficient conditions lead to affine SDPs for Question 1 and bilinear SDPs for Question Preliminary Remarks Throughout this thesis, we only consider causal nonlinear input-output systems with no time delay represented by ordinary differential equations of the form ẋ(t) = f(x(t), w(t)) z(t) = h(x(t)), (2.17) where x is the state vector, w denotes the input/disturbance, and z denotes the output. Occasionally, we will drop the time variable t in the notation and use ẋ = f(x, w) z = h(x) (2.18) in short. In several places, a relationship between an algebraic condition on some real variables and input/output/state properties of a dynamical system is claimed. In nearly all of these types of statements, we use same symbol for a particular real variable in the algebraic statement as well as the corresponding signal in the dynamical system. This could be a source of confusion, so care on the reader s part is required. In the examples in the following chapters, we occasionally do not provide exact numerical values used for computations. We either state the form or an approximation of the 20

38 specific expression. Similarly, we do not provide certifying Lyapunov functions and multipliers. All missing data and results for all examples in this thesis are available at Chapter Summary We presented a summary of background material focusing on the aspects needed for the development in the subsequent chapters. Mainly, this material coupled with Lyapunov-type theorems introduced in respective chapters will be used to translate system analysis questions to numerical optimization problems. A brief overview of semidefinite programming centered on distinguishing properties of affine and bilinear semidefinite programs was followed by the introduction of sum-of-squares polynomials and sum-of-squares programming. Finally, a generalization of the S-procedure, a central tool for handling set containment conditions, and its link to the Positivstellensatz were discussed. 21

39 Chapter 3 Simulation-Aided Region-of-Attraction Analysis For a dynamical system, the region-of-attraction (ROA) of a locally asymptotically stable equilibrium point is an invariant set such that all trajectories emanating from points in this set converge to the equilibrium point. For nonlinear dynamics, research has focused on determining invariant subsets of the ROA because a closed form characterization of the exact ROA may be too complicated. This potential complication is avoided because of several reasons: (i) Usually there is no systematic (numerical) procedure for computing the exact shape of the ROA. (ii) A complicated characterization of the ROA has limited value for post-analysis purposes. Among all other methods those based on Lyapunov functions are dominant in the literature [24, 27, 75, 21, 20, 46, 58, 59, 32, 63, 62]. These methods compute a Lyapunov function as a local stability certificate and sublevel sets of this Lyapunov function, in which the function decreases along the flow, provide invariant subsets of the ROA. 22

40 Using sum-of-squares (SOS) relaxations for polynomial nonnegativity [49], it is possible to search for polynomial Lyapunov functions for systems with polynomial and/or rational dynamics using semidefinite programming [46, 58, 59, 32]. However, the SOS relaxation for the problem of computing invariant subsets of the ROA leads to nonconvex optimization problems with bilinear matrix inequality constraints (namely bilinear SOS problems). By contrast, simulating a nonlinear system of moderate size, except those governed by stiff differential equations, is computationally efficient. Therefore, extensive simulation is a tool used in real applications. Although the information from simulations is inconclusive, i.e., cannot be used to find provably invariant subsets of the ROA, it provides insight into the system behavior. For example, if, using Lyapunov arguments, a function certifies that a set P is in the ROA, then that function must be positive and decreasing on any solution trajectory initiating in P. Using a finite number of points on finitely many convergent trajectories and a linear parametrization of the Lyapunov function V, those constraints become affine, and the feasible polytope (in V -coefficient space) is a convex outer bound on the set of coefficients of valid Lyapunov functions. It is intuitive that drawing samples from this set to seed the bilinear SDP solvers may improve the performance of the solvers. In fact, if there are a large number of simulation trajectories, samples from the set often are suitable Lyapunov functions (without further optimization) themselves. Information from simulations is also used in [52] and [55] for computing approximate Lyapunov functions. Effectively, we are relaxing the bilinear problem (using a very specific system theoretic interpretation of the problem) to a linear problem, and the true feasible set is a subset of the linear problem s feasible set. By contrast, a general relaxation for bilinear problems based on replacing bilinear terms by new variables and nonconvex equality constraints by convex inequality constraints is proposed in [14]. This general relaxation increases the dimension 23

41 of decision variable space, so that the true feasible set is a low dimensional manifold in the relaxed feasible space. There may be efficient manners to correctly project solutions to the relaxed problem into appropriate solutions to the original problem, but we do not pursue this. 3.1 Characterization of Invariant Subsets of ROA and Bilinear SOS Problem Consider the autonomous nonlinear dynamical system ẋ(t) = f(x(t)), (3.1) where x(t) R n is the state vector and f : R n R n is such that f(0) = 0, i.e., the origin is an equilibrium point of (3.1), and f is locally Lipschitz. Let φ(t; x 0 ) denote the solution to (3.1) at time t with the initial condition x(0) = x 0. If the origin is asymptotically stable but not globally attractive, one often wants to know which trajectories converge to the origin as time approaches. The region-of-attraction R 0 of the origin for the system (3.1) is R 0 := { } x 0 R n : lim φ(t; x 0 ) = 0. t A modification of a similar result in [76] provides a characterization of invariant subsets of the ROA in terms of sublevel sets of appropriately chosen Lyapunov functions. Lemma Let γ R be positive. If there exists a C 1 function V : R n R such that Ω V,γ is bounded, and (3.2) V (0) = 0 and V (x) > 0 for all x R n (3.3) Ω V,γ \ {0} {x R n : V (x)f(x) < 0}, (3.4) 24

42 then for all x 0 Ω V,γ, the solution of (3.1) exists, satisfies φ(t; x 0 ) Ω V,γ for all t 0, and lim t φ(t; x 0 ) = 0, i.e., Ω V,γ is an invariant subset of R 0. In order to enlarge the computed invariant subset of the ROA by choice of V, we define a variable sized region Ω p,β, where p R[x] is a fixed positive definite convex polynomial, and maximize β while imposing the constraint Ω p,β Ω V,γ along with the constraints (3.2)-(3.4). This can be written as β (V) := max β β>0,v V subject to (3.2) (3.4), Ω p,β Ω V,γ. (3.5) Here V denotes the set of candidate Lyapunov functions over which the maximum is computed, for example all continuously differentiable functions. Remarks The objective in (3.5) is to compute a tight inner estimate of the robust ROA by choice of V. The β-sublevel sets of the fixed shape factor p are used to scalarize this objective. However, note that for two fixed functions V 1 and V 2 satisfying constraints of (3.5) with β equal to β 1 and β 2 respectively, even when 0 < β 1 < β 2 are largest positive scalars such that Ω p,β1 Ω V1 and Ω p,β2 Ω V2 hold, it is not necessarily true that Ω V1 Ω V2. In literature, mainly for quadratic Lyapunov functions, the volume of the computed sublevel sets is used as the objective function (e.g. in [20]). Our choice of using the shape factor p instead of the volume in the optimization objective is that it may be possible to choose p to reflect the intent of the analyst or utilize prior knowledge about the system (see section for the latter). The problem in (3.5) is an infinite dimensional problem. In order to make it amenable to numerical optimization (specifically SOS optimization), we restrict V to be all polynomials of some fixed degree and use SOS sufficient conditions for polynomial nonnegativity. 25

43 Using simple generalizations of the S-procedure (Lemmas and 2.3.3), we obtain sufficient conditions for set containment constraints. Specifically, let l 1 and l 2 be a positive definite polynomials (typically ɛx T x for some small real number ɛ). Then, since l 1 is radially unbounded, the constraint V l 1 Σ[x] (3.6) and V (0) = 0 are sufficient conditions for (3.2) and (3.3). By Lemma 2.3.2, if s 1 Σ[x], then [(β p)s 1 + (V γ)] Σ[x] (3.7) implies the set containment Ω p,β Ω V,γ, and by Lemma 2.3.3, if s 2, s 3 Σ[x], then [(γ V )s 2 + V fs 3 + l 2 ] Σ[x] (3.8) is a sufficient condition for (3.4). Using these sufficient conditions, a lower bound on β (V) can be defined as βb (V, S) := max β V V,β,s i S i (3.6) (3.8), subject to (3.9) V (0) = 0, and β > 0. Here, the sets V and S i are prescribed finite-dimensional subsets of polynomials. Although β B depends on these subspaces, it will not always be explicitly notated. Note that since the conditions (3.6)-(3.8) are only sufficient conditions, β B(V, S) β (V) β (C 1 ). The optimization problem in (3.9) is bilinear because of the product terms βs 1 in (3.7) and V s 2 and V fs 3 in (3.8). However, the problem has more structure than a general BMI problem. If V is fixed, the problem becomes affine in S = {s 1, s 2, s 3 } and vice versa. In section 3.2, we will construct a convex outer bound on the set of feasible V and sample from 26