Extremum-seeking in Singularly Perturbed Hybrid Systems

Transcription

1 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI.9/TAC , IEEE Extremum-seeking in Singularly Perturbed Hybrid Systems Ronny J. Kutadinata, William H. Moase, and Chris Manzie Abstract This paper considers the stability of a class of singularly perturbed hybrid systems with a continuous slow subsystem. This type of system encompasses many slowly-varying adaptation schemes, such as extremum-seeking, acting on hybrid systems, such as urban traffic control and switching control. Conditions are given for semi-global practical asymptotic stability of the closed-loop system when the dynamics of the reduced system are also semi-globally practically asymptotically stable. This extends previous results which require global asymptotic stability of the reduced system. Furthermore, the stability result is used to show convergence of an extremum seeking scheme acting upon a hybrid system. Index Terms Extremum-seeking, adaptive control, hybrid systems. I. INTRODUCTION Extremum-seeking ES) is a non-model based steady-state optimisation scheme for dynamical plants. An ES controller regulates the input of a dynamical plant to the value that optimises the steady-state output of the plant, without requiring knowledge of the underlying dynamics. There are many varieties of ES schemes [] [4], the simplest of which is outlined by [5]. However, all of these ES results address continuous systems. A typical ES controller acts on a slower time scale than that of the plant dynamics, creating a singularly perturbed full closed-loop system. Therefore, a singular perturbation analysis is usually used to prove stability of an ES scheme, which separates the full closed-loop system into: an approximation of the fast part of the system the plant with a constant input) called the boundary layer system ; and an approximation of the slow part of the system the ES scheme acting on the equilibrium manifold of the fast part) called the reduced system. However, the classical singular perturbation result [6] only applies to systems with continuous dynamics. It is only recently that singular perturbation results for a class of hybrid systems have been developed [7] [9]. However, these results for singularly perturbed hybrid systems only consider systems with a globally-asymptotically stable GAS) reduced system. In addition, some [7], [8] only deal with continuous boundary layer systems, i.e. the jump is caused by the slow state. On the other hand, most ES schemes can only guarantee semi-global practical asymptotic SPA) stability [2], [5], [] [2]. Also, applying ES to a hybrid system results in a singularly perturbed hybrid system with a continuous slow subsystem and a hybrid fast subsystem. Hence, the results in [7] [9] are not directly applicable and must be extended. This extension is essential to deal with many engineering problems that can naturally be modelled as hybrid systems, such as engine combustion dynamics [], [4], systems with sample-and-hold control [5], and traffic light control [6], [7]. Many of these systems have tunable parameters which affect their steady-state performance. The selection of these parameters is important, and can be posed as an optimisation problem. When the influence of these parameters on performance is not known a priori, real time optimisation can be achieved through the use of extremum-seeking, creating a singularlyperturbed hybrid system with a continuous controller and a hybrid fast-system. Manuscript received July 6, 25. R. J. Kutadinata, W. H. Moase and C. Manzie are with the Department of Mechanical Engineering, The University of Melbourne,, Victoria, Australia r.kutadinata@student.unimelb.edu.au; moasew@unimelb.edu.au; manziec@unimelb.edu.au). This paper proposes an extension of the singular perturbation results of [9] to deal with a class of ES problems. The slow subsystem is in a generalised form of ES controllers to widen the applicability of the singular perturbation result. Then, the resulting closed-loop system is shown to also be SPA stable. The importance of the singular perturbation result is highlighted when being applied to prove SPA stability of an extremum-seeker ES) acting on a hybrid plant. This is the first stability result for ES acting upon a hybrid system, thereby expanding the applicability of ES to a wider range of systems. A. Preliminaries II. STABILITY OF A CLASS OF HYBRID SYSTEMS A set-valued mapping F : R N R N is said to be outer semicontinuous at x if for all sequences x i x and y i F x i) such that y i y, then y F x). If this applies for all x R N, then it is said that F is outer semi-continuous. Furthermore, F is locally bounded if for any compact set K R N, there exists r such that F K) := x KF x) rb, where rb is a closed ball of radius r. The time domain of a solution to a hybrid system consists of two elements t, j) R Z. A compact hybrid time domain is a set S R Z which is defined as S = J j= [tj, tj+], j) with a finite sequence of times = t t t J. A set S is said to be a hybrid time domain if S [, t], {,,..., j}) is a compact hybrid time domain for all t, j) S. Furthermore, the time domain of a solution x to a hybrid system is denoted as dom x. In addition, a solution to a hybrid system x is maximal if there is no other solution x such that dom x is a proper subset of dom x and x is equal to x on dom x. A maximal solution to a hybrid system that is complete is characterized by its time domain being unbounded. A hybrid system is forward pre-complete from a compact set K if all solutions starting from K that are maximal are either complete or bounded. A solution to a hybrid system x is Zeno if it is complete and sup{t R : j Z such that t, j) dom x} <. A continuous function σ : R R is said to be of class L if it is decreasing with respect to its argument and σl) as L. Furthermore, a function β : [, a) R R is of class KL if: it is continuous with respect to both arguments; δ [, a), βδ, ) is an L function; and t, β, t) is strictly increasing with respect to its first argument with β, t) =. In addition, consider a general continuous nonlinear dynamical system with a small in each of its component) positive parameter vector ε: ẋ = ft, x, ε), ) /$. c 27 IEEE where x R Nx, ε R Nε >, and f : R RNx R Nε R Nx. The following is the definition of a semi-global practical asymptotic stability of system ). Definition : System ) is semi-globally practically asymptotically SPA) stable for a small parameter vector ε uniformly in small ε,..., ε l ), l {,..., N ε} if there exists β KL such that for any, ν) R 2 >, the following holds. There exists ε,..., ε l ) R l > such that for any ε,..., ε l ), ε ], ε l ], there exists ε l+,..., ε N ε ) R Nε l > such that for any ε l+,..., ε Nε ), ε l+], ε N ε ] and any x), the solutions of ) satisfy xt) β x), ε ε 2... ε Nε )t) + ν, t. 2) Remark : The values of ε l+,..., ε N ε ) depend on the values of ε,..., ε l ). This depicts many tuning guidelines of recent ES c) 26 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See for more information.

2 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI.9/TAC , IEEE 2 schemes [], [5], [], [2], where the parameters are chosen subsequently to ensure time-scale separation between the system and the ES controller. In addition, note the abuse of the uniformity term, where in this case the uniformity holds for a given selection of, ν). This definition is consistent with the typical SPA stability definition for recent ES results [], [5], [], [2]. B. Motivation Before presenting the results of the paper, the motivation of this work is going to be discussed in more detail. Consider a nonlinear hybrid system of the form x + Mx, u), x µu), a) ẋ F x, u), x ψu), b) y = gx), c) where x X R Nx ; u, y R; M, F : R Nx R R Nx ; µ, ψ : R R Nx are the jump and flow sets respectively; and g : R Nx R is continuous in x. Many systems can be modelled using ), such as traffic light control [6], [7], PWM control [9], quantized control systems, and reset linear control systems [8]. Furthermore, consider an arbitrary ES scheme with the following structure: κ = ε f G κκ, y, dε f t, ε s), ε s), u = G uκ, dε f t, ε s), ε s), 4a) 4b) where κ R Nκ ; ε f R and ε s R Nεs are small positive parameters; dε f t, ε s) R is a dither signal its use is going to be mentioned later); G κ : R Nκ R 2 R Nεs R Nκ ; and G u : R Nκ R R Nεs R. Many ES schemes can be represented by 4), including the family of ES schemes considered in the unifying ES framework of [9], the Nash equilibrium seeking schemes of [], and the simple scheme in [5]. The full closed-loop system resulting from ) with the ES scheme 4), stated in a new time-scale τ := ε f t, is as follows 5c) x + Mx, u), x µu), 5a) dx ε f F x, u), x ψu), 5b) dκ = Gκκ, gx), dτ, εs), εs), u = G uκ, dτ, ε s), ε s) 5d) A typical approach to prove stability of an ES scheme is to use a singular perturbation analysis by introducing a time-scale separation in system 5) by making ε f. This leads to the derivation of a reduced system, which is an approximation of the slow subsystem that has become independent of the dynamics of the fast state x. However, many ES schemes can only ensure that the reduced system is SPA stable [2], [5], [] [2], whereas the results of literature on singularly perturbed hybrid systems assume a GAS slow subsystem [7] [9]. Thus, it is clear that an extension to singular perturbation results is required to prove stability of ES schemes acting on hybrid systems. C. Singularly Perturbed Hybrid Systems In this section, the main stability results on the singularly perturbed hybrid systems 5) is presented. Firstly, some assumptions on the system 5) are presented that imposes robustness, the absence of Zeno solutions, and the existence of some average output mapping. The equivalence of these assumptions are commonly used in hybrid systems and extremum-seeking analysis [5], [8] [2], [8]. Furthermore, these properties underpins the usability of the ES scheme 4) on the hybrid system ). Then, the discussion culminates in the main stability result, Theorem. Note that in this paper, ε f is a scalar for notational compactness. However, the developed results can be readily extended to consider the case where ε f is a vector. Consider the full-closed loop system 5). In order to use results on robustness of hybrid systems [8], [9], [8], the set-valued mappings have to satisfy the following properties obtained from the so-called basic assumptions of hybrid systems [8, Assumption 6.5]. Assumption : The following hold: The set-valued mappings µ and ψ are outer semi-continuous and locally bounded. For each u R, µu) X and ψu) X are nonempty. The set-valued mapping F is outer semi-continuous and locally bounded. For each x, u) ψu) R, the set-valued mapping F x, u) is nonempty and convex. The set-valued mapping M is outer semi-continuous and locally bounded. For each x, u) µu) R, the set-valued mapping Mx, u) is nonempty and Mx, u) µu) ψu). ) Time-scale properties of the system: The stability of 5) relies upon two key assumptions: the system 5) admits a reduced system via singular perturbation and averaging analysis similar to those in [9]; and the reduced system admitted is SPA stable. Firstly, the boundary layer system is obtained by setting ε f = as follows stated in t time-scale) x + bl Mx bl, u bl ), x bl µu bl ), 6a) ẋ bl F x bl, u bl ), x bl ψu bl ), 6b) κ bl =, κ bl R Nκ, 6c) u bl = G uκ bl, d, ε s), ε s). 6d) Secondly, for practical purposes, the boundary layer system 6), and consequently the system ), satisfies the following property. Assumption 2: No complete solution x bl to the boundary layer system 6) is Zeno. The result in this paper relies on the fact that the trajectories of the boundary layer system can be averaged with respect to the dynamics of the slow states. Therefore, it is assumed that a well-defined average exists in order to derive the reduced system, which utilises a similar averaging approach in [8], [9], as follows. Assumption : There exists a map J : R R such that for each compact set K R Nκ, there exists σ K L, such that for: each L >, each κ K, each solution x bl to the boundary layer system 6) with dom x bl [L, ] Z ), each Lebesgue measurable function g bl : [, L] R that satisfies g bl s) = gx bl s, j)) for each s, j) dom x bl, each ε s > and each τ, the following holds with u = G uκ, dτ, ε s), ε s), L G κκ, g bl s), dτ, ε s), ε s) G κκ, Ju), dτ, ε s), ε s) ds Lσ KL). 7) Remark 2: Although not stated explicitly, Assumption implies that the fast state is also well-behaved such that when evaluating the average of the vector field G κ, the output y can be approximated by some mapping J ). Example : For many ES schemes, 7) can be simplified due to the affinity of G κ with respect to y. For instance, consider the ES scheme 4), as shown in Figure, with κ ξ, ū), [ G κ Aξ + By k sinε f t φ)cξ G u ū + a sinε f t), ], 8a) 8b) c) 26 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See for more information.

3 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI.9/TAC , IEEE a sinωt) u kω x + Mx, u) ẋ F x, u) y = gx) sinωt φ) C y ξ ωb ωa Fig. : An ES scheme with a linear filter. where A R N ξ N ξ, B R N ξ, C R N ξ, ε s a, k), and d, ε s) a sin ). By defining σ K ) := σ K )/ B, if the following holds L L then 7) also holds since σ KL) = σ KL) B B L L g bl s) Ju) ds L L Bg bl s) Ju)) ds = L [ ] Aξ + Bgbl s) L k sinτ φ)cκ g bl s) Ju) ds σ KL), 9) [ ] Aξ + BJu) k sinτ φ)cκ ds. ) Thus, 7) can be replaced by 9). The inequality 9) more clearly shows that to satisfy Assumption, the output of the system y = gx) has to, on average, converge to a quasi stead-state map J ). Therefore, it is easier to verify 9) in practice. In essence, when deriving the reduced system, this paper considers a combination of the use of averaged dynamics and the use of an appropriate mapping to replace the fast state with some measure of its quasi steady-state. Therefore, under Assumption, the term gx) in 5c) can be replaced by Ju) to produce the following reduced system dκ r = Gκκr, JGuκr, dτ, εs), εs)), dτ, εs), εs). ) Assumption 4: The system ) is SPA stable uniformly in small ε s. Remark : Under Assumption 4, the SPA stability of an arbitrary ES scheme acting on a static map Ju) is guaranteed. Given a specific ES scheme, this assumption can be justified by the stability proof of the considered ES scheme, as demonstrated in Section III for the ES scheme 8) given in Figure. In addition, if the equilibrium of ) is not at the origin, it can always be translated such that the origin becomes the equilibrium. 2) SPA stability of the closed-loop hybrid system: This section essentially provides a guarantee that the slow state of 5) has a similar behaviour to the solution of ). However, the time domains of the solutions of 5) and ) are different since the former is a hybrid system while the latter is a continuous system. Therefore, define a mapping κ c that satisfies graphκ c) = τ, κτ, j)). 2) τ,j) domx,κ) Hence, κ c is now a continuous time signal, with values identical to those of κ, and will be used in the result statement. The stability proof of the system uses the following results on closeness of κ c to the solution of the reduced system κ r. Lemma Theorem of [9]): Consider system 5) under Assumptions. If the reduced system ) is forward pre-complete from a compact set K R Nκ, then for any T, ρ) R 2 >, there exists ε f such that for any ε f, ε f ], and each solution of 5c) with κ, ) K, there exists some solution κ r to the reduced system ) with κ r) K such that for each τ T, there exists s such that τ s ρ and κ cτ) κ rs) ρ. Thus, the stability of the full system can now be stated. Theorem : Consider system 5) under Assumptions 4. There exists β KL such that for any, ν) that satisfy ν >, there exists ε s > such that for any ε s, ε s], there exists ε f > such that for any ε f, ε f ], and any maximal solutions of 5c) with κ, ), the following holds κ cτ) β κ c), ε sτ) + ν, τ. Proof: The proof is as follows. ) Choose, ν) that satisfy ν >. 2) From Assumption 4, use, ν/) to generate β, ) and ε s from the SPA stability definition of ). ) Let T be large enough that β, ε sτ) ν/ for all τ T. 4) Let the pair of small positive constants η, ρ ) be selected such that ρ, ρ ], the following hold. a) κ r) and τ ρ, κ r) κ rτ) η. b) The following holds τ sup δ, ] [ β δ + η + ρ, ε s max { τ ρ }) βδ, ε sτ) ν/ ρ. ) The existence of the pair η, ρ ) is guaranteed by the continuity of κ r in Step 4a) and the definition of a KL function in Step 4b). To see the existence of η, ρ ) in Step 4b, consider the following. By letting η = ρ =, ) simplifies to ν/ and ) holds. Then, as η, ρ) continually increases, the LHS and RHS of ) increases and decreases respectively. Note that given a δ, ], there exist many possible pairs of η, ρ) that can be selected such that the LHS and RHS of ) are equal. The existence of η, ρ ) is guaranteed by noting that this exercise can be carried out for all δ, ]. 5) Use 2T, ρ) to generate ε f from Lemma, such that: a) For all τ 2T, s such that τ s ρ, κ cτ) κ rs ) ρ. Also from Lemma, s 2 such that s 2 ρ, κ c) κ rs 2) ρ. It follows that, κ cτ) κ rs ) + ρ β κ r), ε ss ) + ρ + ν β κ rs 2) + η, ε ss ) + ρ + ν β κ c) + η + ρ, ε s max by Lemma ) by Assumption 4) { τ ρ ] from Step 4a) }) + ρ + ν β κ c), ε sτ) + 2ν from Step 4b). b) More specifically, by using Step, it follows that κ cτ) β, ε sτ) + 2ν ν, τ [T, 2T ] 6) Noting that ν, repeat Step 5 by re-applying Lemma for τ [T, T ]. As in Step 5b, it follows that τ [2T, T ] κ cτ) β, ε sτ T )) + 2ν ν. 4) 7) Repeat Step 6 while advancing the considered time period by T. This shows that κ cτ) ν for all τ [T, 4T ]. Continually c) 26 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See for more information.

4 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI.9/TAC , IEEE 4 repeating this process and combining the resulting group of inequalities with those in Steps 5b and 6 leads to: κ cτ) ν, τ [T, ). 8) The conclusion of Theorem is obtained by noting that the last inequality in Step 5a holds for all τ < T and the inequality of Step 7 holds for all τ T. where i is the imaginary unit and Ĵ nū r) = 2π π π Jū r + a sinτ))e inτ, 7) the following approximate equilibrium solution of ξ r can be obtained: hū r, τ) = n ZinI A) BĴnūr)einτ, 8) Remark 4: The low convergence speed that is caused by ε f and ε s being small is typical of an ES stability result. Since the objective of ES is to optimise the plant s behaviour in the steady-state, and not its transients, the slow convergence is generally acceptable. III. STABILITY OF AN EXTREMUM-SEEKING SCHEME ON A CLASS OF HYBRID SYSTEMS Theorem is now used to prove stability of an ES scheme acting on a hybrid system ) leading to the second main result, Theorem 2. Essentially, this section demonstrates that the ES scheme given by 8) satisfies Assumption 4. The proposed ES scheme is equipped with a second-order linear filter, through which the output y is processed. An example of a second-order linear filter is a band-pass filter, which assists in extracting the sinusoidal component of the output signal corresponding to the effect of the dither. Assumption 5: Ju) from Assumption is differentiable and there exists a unique minimum u such that the following hold: J u ) = ; and ζ > such that J u + δ)/δ > ζ for all δ. Remark 5: Assumption 5 does not require the exact knowledge of the shape of Ju), and is a common property in ES stability results [5], [], [2], [2]. The control objective is to regulate the input u to the extremum u. In order to do so, the dither is used to perturb the input u such that the local gradient of Ju) can be estimated. Then, the gradient estimate is used in a gradient descent approach that drives u to converge to the vicinity of u. Remark 6: The role of φ is to compensate for any phase shift at the dither frequency, ε f ) caused by the filter in the first component of G κ in 8a). For example, let Hs/ε f ) = CsI/ε f A) B be the transfer function for the filter, where I is the identity matrix. Then, the dither experiences a phase shift, Hi), due to the filter. This can be compensated for by letting φ = Hi), thereby bringing the demodulation signal in-phase with the oscillatory component of the plant output attributed to the dither. φ can be chosen to also compensate for the plant dynamics, if they are known [2]. A. Reduced system Before stating the stability of the full closed-loop system, it is necessary to prove that the reduced system satisfies Assumption 4. The reduced system is obtained, as in Section II-C, by replacing y in 8) with Ju), which leads to dξ r = Aξr + BJūr + a sinτ)), 5a) dū r = k sinτ φ) Cξr. 5b) Note that Jū r +a sinτ)) is 2π-periodic in τ and satisfies Dirichlet conditions by Assumption 5). Then, using Fourier series representation while considering ū r as a constant) Jū r + a sinτ)) = n Z Ĵ nū r)e inτ, 6) where I is the identity matrix. Define the error coordinates ũ r = ū r u, ξ r = ξ r hũ r+u, τ). Then, the reduced system in the error coordinate is as follows: d ξ r = A ξ r hũ + u, τ) dũ r ū r, 9a) dũ r = k sinτ φ)c ξ r + hũ + u, τ)). 9b) Finally, recall the definition Hs) = CsI A) B and introduce the following assumption. Assumption 6: Re { e iφ Hi) } > and A is Hurwitz. Remark 7: Assumption 6 can be satisfied by appropriate design. Finally, the SPA stability of the reduced system can be stated. Lemma 2: Under Assumptions 5 6, 9) is SPA stable uniformly in small a, k). Proof: Refer to the Appendix B. Stability of the full closed-loop system Now consider the full closed-loop system in the error coordinate ũ = ū u, ξ = ξ hũ + u, ωt), and the τ time-scale x + Mx, ū + a sinτ)), x µū + a sinτ)), 2a) ω dx F x, ū + a sinτ)), x ψū + a sinτ)), 2b) d ξ = A ξ + Bgx) Jũ + u + a sinτ)), 2c) dũ = k sinτ φ) C ξ + hũ + u, τ)). 2d) As before, define a mapping κ c that satisfies: )) graph κ c) = τ, ξτ, j), ũτ, j). 2) τ,j) domx, ξ,ũ) The stability of the complete closed-loop system 2) follows directly from Lemma 2 and Theorem. Theorem 2: Consider system 2) under Assumptions, 5 and 6, for any, ν) that satisfy ν >, there exist a, k ) R 2 > such that for all a, k), a ], k ], there exists ω R > such that for any ω, ω ], any maximal solutions of 2c) 2d) with ξ, ), ũ, ), the following holds κ cτ) β κ c), kaτ) + ν, τ. Remark 8: Theorem accommodates a wide range of ES schemes, such as: the family of ES schemes considered in the unifying ES framework of [9]; the Nash equilibrium seeking schemes of []; and the simple scheme in [5], leading to subsequent restatement of Theorem 2. IV. SIMULATION EXAMPLE Consider the problem of maximising the steady-state trade-off between yield and productivity in a bioprocess reaction [22] controlled by a PWM valve. For simplicity, the dynamics of the enzymatic reaction are normalised by using the maximum reaction rate v m and c) 26 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See for more information.

5 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI.9/TAC , IEEE 5 the liquid volume V, which is kept constant. The resulting chemical model becomes. x ẋ = f x, θ) = + θqc x ), K m + x x ẋ 2 = f 2x, x 2, θ) = θq x 2, K m + x 22a) 22b) where x is the reactant s concentration, x 2 is the product s concentration, K m =. is the half-saturation constant, c = is the reactant s concentration in the feed stream, q is the maximum volumetric flow rate of both the reactant feed stream and the reaction medium withdrawal stream feed and withdraw flow rate is equal so that V is constant), and θ is the state/position of the PWM solenoid valve controlling the flow rates. The dynamics of the solenoid valve are assumed to be extremely fast so that there are effectively only two positions, θ {, } fully closed or open). PWM is used to achieve flow rates that are, in an average sense, in the range [, q]. Also, q and time are normalised by v mv and vm respectively, and q =.6 v mv. The output considers a linear combination of the productivity the rate of harvesting the product = θq x 2) and the yield the amount of product made per unit of substrate fed = x 2/c) with a weighting factor λ =.5 as follows y = λθq x 2 + λ)c x 2. 2) It is assumed that x 2 is measurable such that y can be computed. The control objective is to determine the duty cycle u, ) of the PWM valve that maximises y in the steady-state on average) given a cycle time C =. vm. The dynamics 22) can be written in the form a) b) by introducing a new state s [, ] as a timer, with x x, x 2, θ, s) and Mx) x, x 2, θ, ), µu) {[, ] 2 u} {[, ] 2 u}, F x) f x, θ), f 2x, x 2, θ),, /C), 24a) 24b) 24c) ψu) {[, ] 2 [, u)} {[, ] 2 [, u)}. 24d) In this case, the continuity of the set-valued mappings M, µ, F, ψ with respect to their own input are sufficient to ensure outer semicontinuity. Moreover, for any x, x 2, θ, s, u) [, ] 2 {, } [, ], ), they are never empty and are locally bounded since each element of each set-valued mapping is bounded. In addition, they do not exhibit any Zeno solution. Thus, the set-valued mappings satisfy Assumptions and 2. Finally, the mapping Ju) from Assumption can be represented by the steady-state mapping of the continuous case in [22], which satisfies Assumption 5. The ES scheme 8) uses a band-pass filter as follows, [ ].2 A = 5 5, B = [.2 5 ], C = [ ]. 25) The following ES parameters shown in Figure ) are used: a =.5, k =.8, ω =. rad/vm, and φ = 2. In order to validate the convergence of the scheme to the optimum, u is estimated by converting the optimum value q =. of the continuous case in [22] as follows: u = q /q =.9. Similarly, the optimal value of the output is expected, on average, to be y =.8. For the sake of presentation, the output is averaged to smooth out the effects of the PWM valve. Figure 2 shows the trajectories of κ c, u, and y av, which is the output y subjected to a moving average filter with T av = C vm. As predicted by Theorem 2, κ c converges to the vicinity of the origin. In addition, it can be seen that the input of the closed-loop system converges to the vicinity of the optimum u, and consequently y av converges to a vicinity of y. κc u yav t vm t vm t vm Fig. 2: The evolution of κ c, u and y av in time. The dashed lines indicate the expected optima u and y. V. CONCLUSIONS Conditions for semi-global practical asymptotic stability of a class of singularly perturbed hybrid systems with a hybrid fast subsystem, a continuous and SPA stable reduced system rather than GA stable) have been given. The stability result encompasses hybrid systems equipped with an arbitrary ES scheme. In particular, this result is used to prove the stability of an ES scheme with a second-order linear filter acting upon a hybrid system. This generalises ES stability results to consider a broader class of systems. Further opportunities for generalisation exist through consideration of other filters and ES approaches. APPENDIX PROOF OF LEMMA 2 Using the approximate equilibrium solution of ξ r, the error coordinate ξ r, ũ r), and defining for algebraic simplicity) Jσ) := Jσ + u ), hσ, t) := hσ + u, t), Q nσ) := Ĵnσ + u ), and σ := dσ/ instead of using the t time-scale), the reduced system written in the error coordinate is: ξ r = A ξ r hũ, τ) ũ r, ū r 26a) ũ r = k sinτ φ)c ξ r k sinτ φ)c hũ, τ), 26b) From 26), it can be observed that by making k small, the dynamics of ũ r is at a slower time-scale than those of ξ r. Hence, using the result in [2], the stability of 26) can be studied by analysing the stability of its boundary layer system and its averaged system. ) Boundary layer system: The boundary layer system is obtained by setting k =, such that: ξ bl = A ξ bl, ũ bl =, ũ bl = ũ). 27) It is easy to see that the origin of the boundary layer system is exponentially stable c) 26 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See for more information.

6 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI.9/TAC , IEEE 6 2) Averaged system: To analyse the averaged system, consider the average dynamics of 26b) when ξ r =. Using the definition of Hs) = CsI A) B, consider the second term in 26b): sinτ φ)c hũ, τ) = Hin)Q nũ) e i[n+)τ φ] e i[n )τ+φ]). 28) 2i n Z It follows that the average is non-zero only when n = ±. In that case, ) e iφ H i)q ũ) e iφ Hi)Q ũ) 2i { } = Im e iφ Hi)Q ũ) =: F avũ) 29) Thus, the averaged system is simply Expanding 29), { } Im e iφ Hi)Q ũ av) = Θ 2π ũ av = kf avũ av). ) π Φ 2π π π Jũ av + a sinτ)) cosτ) π Jũ av + a sinτ)) sinτ), ) where Θ := Im { e iφ Hi) } and Φ := Re { e iφ Hi) }. Using Taylor series expansion: Jũ av + a sinτ)) Jũ av) + a sinτ) J ũ av) + Oa 2 ), we can observe that the average of the first term of ) is Oa 2 ), and the average of the second term is a J ũ 2 av)+oa 2 ).Then, using small a, the averaged system would approximate a simple gradient system. Next, the following stability result of the averaged system is required for the stability of the reduced system. Lemma : Under Assumptions 5 and 6, there exist β av KL such that for any, ν) R 2 >, there exist a R > such that for all a, k), a ] R >, the solutions of ) with the initial condition ũ av), will satisfy ũ avτ) β av ũ av), kaτ) + ν, τ. Proof: The proof follows a similar approach to [, Lemma ]. Using the Lyapunov function V = 2 ũ2 av, the stability of the averaged system can be proven. First define Γũ av) := } {e a Im iφ Hi)Q ũ) + Φ 2 J ũ av). 2) Then, ) can be expressed as ũ av = kaφ J ũ av) + kaγũ av). ) 2 Therefore, by using Assumption 5, ka V = Φ 2 J ũ av)ũ av + Γũ av)ũ av = Φ J ũ av) ũ av 2 + Γũ av)ũ av 2 ũ av Φ 2 ζ ũav 2 + Γũ av) ũ av. 4) It is desired that V be negative, which is true noting that Φ > by Assumption 6) and when the following holds: Φ 2 ζ ũav 2 + Γũ av) ũ av. 5) The inequality 5) can be solved for ũ av which yields ũ av 2 Γ Φζ. 6) Lemma follows after noting that Γ is of Oa) and hence, decreasing a makes the region to which ũ av converges arbitrarily small. Then, Lemma 2 now follows from Lemma and [2, Theorem ]. REFERENCES [] M. Krstić and H. H. Wang, Stability of extremum seeking feedback for general nonlinear dynamic systems, Automatica, vol. 6, no. 4, pp , Apr. 2. [2] S.-J. Liu and M. Krstić, Stochastic averaging in continuous time and its applications to extremum seeking, IEEE Transactions on Automatic Control, vol. 55, no., pp , 2. [] W. Moase, C. Manzie, and M. Brear, Newton-like extremum-seeking for the control of thermoacoustic instability, IEEE Transactions on Automatic Control, vol. 55, no. 9, pp , Sept. 2. [4] M. Guay and D. Dochain, A minmax extremum-seeking controller design technique, IEEE, vol. 59, no. 7, pp , May 24. [5] Y. Tan, D. Nešić, and I. Mareels, On non-local stability properties of extremum seeking control, Automatica, vol. 42, no. 6, pp , 26. [6] H. Khalil, Nonlinear Systems, rd ed. Prentice Hall, 22. [7] R. Sanfelice and A. Teel, On singular perturbations due to fast actuators in hybrid control systems, Automatica, vol. 47, pp , 2. [8] W. Wang, A. Teel, and D. Nešić, Analysis for a class of singularly perturbed hybrid systems via averaging, Automatica, vol. 48, no. 6, pp , Jun. 22. [9], Averaging in singularly perturbed hybrid systems with hybrid boundary layer systems, in Proceedings of the 5st IEEE Conference on Decision and Control, 22, pp [] K. B. Ariyur and M. Krstić, Real time optimization by extremum seeking control. Hoboken, NJ : Wiley Interscience, 2. [] R. J. Kutadinata, W. Moase, and C. Manzie, Dither re-use in Nash equilibrium seeking, IEEE, vol. 6, no. 5, pp. 4 48, May 25. [2] W. Moase, Y. Tan, D. Nešić, and C. Manzie, Non-local stability of a multi-variable extremum-seeking scheme, in Proceedings of the 2 Australian Control Conference AuCC), Nov. 2, pp [] A. Mohammadi, C. Manzie, and D. Nešić, Extremum seeking methods for online optimisation of spark advance in alternative fueled engines, in Proceedings of the IFAC Workshop on Engine and Powertrain Control, Simulation and Modelling, vol., no., 22, pp. 8. [4] J. Sharafi, S. Hager, W. Moase, P. Dennis, M. Brear, and C. Manzie, Fast extremum seeking for optimization of brake specific fuel consumption, in Proceedings of the 5rd IEEE Conference on Decision and Control, 24, pp [5] S. Khong, D. Nešić, Y. Tan, and C. Manzie, Unified frameworks for sampled-data extremum seeking control: Global optimisation and multiunit systems, Automatica, vol. 49, no. 9, pp , Sep. 2. [6] R. J. Kutadinata, W. Moase, C. Manzie, L. Zhang, and T. Garoni, Extremum-seeking for adaptation of urban traffic signal control, in Proceedings of the 9th IFAC World Congress, 24. [7], Enhancing the performance of existing urban traffic light control through extremum-seeking, Transportation Research Part C: Emerging Technologies, vol. 62, pp. 2, 26. [8] R. Goebel, R. Sanfelice, and A. Teel, Hybrid Dynamical Systems : Modeling, Stability, and Robustness. Princeton University Press, 22. [9] D. Nešić, Y. Tan, C. Manzie, A. Mohammadi, and W. Moase, A unifying framework for analysis and design of extremum seeking controllers, in Proceedings of the 24th IEEE Chinese Control and Decision Conference CCDC), 22, pp [2] W. H. Moase and C. Manzie, Semi-global stability analysis of observerbased extremum-seeking for Hammerstein plants, IEEE Transactions on Automatic Control, vol. 57, no. 7, pp , 22. [2] W. Moase and C. Manzie, Fast extremum-seeking for Wiener Hammerstein plants, Automatica, vol. 48, no., pp , Oct. 22. [22] G. Bastin, D. Nesic, Y. Tan, and I. Mareels, On extremum seeking in bioprocesses with multivalued cost functions, Biotechnology Progress, vol. 25, no., pp , 29. [2] A. Teel, L. Moreau, and D. Nešić, A unified framework for input-tostate stability in systems with two time scales, IEEE Transactions on Automatic Control, vol. 48, no. 9, pp , Sep c) 26 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See for more information.