A Unifying Approach to Extremum Seeking: Adaptive Schemes Based on Estimation of Derivatives

Transcription

1 49th IEEE Conference on Decision and Control December 15-17, 2010 Hilton Atlanta Hotel, Atlanta, GA, USA A Unifying Approach to Etremum Seeking: Adaptive Schemes Based on Estimation of Derivatives D. Nešić, Y. Tan, W. H. Moase and C. Manzie Abstract A unifying, prescriptive framework is presented for the design of a family of adaptive etremum seeking controllers. It is shown how etremum seeking can be achieved by combining an arbitrary continuous optimization method (such as gradient descent or continuous Newton) with an estimator for the derivatives of the unknown steady-state reference-to-output map. A tuning strategy is presented for the controller parameters that ensures non-local convergence of all trajectories to the vicinity of the etremum. It is shown that this tuning strategy leads to multiple time scales in the closed-loop dynamics, and that the slowest time scale dynamics approimate the chosen continuous optimization method. Results are given for both static and dynamic plants. For simplicity, only singleinput-single-output (SISO) plants are considered. I. INTRODUCTION Etremum seeking (ES) is an optimal control approach that deals with situations when the plant model and/or the cost function (in this case, the steady-state reference-tooutput map) to optimize are not available to the designer but the plant input and output signals are measured. Using these available signals, an etremum seeking controller dynamically searches for the optimizing inputs in real time. One of the earliest eamples of etremum seeking control can be found in a 1922 paper by Leblanc (see references in [15]). The area was quite popular in the 1950 s and 1960 s, see for eample, [19], [20], [32]. Good surveys of the literature on this topic can be found in [3], [24]. Numerous applications of etremum seeking were reported in the literature, such as biochemical reactors [11], [7], ABS control in automotive brakes [9], variable cam timing [22], electromechanical valves [21], aial compressors [30], mobile robots [33], [34], mobile sensor networks [5], [18], optical fibre amplifiers [10], the Frascati Tokamak Upgrade [8], bluff-body drag reduction [4], human eercise machines [35] and thermoacoustic instability [16]. An important contribution to this area was presented in [25]. It was shown, for a class of discrete-time nonlinear models, that ES can be implemented by combining an arbitrary nonlinear programming (NLP) optimization method with an appropriate derivative 1 estimator. Under appropriate assumptions, the closed loop was shown to ehibit semiglobal practical convergence to the etremum if a tuning This work was supported under the Australian Research Council s Discovery Projects funding scheme (Projects DP and DP ). D. Nešić and Y. Tan are with the Department of Electrical and Electronic Engineering, The University of Melbourne, VIC 3010, Australia. d.nesic@ee.unimelb.edu.au; yingt@unimelb.edu.au. W. H. Moase and C. Manzie are with the Department of Mechanical Engineering, The University of Melbourne, VIC 3010, Australia moasew@unimelb.edu.au; manziec@unimelb.edu.au. parameter in the derivative estimator was appropriately adjusted (i.e. if the so called waiting time was long enough). This framework allows the designer to combine any NLP optimization method with a proposed derivative estimator in order to achieve etremum seeking. The class of adaptive etremum seekers considered in [3], [27] and references cited therein, is very popular in the literature. However, the adaptive ES schemes in [3] can all be interpreted as variations of the gradient descent method, see [27]. The authors are unaware of a systematic design framework for this class of ES scheme that would parallel the framework proposed in [25], thereby allowing the combination of an arbitrary continuous optimization method with a derivative estimator. It is the purpose of this paper to present a unifying, prescriptive framework for the design of adaptive etremum seekers that parallels [25]. It is shown how to combine an arbitrary continuous optimization method with an appropriate method for estimating the derivatives of the unknown, steady-state reference-to-output map. The main results show how the parameters in the controller can be tuned in order to achieve semi-global practical etremum seeking. The closedloop dynamics are shown to ehibit multiple time scales. Singular perturbation and averaging techniques are used to show that the slowest time scale approimates the chosen continuous optimization method. Static and dynamic plants are both considered. For simplicity, only SISO plants are discussed, however MISO plants will be studied in future work. An eample derivative estimator is presented and, to illustrate the main result, is combined with a modified Newton optimization scheme in a basic simulation. Although, at a high level, the results in this paper are similar to [25], they are presented for a very different class of ES scheme. In [25] a series of discrete input output measurements is used to estimate the local shape of the reference-to-output map for use in a NLP algorithm. In contrast, here the optimization and gradient estimation are performed continuously and simultaneously. It is also noted that a similar framework is proposed in [17] under different assumptions. In [17], it is assumed that the reference-tooutput map is a known function of some unknown parameters, and the etremum seeker is based on a parameter estimator. Here, no knowledge of the reference-to-output map is assumed. This paper is organized as follows. The problem formula- 1 Throughout this paper, derivative is taken to mean first and higher order derivatives /10/$ IEEE 4625

2 tion is given in Section II. A derivative estimator is proposed in Section III and it is shown how etremum seeking can be achieved for a static plant. This result is etended in Section IV for dynamic plants. An eample with simulations is given in Section V and conclusions are presented in Section VI. Proofs of the main results are provided in the Appendices. II. PROBLEM FORMULATION Let R denote the set of reals and N the set of positive integers. A function γ : R 0 R 0 is of class-g if it is zero at zero, continuous and non-decreasing. It is of class K if it is of class-g and strictly increasing. The continuous function β : R 0 R 0 R 0 is of class KL if it is strictly increasing in its first argument and strictly decreasing to zero in its second argument. For a vector R n with n N, is the Euclidean norm. Given a measurable function w : R 0 R n, its infinity norm is w = ess sup t [0, ) w(t). The notation w L, is defined to mean w <. For a nonlinear smooth function Q : R n R, define D j Q := j Q/ j where j N. When j = 1, the superscript is often omitted, e.g. DQ := D 1 Q. For convenience, let d N Q(u) := [ Q(u) DQ(u) D N Q(u) ] T, N N. The following nonlinear static map is first considered, y = Q(θ), (1) where Q( ) is not known, but it is known that Q( ) has a maimum y = Q(θ ) as stated in the following assumption. Assumption 1: Q( ) is N + 1 times continuously differentiable and has a unique global maimum 2, θ R, such that DQ(θ) = 0, iff θ = θ, (2) D 2 Q(θ ) < 0. (3) Similarly to [25], here it is assumed that the practitioner has selected a continuous optimization algorithm that could find the etremum of (1) if Q(u) was known. The purpose of this paper is to show how to implement the same algorithm approimately in an etremum seeking algorithm when Q(u) is not known. For simplicity, optimization algorithms of the following form are considered, θ = F(d N Q(θ)), (4) where F : R N+1 R. Remark 1: An eample of F( ) is: F(d 1 Q(θ)) = DQ(θ). This is the well-known gradient descent method that is frequently used in ES schemes [3], [27]. Another eample is the continuous Newton method in which F( ) is given by F(d 2 Q(θ)) = DQ(θ)/D 2 Q(θ), see for eample, [2], [16] and references cited therein. Remark 2: It is worthwhile to point out that other forms of continuous optimization can be used. For eample, some 2 Maimization of Q( ) is considered here, however it is possible to consider minimisation of Q 1 ( ) by letting Q( ) = Q 1 ( ) and applying the presented results unchanged. + a sin(t) ˆθ Fig. 1. θ δ s y = Q(θ) F(.) y Derivative estimator Structure of ES scheme for a static plant. internal states are needed in some modified continuous Newton algorithms as indicated in [2], [23]. An eample is as follows, θ = F 1 (θ,d N Q(θ),μ), θ = F 2 (θ,d N Q(θ),μ), where μ is some constant that can determine the convergence speed. The proposed framework can be applied to this and other types of continuous optimization algorithms with appropriate modifications. If Q( ) was known, then d N Q( ) could be calculated directly and the ES scheme could be implemented using (4). However, since Q( ) is unknown, then an estimator is needed to approimate d N Q( ). The purpose of this work is to generate a design framework that combines the continuous optimization scheme (4) with a derivative estimator (DE) to achieve etremum seeking in such a way that the closedloop system approimates the behavior of (4). Furthermore, semi-global practical asymptotic (SPA) stability of the closed loop system is desirable. To this end, when d N Q( ) is given, (4) must seek the optimal value θ with some robustness to disturbances in d N Q( ). This is stated more precisely in the following assumption: Assumption 2: There eistsβ θ KL such that the following holds: for any positive pair (Δ,ν), there eists ε > 0 such that for any ε (0,ε ) and w ε, the solutions of the following system satisfy θ = F (d N Q(θ)+w(t)), (5) θ(t) θ β θ (θ(0) θ,t)+ν, (6) for all θ(0) θ Δ. Remark 3: When w = 0, Assumption 2 is equivalent to [27, Assumption 4] with ν = 0. Figure 1 shows the proposed structure for an ES scheme acting on a static plant. a, and δ are tuning parameters that the designer must choose. One of the key components is the DE. In this note, a simple DE is presented, however there are potentially many alternative forms for the DE. III. MAIN RESULTS FOR STATIC MAPPING In this section, it is shown how one can properly adjust key parameters of the ES scheme, consisting of a simple DE 4626

3 Fig. 2. g, 0 L 1 L 2 L N L 1 y(t) sin(t) sin(t) sin(t) The structure of a eample derivative estimator. and an arbitrary optimization scheme, in order to obtain SPA stability results for the closed-loop. Figure 2 shows the proposed DE, which uses a series of sinusoidal dither signals to generate an approimate of d N Q( ). The nonlinear mapping g(, ) is given by where g(a,μ(θ,a)) := A 1 N A 1 0 (a)μ(θ,a), (7) A 0 (a) := diag ( 1 a a 2 a N), (8) α α 1 α 0 2! N 1 α N (N 1)! N! α α 2 α 1 2! N α N+1 (N 1)! N! A N := , (9) α α N+1 α N 2! 2N 1 α 2N (N 1)! N! 2π α i := 1 sin i (τ)dτ, i = 0,1,...,N, (10) 2π 0 ζ i (t,θ,a) := Q(θ +asin(t))sin i (t), i = 0,1,...,N, (11) ζ(t,θ,a) := [ζ 0 (t,θ,a) ζ 1 (t,θ,a) ζ N (t,θ,a)] T, (12) μ(θ,a) := 1 2π 2π 0 ζ(t,θ,a)dt. (13) Simple computation shows that the inverse of the matri A N is well-defined in (7). However, it should be noted that the condition number of A 0 (a) is a N. Therefore, as N is increased, the estimates of the derivatives of Q( ) become increasingly sensitive to any noise on y. The closed-loop dynamics of the system in Figure 1 with the DE from Figure 2 are given by ˆθ = δ F(g(a,ξ)), ξ = (ξ ζ(t,θ,a)), (14) where θ = ˆθ + asin(t), ξ = [ ] T, ξ 0 ξ 1 ξ N and ζ(t,θ,a) is defined in (12). Let θ = ˆθ θ and τ = t. In the new time scale τ, the closed-loop system (14) becomes d θ dτ = δf(g(a,ξ)), ( ( ( dξ τ τ dτ = ξ ζ, θ +θ +asin ) )),a, (15) whose averaged system is defined in (θ av,ξ av ) as: dθ av = δ F(g(a,ξ dτ av )), dξ av = (ξ dτ av μ(θ av,a)). (16) The system (16) is in the standard singular perturbation form with fast dynamics, ξ av. Denoting a new time scale s = δτ, it follows that dθ av ds = F(g(a,ξ av)), δ dξ av ds = (ξ av μ(θ av,a)). (17) To this end, let δ = 0 and freeze ξ av at its equilibrium : ξ av = μ(θ av,a) to obtain the reduced system in θ r : dθ r ds = F (g(a,μ(θ r,a))). (18) Let z(τ) := ξ av (τ) μ(θ av (τ),a). The first main result shows SPA stability of the closed-loop system consisting of the plant and ES scheme. Moreover, tuning guidelines for the ES scheme are proposed. Theorem 1: Suppose that Assumptions 1 and 2 hold. Then, there eist β θ and β ξ KL such that the following holds: for any given positive pair (Δ,ν), there eist ωl > 0 and a > 0, such that for any (0,ωL ) and any a (0,a ), there eists δ (a) > 0 such that for any δ (0,δ (a)), the solutions of the system (14) satisfy ( ) θ(t) β θ θ(t0 ),δ (t t 0 ) +ν, (19) z(t) β ξ (z(t 0 ), (t t 0 ))+ν, (20) for all [ θ(t0 ) z(t 0 ) ] T Δ and t t0 0. Proof: See the Appendi. Remark 4: Note that since Q( ) is continuous, then for any ν > 0, there eists ν 1 > 0 such that θ ν 1 = Q(θ) y ν. (21) Therefore Theorem 1 states that for any (Δ,ν) it is possible to adjust a, and δ so that limsup t y(t) y ν for any initial condition satisfying [ θ(t0 ) z(t 0 ) ] T Δ. In other words, the output of the system can be regulated arbitrarily close to the etremum value y from an arbitrarily large set of initial conditions by adjusting a, and δ. Remark 5: The DE generates an approimate of d N Q( ). The approimation error, (g(a,μ(θ,a)) d N Q(θ)), can be made arbitrarily small by reducing the parameter, a. This property, which is proven in Proposition 1, is important in proving Theorem 1. If a different DE was to be used in place of the presented DE, then it would be important for it to also have this property. For eample, consider an alternative DE, which is given by replacing (7) with g(a,μ(θ,a)) := A 1 0 (a)μ(θ,a), 4627

4 and replacing (11) with ζ i (t,θ,a) := { Q(θ +asin(t))i!2 i cos(it)( 1) i/2 sin(it)( 1) (i 1)/2 if i is even, if i is odd, for i = 0,1,...,N. It is possible to show the approimation error of the alternative DE is O(a 2 ). Moreover, it can be shown that Theorem 1 (and Theorem 2) holds for an ES scheme using the alternative DE. Remark 6: As can be seen from (19) and (20), there eists a clear time-scale separation in the closed-loop dynamics. The dynamics of the DE are fast while θ updates slowly. In [27], [29], [28], the convergence speed of the ES scheme depends on the choice dither signal amplitude, a. In this note, by choosing g(, ) carefully, the convergence speed is no longer determined by a. Similar to the discussion in [27], there is also a design trade-off between the convergence speed and the domain of attraction. Remark 7: Theorem 1 also provides a tuning guideline to achieve SPA stability. In this note, the SPA stability result is not uniform in the tuning parameters. A stronger stability property (SPA stability uniform in the tuning parameters) may be obtained if stronger conditions are used as in [27]. Remark 8: In contrast to the proof of Corollary 1 [27], an averaging method is first applied to system (14), followed by a singular perturbation method. The proof in the Appendi shows that the dynamics of the slowest time scale approimate the behavior of the continuous optimization scheme F( ) in (4). This eplains how the unifying ES approach works. Remark 9: When the gradient descent optimization method is used, the proposed derivative estimators can be simplified as in [27]. IV. DYNAMIC MAPPING Now consider the dynamic plant, ẋ = f(,u), y = h(), (22) where f : R n R R n and h : R n R. The state is assumed to be measurable, thereby allowing the use of the following family of control laws: u = α(,θ), (23) where θ R is a scalar parameter. The closed-loop system (22) with (23) then becomes ẋ = f(,α(,θ)). (24) The following assumptions are made for the system (24). Similar assumptions are used in [15], [27]. Assumption 3: There eists a function, l : R R n, such that f(,α(,θ)) = 0, if and only if = l(θ). (25) Assumption 4: For each θ R, the equilibrium = l(θ) of the system (24) is globally asymptotically stable, uniformly in θ. + Fig. 3. ˆ asin( t) L s F f,, y h g, y 0 L 1 L 2 L N L sin( t) sin( t) sin( t) An ES feedback scheme for a dynamic mapping. By using Assumptions 3 and 4, the reference-to-output map becomes Q( ) := h l( ). The results obtained for the static case can be easily etended to dynamic systems with the help of the singular perturbation technique and time-scale separation. The proposed etremum seeking scheme for the dynamic system (shown in Figure 3) is given by, ( ẋ = f,α(, ˆθ ) +asin(ω t)), ˆθ = δ ω F(g(a,ξ)), ξ = ω (ξ ζ(ω t,θ,a)), (26) where g(, ) is defined in (7) and ζ(,, ) is defined in (12). Let σ = ω t and θ = ˆθ θ in the new time scale σ, the system (26) becomes ω d dσ = f (,α(,θ +θ +asin(σ))), d θ dσ = δ F(g(a,ξ)), dξ dσ = (ξ ζ(σ,θ,a)). (27) As the system (27) ehibits two time scales, the singular perturbation method [13, Chapter 11] is applicable. Letting ω = 0 and freezing at its equilibrium, = l(ˆθ +a sin(σ)) gives the reduced system in variables θ s and ξ s in the time scale τ = σ : dθ s dτ = δ F(g(a,ξ s)), ( ( dξ s τ τ dτ = ζ,θ s +θ +asin ) ),a ξ s, (28) which is the same as (15), and which therefore has the same averaged system, (16). Let z 1 (τ) = ξ av (τ) μ(θ av (τ),a) and z 2 (t) = (t) l(θ + θ(t)+asin(ωt)). The second main result of this paper is stated as follows. Theorem 2: Suppose Assumptions 1 4 hold. Then there eist β θ,β ξ, and β KL such that the following holds: for any given positive pair (Δ,ν), there eists a > 0 and ωl > 0 such that for any a (0,a ) and (0,ωL ), there eist δ (a) > 0 such that for any δ (0,δ (a)), there eists ω (a,,δ) > 0 such that for any ω (0,ω (a,,δ)), the 4628

5 solutions of the system (26) satisfy ( ) θ(t) β θ θ(t0 ),δ ω (t t 0 ) +ν, (29) z 1 (t) β ξ (z 1 (t 0 ),ω (t t 0 ))+ν, (30) z 2 (t) β (z 2 (t 0 ),(t t 0 ))+ν, (31) for all [ θ(t0 ) z 1 (t 0 ) z 2 (t 0 ) ] T Δ and all t t0 0. Proof: The proof is omitted due to space limitations. Remark 10: Theorem 2 also provides tuning strategies for the etremum seeking controller to achieve semi-global practical convergence to the etremum. They are similar in spirit to the results found in [27]. Remark 11: The proof to Theorem 2 shows that the closed-loop system ehibits multiple time scale separation and that dynamics of the system in the slowest time scale approimate (on average) the chosen continuous optimization method. V. SIMULATION RESULTS Due to space limitations, only a static mapping is considered: y = h(u) = (u 2) 2. This mapping reaches the global maimum when u = u = 2. Therefore, Assumption 1 is satisfied. An ES scheme is implemented with the same structure as in Figure 1 and with the DE as given in Section III. A continuous Newton optimization method is usually of the following form: θ = DQ(θ) D 2 Q(θ)). In order to avoid the trouble caused by division by zero (or very small quantities), the following modified Newton method is used: { DQ(θ) if D 2 Q(θ) ε, F (d 2 Q(θ)) = D 2 Q(θ)) DQ(θ) ε otherwise, (32) where ε = 0.1 in the simulation. By a careful check, Assumption 2 is satisfied. Figure 4 shows the output of the plant when a = 0.2, δ = 0.5 and = 0.02 (and the initial outputs of the low-pass filters are all set to zero). The proposed ES scheme achieves convergence of the output to within a small neighbourhood of its maimum. VI. CONCLUSIONS A unifying, prescriptive design framework has been presented for etremum seeking where an arbitrary continuous optimization method can be combined with an estimator of the derivatives of the unknown reference-to-output map. It was shown how the controller parameters can be tuned in order to achieve semi-global practical convergence to the etremum under appropriate assumptions. Both static and dynamic plants were considered. For simplicity, only SISO plants were discussed. MISO plants will be addressed in future work. Output y time (second) Fig. 4. The output of the static system with an ES scheme composed of a simple DE and a modified continuous Newton optimization scheme. REFERENCES [1] R. G. Airapetyan, A. G. Ramm and A. B. Smirnova, Continuous analog of the Gauss-Newton method, Mathematical Models and Methods in Applied Sciences, vol. 9, pp , [2] R. G. Airapetyan, Continuous newton method and its modification, Applicable Analysis, vol. 73, pp , [3] K. B. Ariyur and M. Krstić, Real-Time Optimization by Etremum- Seeking Control. Hoboken, NJ: Wiley-Interscience, [4] J. F. Beaudoin, O. Cadot, J. L. Aider and J. E. Wesfreid, Bluff-body drag reduction by etremum-seeking control, Journal of Fluids and Structures, vol. 22, pp , [5] E. Biyik and M. Arcak, Gradient climbing in formation via etremum seeking and passivity-based coordination rules, Proc. 46 th IEEE Conf. Decis. Contr., New Orleans, USA, [6] P. F. Blackman, Etremum-seeking regulators, In J. H. Westcott, An eposition of adaptive control New York: The Macmillan Company, [7] G. Bastin, D. Nešić, Y. Tan and I. M. Y. Mareels, On etremum seeking in bioprocesses with multi-valued cost functions, Biotechnology Progress, vol. 25, No 3, pp , [8] D. Carnevale, A. Astolfi, C. Centioli, S. Podda, V. Vitale and L. Zaccarian, A new etremum seeking technique and its application to maimize RF heating on FTU, Fusion Engineering and Design, vol. 84, pp , [9] S. Drakunov, U. Özgüner, P. Di and B. Ashra, ABS control using optimum search via sliding modes, IEEE Transactions on Control Systems Technology, vol. 3, pp , [10] P. M. Dower, P. Farrell and D. Nešić, Etremum seeking control of cascaded Raman optical amplifiers, IEEE Transactions on Control Systems Technology, vol. 16, pp , [11] M. Guay, D. Dochain and M. Perrier, Adaptive etremum seeking control of continuous stirred tank bioreactors with unknown growth kinetics, Automatica, vol. 40, pp , [12] M. Guay and T. Zhang, Adaptive etremum seeking control of nonlinear dynamic systems with parametric uncertainties, Automatica, vol. 39, pp , [13] H. K. Khalil, Nonlinear Systems, Third edition Prentice Hall, Upper Saddle River, New Jersey, [14] M. Krstić, Performance improvement and limitations in etremum seeking control, Systems and Control Letters, vol. 39, pp , [15] M. Krstić and H.-H. Wang, Stability of etremum seeking feedback for general nonlinear dynamic systems, Automatica, vol. 36, pp , [16] W. H. Moase, C. Manzie and M. J. Brear, Newton-like etremumseeking for the control of thermoacoustic instability, IEEE Transactions on Automatic Control, vol. 55, pp , 2010 [17] D. Nešić, A. Mohamadi and C. Manzie, A systematic approach to etremum seeking based on parameter estimation, submitted to Conf. Decis. Contr., Atlanta, Georgia, USA, [18] P. Ögren, E. Fiorelli and N. E. Leonard, Cooperative control of mobile sensor networks: adaptive gradient climbing in a distributed 4629

6 environment, IEEE Transactions on Automatic Control, vol. 49, pp , [19] I. I. Ostrovskii, Etremum regulation, Automatic and Remote Control, vol. 18, pp , [20] A. A. Pervozvanskii, Continuous etremum control system in the presence of random noise, Automatic and Remote Control, vol. 21, pp , [21] K. S. Peterson and A. G. Stefanopoulou, Etremum seeking control for soft landing of an electromechanical valve actuator, Automatica, vol. 40, pp , [22] D. Popović, M. Janković, S. Manger and A. R. Teel, Etremum seeking methods for optimization of variable cam timing engine operation, IEEE Transactions on Control Systems Technology, vol. 14, pp , [23] A. G. Ramm, A. B. Smirnova and A. Favini, Continuous modified Newtons-type method for nonlinear operator equations, Annali di Matematica, vol. 182, pp , [24] J. Sternby, Etremum control systems: An area for adaptive control?, Proc. Joint Amer. Contr. Conf., San Francisco, CA, [25] A. R. Teel and D. Popović, Solving smooth and nonsmoooth multivariable etremum seeking problems by the methods of nonlinear programming, Proc. Amer. Contr. Conf., Arlington, VA, June 25 27, 2001, pp [26] Y. Tan, D. Nešić and I. M. Y. Mareels, On non-local stability properties of etremum seeking control, Proc. 16th IFAC World Congress, Prague, Czech Republic, 2005, pp [27] Y. Tan, D. Nešić and I. M. Y. Mareels, On non-local stability properties of etremum seeking controllers, Automatica, vol. 42, pp , [28] Y. Tan, D. Nešić, I. M. Y. Mareels and A. Astolfi, On the global etremum seeking control, Automatica, vol. 45, No. 1, pp , [29] Y. Tan, D. Nešić and I. M. Y. Mareels, On the dither choice in etremum seeking control, Automatica, vol. 44, pp , [30] H.-H. Wang, S. Yeung and M. Krstić, Eperimental application of etremum seeking on an aial-flow compressor, IEEE Transactions on Control Systems Technology, vol. 8, pp , [31] H.-H. Wang and M. Krstić, Etremum seeking for limit cycle minimization, IEEE Transactions on Automatic Control, vol. 45, pp , [32] D. J. Wilde, Optimum Seeking Methods. Englewood NJ: Prentice-Hall Inc., [33] C. Zhang, D. Arnold, N. Ghods, A. Siranosian and M. Krstić, Source seeking with non-holonomic unicycle without position measurement and with tuning of forward velocity, Systems and Control Letters, vol. 56, pp , [34] C. Zhang, A. Siranosian and M. Krstić, Etremum seeking for moderately unstable systems and for autonomous vehicle target tracking without position measurements, Automatica, vol. 43, pp , [35] X. T. Zhang, D. M. Dawson, W. E. Dion and B. Xian, Etremumseeking nonlinear controllers for a human eercise machine, IEEE/ASME Transactions on Mechatronics, vol. 11, pp , A. Proposition 1 APPENDIX Let O ( a j) R N+1 denote a vector of O ( a j) quantities. The following proposition is needed to prove the main result. Proposition 1: For any a > 0 and θ R, the following equality holds g(a,μ(θ,a)) = d N Q(θ)+O ( a 2). (33) Proof: The Taylor series epansion of Q(θ+asin(t)) yields Q(θ +asin(t)) = N+1 j=0 a j j! Dj Q(θ)sin j (t)+o(a N+2 ). (34) Averaging gives 2π μ i = 1 Q(θ +asin(t)) sin i (t)dt 2π 0 = A 0 (a) [ ] α i dn Q(θ) Consequently, α i+n N! + α i+n+1 (N +1)! an+1 D N+1 Q(θ)+O ( a N+2). μ(θ,a) = A 0 (a)a N d N Q(θ)+O ( a N+2) +diag ( α N+1 α N+2 α 2N+1 ) O ( a N+1 ). (35) Substituting (35) into (7) leads to g(a,μ(θ,a)) = d N Q(θ)+A 0 (a) 1 O ( a N+2) +diag ( α N+1 α N+2 a α 2N+1 a N ) O ( a N+1 ). Equation (33) follows after noting that α 2N+1 = 0. B. Proof of Theorem 1 The proof of Theorem 1 is carried out in the following steps: Step 1: Using Assumption 2 and Proposition 1, it can be shown that for any given positive pair (Δ,ν/2), there eists a > 0 such that for any a (0,a ), the solutions of the system (18) satisfy θ r (s) β θ (θ r (s),s)+ ν 2, (36) for all θ r (0) Δ. Sketch of proof: Using Proposition 1, the reduced system (18) can be re-written as dθ r ds = F ( d N Q(θ r )+O ( a 2)), (37) which is in the form of (5). By using Assumption 2, the result holds. Step 2: For any given positive pair (Δ,ν/2), there eists a > 0 such that for any a (0,a ) there eists δ (a) > 0 such that for any δ (0,δ (a)), the solutions of boundary layer system satisfy z(τ) β ξ (z(0),(τ τ 0 )), for all z(0) Δ. Sketch of proof: The equilibrium of the boundary layer is z e = 0, which is globally asymptotically stable. In Step 1, it was shown that the reduced system of the singularly perturbed system (17) is SPA stable in a. Hence, using [26, Lemma 2], the result holds. Step 3: For any given positive pair (Δ,ν), there eists ωl > 0 and a > 0 such that for any (0,ωL ) and any a (0,a ), there eists δ (a) > 0 such that for any δ (0,δ (a)), the solutions of the system (14) satisfy (19) and (20) in the time scale t. Sketch of proof: By using averaging results in [26, Lemma 1], inequalities (19) and (20) hold. Combining the above steps completes the proof. 4630