Newton-Like Extremum-Seeking Part I: Theory

Joint 48th IEEE Conference on Decision and Control and 28th Chinese Control Conference Shanghai, P.R. China, December 16-18, 29 ThAIn1.9 Newton-Like Extremum-Seeking Part I: Theory William H. Moase, Chris Manzie and Michael J. Brear Abstract In practice, the convergence rate and stability of perturbation based extremum-seeking ES schemes can be very sensitive to the curvature of the plant map. This sensitivity arises from the use of a gradient descent adaptation algorithm. Such ES schemes may need to be conservatively tuned in order to maintain stability over a wide range of operating conditions, resulting in slower optimisation than could be achieved for a fixed operating condition. This can severely reduce the effectiveness of perturbation based ES schemes in some applications. It is proposed that by using a Newton-like step instead of a more typical gradient descent adaptation law, then the behaviour of the ES scheme near an extremum will be independent of the plant map curvature. In this paper, such a Newton-like ES scheme is developed and its stability and convergence properties are explored. I. INTRODUCTION Consider a plant with an output, y, which is an unknown function of an input,. The goal of extremum-seeking is to find the input,, which minimises or maximises y using only measurements of the output. Without loss of generality, this paper will deal with the problem of minimising y. Furthermore this paper will deal only with the problem of seeking a local minimum of y. Fig. 1 shows a basic schematic of sinusoidally perturbed ES. The control input is the superposition of a slowly changing component,, and a small dither signal,asinω t. The dither signal is used to determine ˆy, where y = dy/d, is a quantity evaluated at =, and ˆ is an estimate. The exact method for estimating y varies between different schemes but requires to be slowly changing compared to the dither signal. With an estimate ofy available, can be driven towards using an approximated gradient descent law, d dt = ω k ˆy. 1 Sinusoidally perturbed ES schemes were amongst the first adaptive controllers developed and were popular in the 195s and 196s [1]. In 2, there was a resurgence of interest in ES, largely due to the development of the first stability analysis of sinusoidally perturbed ES on a general, nonlinear plant in [2]. In the same year, a number of other papers on sinusoidally perturbed ES were published [3] including extensions to multi-parameter optimisation [4], [5]. There have been a number of developments in sinusoidally perturbed ES since 2 including: extension to discrete This research was partially supported under Australian Research Council s Discovery Projects funding scheme project number DP984577. W. H. Moase, C. Manzie and M. J. Brear are with the Department of Mechanical Engineering, The University of Melbourne, 31, Victoria, Australia moasew@unimelb.edu.au, manziec@unimelb.edu.au, mjbrear@unimelb.edu.au Fig. 1. + Plant a sinω t y Gradient estimator k ω Basic schematic of sinusoidally perturbed ES. time [6]; semi-global stability results [7]; the use of periodic non-sinusoidal dither signals [8]; the use of a time-dependent dither signal amplitude [9]; and the development of stochastically perturbed ES [1]. As a result of using an approximated gradient descent adaptation law, the local convergence speed of a perturbation based ES scheme is typically proportional to y, where is a quantity evaluated at = and y = d 2 y/d 2. This dependence is evident in the mathematical analysis of particular ES schemes [11], and can severely reduce the effectiveness of perturbation based ES in applications with a wide range of plant behaviours [12]. If the operating condition changes, then an increase in y may destabilise the scheme since the rate of change of must remain sufficiently small in order to estimate y whereas a decrease in y will result in a reduced rate of convergence. This curvature dependence may be reduced by introducing further compensators to the ES scheme [11]. By appropriate selection of these compensators, the behaviour of the averaged, linearised closed-loop system can be tuned. This tuning could aim to increase the speed at which the extremum is tracked or, of more interest here, it could reduce the sensitivity of the closed-loop system to perturbations iny by using, for example, H techniques. Although this reduced sensitivity to y is an improvement over more simple ES schemes, this technique requires an a priori estimate of y and is most effective for small variations in y. Some schemes are capable of seeking minima in a fashion which is independent of the curvature at that minimum. One such scheme is the discrete-time triangular search algorithm [13] which uses information from previous steps rather than a dither signal in order to determine its next step. The magnitude of each step is not based on any estimate of the gradient, so the performance of the scheme is not related to the magnitude of y. This is evident in [14], where triangular search and sinusoidally perturbed ES are compared in the role of minimising the thermoacoustic limitcycle pressure oscillations in a gas-turbine combustor. The adaptation gain, k, used in the sinusoidally perturbed ES ŷ 978-1-4244-3872-3/9/$25. 29 IEEE 3839

scheme had to be changed between operating conditions, whereas triangular search required no such tuning. The Kiefer-Wolfowitz KW algorithm [15] is a popular stochastic approximation scheme for online minimisation. It uses a gradient descent law, n+1 = n k nˆy n, where n denotes a value at the n-th step of the algorithm and k n is a sequence of positive numbers satisfying certain, wellknown, conditions which include k n as n. The gradient is estimated using a finite difference which is performed over an interval [ n a n, n + a n ] where a n as t. There have been a number of extensions to the KW algorithm which include multi-variable optimisation [16], [17], [18], methods for handling constrained optimisation problems [19], and a deterministic result for non-smooth optimisation [2]. Since the KW algorithm, and its mentioned extensions, use a gradient descent algorithm, optimal selection of the series k n and a n requires knowledge of y [21]. This problem is largely solved through the use of a Newton step [22], [23]. In the one-dimensional case, this involves estimating both y and y before progressing n an amount proportional toy /y. In a region near the extremum, the Newton step approximates the difference between the current input and the optimal input, so in the case of perfect estimation of y and y, it has a local rate of convergence which is independent of y. Newton-like SA and triangular search achieve convergence independently of y, however, they lack one major advantage offered by perturbation based ES: the ability to achieve convergence to the extremum on a time-scale comparable to that of the plant dynamics [1] instead, they require the plant dynamics to settle between each step of the algorithm. This motivates the present work: to develop a sinusoidally perturbed ES scheme which uses a Newton-like step. The proposed scheme is described in Section II, and as well as using a Newton-like step, features a dither signal amplitude schedule DSAS. This idea is similar to the shrinking interval used in the KW algorithm for estimating the gradient, or the time-varying dither signal amplitude for ES used in [9]. By initialising a to be large and reducing it as the extremum is approached, then fast convergence rates and accurate convergence are simultaneously achievable. The most significant difference between the proposed DSAS and the aforementioned schemes is the ability for the DSAS to increase the dither signal amplitude should change after some time. In Section III, it is shown that the proposed scheme is stable for a noiseless plant with no dynamics and that arbitrarily accurate convergence to the extremum can be achieved. For a plant with linear time-invariant LTI input and output dynamics, conditions for local stability of the ES scheme are given, and it is shown that the local rate of convergence is independent of y. II. PROPOSED SCHEME Fig. 2 shows a schematic of the proposed scheme. The plant is subject to the input = +asinω t, 2 sinω t + Fig. 2. a Plant y DSAS Gradient estimator Adaptation law aŷ Basic schematic of proposed ES. a 2ŷ where ω >. is progressed according to the adaptation law, and at can be progressed according to the DSAS or may simply be set to a constant, a = a min, where a min >. Meanwhile, the gradient estimator determines ˆy and ˆy for use in the adaptation law. A. Adaptation law Let =, and consider the Taylor expansions, y = y +O 2, y = y +O. 3 It then becomes apparent that the local small behaviour of a regular gradient descent law would yield a rate of change of which is proportional to y, whereas a Newton step would yield a rate of change which is proportional to. Therefore the local behaviour of an adaptation law using a Newton step will be independent of y unlike a regular gradient descent law. A more practical alternative to a Newton-step is, d /ˆy dt = k ω ˆy if ˆy < δa minˆy, 4 k ω δa min sgn ˆy otherwise, where δ,k > are dimensionless quantities. is progressed at a rate corresponding to an approximated Newton step only when δa minˆy > ˆy. Otherwise, is progressed at a rate corresponding to a sign-of-gradient descent. This latter behaviour has two purposes. First of all, because the Newton step seeks satisfying ˆy =, then it may seek a maximum or inflection point instead of a minimum. However, at a maximum or inflection point, y, so sign-of-gradient descent behaviour will instead be followed, giving a more desirable result. The second purpose of the sign-of-gradient descent behaviour is to saturate the rate of change of at ±k ω δa min. This avoids singular behaviour ˆy of the Newton step as, and also makes it possible to ensure changes slowly compared to the dither signal. As will become apparent in the next subsection, the latter of these properties is important for the estimation of y and y. B. Gradient estimator Consider a plant with no dynamics subject to the input as defined in 2. Taking the Taylor series expansion of y about = gives y = y +y asinω t+ 1 2 y a 2 sin 2 ω t+h, 5 384

where h is terms of third and higher order in a. Alternatively, the plant output may be represented in state-space: y = Cx+h, dx dt = ω Ax+ x d dt + x da a dt, where C = 1 1 4 1, y + 1 4 a2 y y as 1 x = y ac 1 y a 2 S 2, A = 1 1 2, y a 2 C 2 2 6a 6b y n = d n y/d n, S n = sinnω t and C n = cosnω t. This system can then be estimated with the state-space observer: ˆy = Cˆx, dˆx dt = ω Aˆx+ω Ly ˆy, 7a 7b where L R 5 is a non-dimensional gain vector. Thus y and y can be estimated from: aˆy = Cˆx, C = S 1 C 1, 7c a 2ˆy = Cˆx, C = S 2 C 2. 7d Since the observer does not explicitly account for the variation of a and with time, it is most effective when a and change slowly compared to the dither signal. Note that although this state-space observer is based on that used in [14], it has been extended to be capable of estimating y. C. Dither signal amplitude schedule As discussed in the previous subsection, the ability of the state-space observer to accurately estimate y and y relies on a and varying slowly compared to the dither signal. In order to increase the maximum allowable rates of change of a and, then one could simply increase ω. However, the maximum allowable value of ω depends upon plant noise and dynamics. If faster convergence of to was required than could be achieved by simply increasing ω, then a could be increased. However, even if the ES scheme was able to achieve perfect convergence of to, then the superposition of a large dither on the plant input would result in large fluctuations in y about its minimum. Instead, it would be ideal to have large a when the rate of change of is large, and small a when the rate of change of is small. The most simple way of achieving such behaviour would be to let a d /dt, however, this is impractical since: from 4 and 7c 7d, a circular algebraic relationship arises between a and d /dt; and noise in the measurement of y will prevent the use of an arbitrarily small dither signal amplitude. Instead, it is proposed that a be related to d /dt by the differential equation, da dt = k aω α a, 8 3841 where k a >. Therefore at any given instant, 8 attempts to drive a towards α, which is given by 1 d α = a min +a min χ, 9 a min ω k dt where χ: R R > and χz γz for all z and some γ. The quantities, a min, k and ω, are defined in 2 and 4. Remark 1: If there was no noise on y and was a constant, then a min could be made arbitrarily small, however, a min should be finite in any practical scenario. Generally χ should be chosen to be some function which increases with d /dt so that a scales, in some sense, with d /dt. A. Plant with no dynamics III. STABILITY RESULTS Assumption 1: The plant see Fig. 2 has no dynamics, and is given by a time-invariant function y : R R with y >. Furthermore, there exists a domain D containing the origin such that for all D : y + is twice continuously differentiable with respect to ; and sgny + = sgn. Assumption 2: A LC is Hurwitz. Remark 2: Assumption 2 can be satisfied by appropriate selection of L. Theorem 1: Under Assumptions 1 and 2, for any compact set D D R 5 containing the origin, there exist a min >, k > and ka >, such that for all a min,k,k a,a min,k,k a, the solution t, at of system 2, 4, 7a 9, with initial conditions a,, x [,γδa min ] D satisfies limsup t = O t limsup at = O t a 2 miny 3 1 /y a 2 miny 3 1 /y, 1a, 1b where a = a a min, x = ˆx x, and 1 is some input satisfying 1 a. Furthermore, if R satisfies the conditions placed on D in Assumption 1, then 1a and 1b can be satisfied for arbitrarily large and x. Proof: Consider the non-dimensional parameters, = a min, a = a a min, x = x a 2 min y, t = ω t. Letting A x = A LC, then system 2, 4, 7a 9 can be expressed in non-dimensional terms as d d t = k f, a, x, t, 11a d a d t = k a χ f, a, x, t a, 11b d x d t = A 1 x d x x+l h d t + x d a, 11c a d t

where h = h/a 2 min y, / f, a, x, t ˆy = δsgn ˆy a minˆy if ˆy < a min δˆy otherwise, ˆy = y +C x1+ a 1, ˆy = y +y C x1+ a 2, Under Assumption 2 there exists a symmetric, positive definite matrix, P, which satisfies the Lyapunov equation, PA x +A T xp = I. Let V = y y k a 2 +γ +1 min y, k + a k a +μ x T P x, 12 where μ >. The domain of interest for this proof is restricted to a min,k,k a, a,, x,a min,k,k a [,γδ] D, where D D R 5 is a compact set containing the origin and D = {z/a min : z D }. By Assumption 1, V is a positive definite function of, a, x and is, therefore, a suitable Lyapunov candidate function for system 2, 4, 7a 9. However, it is still necessary to explore the conditions under which dv/d t <. Letting denote the L 2 norm, where dv d t = μ h LT P x xt P x +ξ +ξa a+ξ x x, 13 ξ = g f, a,, t ξ a = μk a a+1y ξ x = g x 1 2 μ xt x x T P x, +μk C a ay P x y xt P x, [ 1 ] 2 S 2 C 2 P x 1, y xt P x and g x = g f, a, x, t g f, a,, t, x gz = g 1 z+g 2 z, [ y g 1 z = y a +γ +1sgn ] z +χz, min g 2 z = μ [ k z x +k a χz x ] T P x a xt P x. In the next part of this proof, it is demonstrated that there exists μ and sufficiently small k and k a such that ξ a,ξ,ξ x < over the entire domain of interest. It is quite obvious that ξ a < if μk a is sufficiently small, however, it is not so clear that ξ and ξ x can also be made negative. First, it is useful to note that sgnf, a,, t = sgn. Using this result and the fact that χz γz, then g 1 f, a,, t [ ] y f, a,, t y a +1 <. min Therefore ξ < if μk and μk a are sufficiently small. One can find a sufficiently large μ to ensure that ξ x < so long as g x is bounded over the domain of interest. This can be guaranteed if gf/ x n is bounded for n = 1,2,3,4,5. Note that gf/ x 1 =. When ˆy = and ˆy a minδˆy then gf/ x n = for n = 2,3,4,5. Furthermore, when ˆy < a minδˆy, it is a relatively simple matter to show { f ˆy x n y 1+ a 1 if n = 2,3, δ1+ a 2 14 if n = 4,5. Because f/ x n is bounded in all of these cases, then it looks promising that g x will also be bounded. However, there are two scenarios that require further discussion: gf/ x n contains a term N = f y x n a min y, 15 which is not obviously bounded as a min. However, the triangular inequality z 1 +z 2 z 1 +z 2 can be used to show that y a min y and when ˆy < a minδˆy, then y δˆy < y It follows that N is, in fact, bounded. ˆy + C x 1+ a, 16 + C x 1+ a. 17 = & ˆy. f/ x n is not bounded when ˆy Nonetheless, all of the terms in gf are bounded, most notably, f, a, x, t δ, and from 16, y /a min y C x/1 + a. It follows that g x is still bounded when ˆy = and ˆy unless such a situation can occur for infinitesimal x. This would require both y = and y. However, by Assumption 1, y = if and only if =, in which case y >. Therefore g x is bounded, and there exists sufficiently large μ such that ξ x <. Sinceξ,ξ a,ξ x < over the entire domain of interest, then dv/d t < unless, a, x = O h. Taking advantage of Assumption 1, then there exists an input, 1, such that h = a miny 3 1 1+ a 3 S1 3 6y, 18 and 1 a. Therefore, h is bounded and can be made arbitrarily small by decreasing a min. Thus, it is possible to ensure that dv/d t < except for a small domain containing the origin. Theorem 1 follows directly after transforming the system back into dimensional variables. Corollary 1: Under Assumptions 1 and 2, for any compact set D D R 5 containing the origin, there exist k > and a min >, such that for all a min,k,a min,k, the solution t of system 2, 4, 7a 7d with a = a min and initial conditions, x D satisfies 1a. Furthermore, if R satisfies the conditions placed on D in Assumption 1, then 1a can be satisfied for arbitrarily large and x. 3842

F i s i f F o s y Assumption 5: The dynamics F i s and F o s can be represented in state-space forms so that: Fig. 3. Plant Plant with LTI input and output dynamics. Remark 3: Theorem 1 demonstrates the stability of the proposed scheme with DSAS, whereas Corollary 1 demonstrates the stability of the scheme with a constant dither signal amplitude a = a min. In both cases, the influence of the ES scheme on the plant output may be found by taking the Taylor series expansion of y about = to gain limsupyt y = O y a 2 min. 19 t B. Plant with LTI input and output dynamics Up to this point, the analysis of the proposed ES scheme has considered the plant to be a static input-output map. If there were potentially nonlinear stable dynamics resulting in a time-invariant steady-state mapping of the input to the output, then it is expected that the dynamics will have only a small effect on the behaviour of the scheme provided that ω is sufficiently small. This choice of ω effectively ensures time-scale separation between the ES scheme and the plant dynamics a similar example of this can be seen in [2]. However, such a requirement limits the rate of convergence that can be achieved by the ES scheme. In this subsection, the local behaviour of the proposed scheme is studied when time-scale separation between the plant dynamics and the ES scheme is not guaranteed. To simplify matters, the influence of DSAS is not investigated. A slight modification of the proposed scheme is considered: lags φ 1 and φ 2 are introduced in the demodulation signals so that C and C appearing in 7c and 7d respectively are replaced with C = sinω t φ 1 cosω t φ 1, C = sin2ω t φ 2 cos2ω t φ 2. The purpose of these lags, as will become apparent in Theorem 2, is to compensate for lags due to the plant dynamics between the dither signal and the corresponding sinusoidal components of the plant output. Let s denote the Laplace variable and let the use of square brackets in the context Gs[ft] denote the time-domain output of the transfer function Gs when ft is its input. Assumption 3: The plant can be expressed as shown in Fig. 3, so that i = F i s[] and y = F o s[f i ], where F o s and F i s are LTI dynamics. Assumption 4: There exists a domain D R containing, such that for all z D, f z = f + 1 2 f z 2, with f >. 2 Furthermore, the dither amplitude, a >, is constant and sufficiently small to ensure that a, +a D. dx i dt = A dx o ix i +B i, = A o x o +B o f i, dt i = C i x i +D i, y = C o x o +D o f i, where A i and A o are Hurwitz and F i = F o = 1. Remark 4: Assumption 5 ensurees F i and F o have stable and proper LTI dynamics. The final part of Assumption 5 can be made without loss of generality. If F i = 1 and/or F o = 1, then it is a simple matter to transform Fi s F i s,f o s,f F i, F os F o,f if o f. Theorem 2: Let i = F i s[ ] and x = ˆx x where ] f + 1 4 a2 f F i iω 2 + 1 2 f F o s [ 2 { ]} i af Im e iωt F i iω F o s+iω [ i { ]} x = af Re e iωt, F i iω F o s+iω [ i a 2 f Im { } e 2iωt F 2 a 2 f Re { } e 2iωt F 2 and are 2π/ω -periodic and F 2 = F 2 i iω F o 2iω. Also let x i = x i x 2π i x o = x o x 2π o solutions of d dt x2π d dt x2π Finally let where x 2π i and x 2π o i = A i x 2π i +B i +asinω t, o = A o x 2π o +B o f +Im { e iωt F i iω }. 1 Hs = s+k ω J sf i s, 21a J s = Re{ } F i iω F o s+iω e iφ1. 21b F 2 cosφ 2 +argf 2 Under Assumptions 2 5, i, x, x i, x o = is a locally exponentially stable equilibrium point of the system given by 2, 4 and 7a 7d provided that: s = ±iω are not zeroes of either F i s or F o s; s = ±2iω are not zeroes of F o s; cosφ 2 +argf 2 > ; k is sufficiently small; and the poles of Hs all have negative real parts. Proof: Due to space restrictions, only a sketch of a proof is provided here. A more complete proof may be found in [24]. Let i = i /a and x = x/a 2 f. Equations 7c and 7d can respectively be simplified to, ˆy af = C x+re { e iφ1 F i iω F o s+iω [ i ]}, 22a ˆy f = C x+f 2 cosφ 2 +argf 2. 22b If x =, then ˆy = f F 2 cosφ 2 + argf 2. Therefore, cosφ 2 + argf 2 must be positive for the adaptation law to locally follow a Newton step. Otherwise, the adaptation 3843

law would follow a sign-of-gradient descent and, at best, i would chatter about zero. Substituting 7a into 7b and 22a 22b into 4 and linearising each resulting equation about i, x, x i, x o = respectively yields d i d t = k F i J s [ i ] k F i s[c x] F 2 cosφ 2 +argf 2, 23 { [ ]} d x d t = A d i x x+lε x t Re ΓF o s+iω 24 d t where Γ = i 1 T F i iω expiω t and ε x t consists of terms that decay to zero independently of i and x i. Therefore, ε x t does not affect the local stability of the ES scheme and is ignored for the remainder of this analysis. If x = in 23, then the dynamics of i are given by Hs, and i converges to zero if the poles of Hs all have negative real parts. Similarly from 24, if i =, then x converges to zero under Assumption 2. Therefore 23 and 24 can be thought of as two interconnected systems which are independently stable. If the interconnections are sufficiently weak, then the ES scheme will remain stable. Theorem 9.2 in [25] quantifies the conditions under which the interconnections can be considered sufficiently weak. In this case, stability of the ES scheme can be guaranteed if k is sufficiently small. Theorem 2 follows directly. Remark 5: The local convergence of i to zero is independent of f since Hs is independent of f. Remark 6: Consider the quantity ˆk where ˆk k = J F i = F 1cosφ 1 +argf 1 F 2 cosφ 2 +argf 2, and F 1 = F i iω F o iω. If Ω := ˆk ω is sufficiently small, then Hs has a dominant pole at s = Ω +OΩ 2. This allows the final dot-point of Theorem 2 to be tested more easily, but effectively restricts the dynamics of i to be well-separated not only from the gradient estimator dynamics, but also from the plant dynamics. Remark 7: When φ 1 = φ 2 = and F i s = F o s = 1, Hs has a single pole at s = k ω. If Ω is small see Remark 6 and non-trivial dynamics are introduced, that pole is shifted to a location near s = ˆk ω. If ˆk <, then the dynamics will destabilise the system. This occurs because sgny = sgnˆy about the equilibrium point. It follows that the scheme will climb the plant map rather than descend it as intended. If ˆk < k, then slower convergence of i to zero will be observed than in the absence of dynamics. If ˆk > k, faster convergence will be observed, however, the region for which the adaptation law follows a Newton-step will decrease. For fixed ω, the influence of the pole shift from k ω to ˆk ω may be compensated through selection of a different value of k or through suitable selection of φ 1 and φ 2. IV. CONCLUSIONS An ES scheme using a Newton-like adaptation law and DSAS was developed and shown to achieve convergence of the control input to a small neighbourhood of the extremum from a potentially infinite domain of initial conditions. Furthermore, for a plant with LTI input and output dynamics, conditions for local stability of the ES scheme without DSAS were found and it was shown that local convergence of the control input is independent of y. In part II of this paper, the behaviour of the proposed ES scheme is further explored in both numerical and experimental tests. REFERENCES [1] K. B. Ariyur and M. Krstic, Real-Time Optimization by Extremum- Seeking Control. John Wiley & Sons, 23. [2] M. Krstic and H. Wang, Stability of extremum seeking feedback for general nonlinear dynamic systems, Automatica, vol. 36, pp. 595 61, 2. [3] R. N. Banavar, D. F. Chichka, and J. L. Speyer, Convergence and synthesis issues in extremum-seeking control, in Proceedings of the American Control Conference, 2, pp. 438 443. [4] M. A. Rotea, Analysis of multivariable extremum seeking algorithms, in Proceedings of the American Control Conference, 2, pp. 433 437. [5] G. C. Walsh, On the application of multi-parameter extremum seeking control, in Proceedings of the American Control Conference, 2, pp. 411 415. [6] J.-Y. Choi, M. Krstic, K. Ariyur, and J. S. Lee, Extremum seeking control for discrete time systems, IEEE T. Automat. Contr., vol. 47, pp. 318 323, 22. [7] Y. Tan, D. Nesic, and I. Mareels, On non-local stability properties of extremum seeking control, Automatica, vol. 42, pp. 889 93, 26. [8] D. Nesic, Y. Tan, and I. Mareels, On the choice of dither in extremum seeking systems: a case study, in Proceedings of the IEEE Conference on Decision and Control, 26, pp. 2789 2794. [9] Y. Tan, D. Nesic, I. Mareels, and A. Astolfi, On global extremum seeking in the presence of local extrema, in Proceedings of the IEEE Conference on Decision and Control, 26, pp. 5663 5668. [1] C. Manzie and M. Krstic, Extremum seeking with stochastic perturbations, IEEE T. Automat. Contr., vol. 54, pp. 58 585, 29. [11] M. Krstic, Performance improvement and limitations in extremum seeking control, Syst. Control Lett., vol. 39, pp. 313 326, 2. [12] A. Banaszuk, K. B. Ariyur, M. Krstic, and C. A. Jacobsen, An adaptive algorithm for control of thermoacoustic instability, Automatica, vol. 4, pp. 1965 1972, 24. [13] Y. Zhang, Stability and performance tradeoff with discrete time triangular search minimum seeking, in Proceedings of the American Control Conference, 2, pp. 423 427. [14] A. Banaszuk, Y. Zhang, and C. A. Jacobson, Adaptive control of combustion instability using extremum-seeking, in Proceedings of the American Control Conference, 2, pp. 416 422. [15] J. Kiefer and J. Wolfowitz, Stochastic estimation of a regression function, Ann. Math. Stat, vol. 23, pp. 462 466, 1952. [16] J. R. Blum, Multidimensional stochastic approximation methods, Ann. Math. Stat, vol. 25, pp. 737 744, 1954. [17] H. J. Kushner and D. S. Clark, Stochastic approximation methods for constrained and unconstrained systems. Springer-Verlag, 1978. [18] J. C. Spall, Multivariate stochastic approximation using a simultaneous perturbation gradient, IEEE T. Automat. Contr., vol. 37, no. 3, pp. 332 341, 1992. [19] H. J. Kushner, Stochastic approximation algorithms for constrained optmization problems, Ann. Stat, vol. 2, no. 4, pp. 713 723, 1974. [2] A. R. Teel, Lyapunov methods in nonsmooth optimization, part II: persistently exciting finite differences, in Proceedings of the IEEE Conference on Decision and Control, 2, pp. 118 123. [21] S. Yakowitz, P. L Ecuyer, and F. Vazquez-Abad, Global stochastic optmization with low-dispersion point sets, Oper. Res., vol. 48, no. 6, pp. 939 95, 2. [22] V. Fabian, Stochastic approximation, in Optimizing Methods in Statistics. Academic Press, 1971, pp. 439 47. [23] J. C. Spall, Adaptive stochastic approximation by the simultaneous perturbation method, IEEE T. Automat. Contr., vol. 45, no. 1, pp. 1839 1853, 2. [24] W. H. Moase, C. Manzie, and M. J. Brear, Newton-like extremumseeking for the control of thermoacoustic instability, IEEE T. Automat. Contr., In press. [25] H. Khalil, Nonlinear systems, 3rd ed. Prentice Hall, 22. 3844