An extremum seeking approach to parameterised loop-shaping control design

Transcription

1 Preprints of the 19th World Congress The International Federation of Atomatic Control An extremm seeking approach to parameterised loop-shaping control design Chih Feng Lee Sei Zhen Khong Erik Frisk Mattias Krysander Division of Vehiclar Systems, Department of Electrical Engineering, Linköping University, SE Linköping, Sweden. ( Department of Atomatic Control, Lnd University, SE-221 Lnd, Sweden. ( Abstract: An approach to loop-shaping feedback control design in the freqency domain via extremm seeking is proposed. Both plants and controllers are linear time-invariant systems of possibly infinite dimension. The controller is assmed to be dependent on a finite nmber of parameters. Discrete-time global extremm seeking algorithms are employed to minimise the difference between the desired loop shape and the estimate of the present loop shape by finetning the controller parameters within a sampled-data framework. The sampling period plays an important role in garanteeing global practical convergence to the optimm. A case stdy on PID control tning is presented to demonstrate the applicability of the proposed method. Keywords: Global extremm seeking, loop-shaping, parameterised controller, freqency response estimation 1. INTRODUCTION Designing a robst controller with good reference tracking and distrbance rejection performances in the face of modelling mismatch can be a challenging task. One renowned control synthesis techniqe for sch a task is loop-shaping which, when properly tilised, achieves robst feedback stability and performance for a given linear time-invariant (LTI) plant (Doyle et al., 1992; McFarlane and Glover, 1992; Vinnicombe, 21). A reqirement to applying the loop-shaping techniqes is the availability of plant models. Nevertheless, sefl plant models are often not known in many practical applications. This paper attempts to address this isse via the se of extremm seeking control (Ariyr and Krstić, 23; Killingsworth and Krstic, 26; Zhang and Ordóñez, 211). It is demonstrated that the global extremm seeking framework of Khong et al. (213b) is sited for performing parameterised loop-shaping controller design for singleinpt single-otpt (SISO) plants. Both the plant and compensator are allowed to be infinite-dimensional (Crtain and Zwart, 1995) for example, modelled by partial differential eqations or contain time-delay terms. The compensator is presmed to be a fnction of a finite nmber of parameters and extremm seeking control is employed to drive the set of parameters to one that gives rise to a desired loop shape. By probing the loop or retrn ratio with appropriate signals and collecting the corresponding otpt measrements, the loop transfer fnction within the interested freqency range can be estimated. This can then be tilised for controller tning within a periodic sampled-data framework. In particlar, extremm seeking can be carried ot with the cost fnction to be minimised being some distance measre between the achieved and desired loop shape in the freqency domain withot exploiting models of the plant and controller. In the proposed loop-shaping procedre, the loop transfer fnction is estimated via the se of the Forier transform, which in trn is approximated sing the Fast Forier Transform (FFT). This inherently introdces errors to the estimation process. It is shown that nder certain robstness condition on the extremm seeking algorithm, global practical convergence to the desired loop shape is achievable by adjsting the ser-designed sampling period. To illstrate the reslts, a case stdy involving the selftning of PID compensator parameters to attain a desired loop shape is presented. This paper evolves along the following lines. First, the general class of systems considered in (Khong et al., 213b) is reviewed in Section 2. Extremm seeking algorithms are discssed in Section 3 and they are established to give rise to non-local practical convergence to global extrema within a sampled-data framework from (Khong et al., 213b). In Section 4, the loop-shaping control design problem is examined and established to be special cases of the class of systems in Section 2, depending on the type of freqency response estimation methods sed. Extremm seeking methods presented in Section 3 are then shown to be applicable to the loop-shaping problem. In Section 5, the theoretical developments are frnished with a case stdy in self-tning PID controller design. 2. DYNAMICAL SYSTEMS It is demonstrated in Khong et al. (213b) that the extremm seeking framework therein can accommodate a rich class of nonlinear time-invariant systems that are pos- Copyright 214 IFAC 1251

2 sibly distribted-parameter and may admit complicated attractors. These are briefly reviewed in this section. To begin with, the following notation is introdced. A fnction γ : R R is of class-k (denoted γ K) if it is continos, strictly increasing, and γ() =. If γ is also nbonded, then γ K. A continos fnction β : R R R is of class-kl if for each fixed t, β(, t) K and for each fixed s, β(s, ) is decreasing to zero (Khalil, 22). The Eclidean norm is denoted. Given any scalar transfer fnction X that is continos on jr { }, let X := sp ω R X(jω). Let X be a Banach space whose norm is denoted. Given any sbset Y of X and a point x X, define the distance of x from Y as x Y := inf a Y x a. Also let U ɛ (Y) := {x X x Y < ɛ}. Definition 2.1. Let the state of a time-invariant dynamical system be represented by x : R X, where X is a Banach space with norm. The inpt to and otpt of the system are denoted, respectively, by : R Ω R m and y : R R. The set Ω denotes the inpt space of interest, and is taken to be a compact sbset of R m. Compactness is not a stringent assmption given the biqity of control inpt satration constraints in physical systems (Khalil, 22). Given any Ω and x X, let x(, x, ) be the state of the dynamical system starting at x with inpt. Assmption 2.2. Given a system described in Definition 2.1, the following hold: (i) There exists a fnction A mapping from Ω to sbsets of X sch that for each constant Ω, A() is a nonempty closed set and a global attractor (Relle, 1989): (a) Given any x X and ɛ >, there exists a sfficiently large t > sch that x(t, x, ) U ɛ (A()); (b) If x(t, x, ) A(), then x(t, x, ) A() for all t t ; (c) There exists no proper sbset of A() having the first two properties above. Frthermore, sp sp x <. (1) Ω x A() (ii) For each triplet (ɛ 1, ɛ 2, ) of strictly positive real nmbers, there exists a waiting time T > sch that if x A(), x(t, x, ) A() ɛ 1 x A() + ɛ 2 t T, Ω. (iii) There exists a locally Lipschitz fnction h : X R sch that the system otpt y(t) = h(x(t, x, )) t for any constant inpt Ω and x X. Moreover, h(x a ) = h(x b ) for every x a, x b A(). Since A() is a global attractor and h is locally Lipschitz, for any Ω and x X, Q() := lim h(x(t, x, )) ( ) = h lim x(t, x, ) = h(x l ), for some x l A() is a well-defined steady-state inpt-otpt map that is Lipschitz on Ω. (iv) Q takes its global minimm vale in a nonempty, compact set C Ω. Examples of systems satisfying Assmption 2.2 inclde finite-dimensional state-space systems with eqilibria or periodic attractors; see Khong et al. (213b) for more details. As it will be shown in Section 4, the parameterised loop-shaping controller synthesis is formlated as an extremm seeking problem involving a plant satisfying Assmption GLOBAL EXTREMUM SEEKING While traditional extremm seeking methods are often only able to locate local steady-state optima (Ariyr and Krstić, 23; Teel and Popović, 21) nless the nderlying objective fnction is convex, Khong et al. (213b) establishes that within its general extremm-seeking setting, a wide range of deterministic nonconvex discrete-time optimisation algorithms may be employed for global extremm seeking, inclding the DIRECT method (Khong et al., 213a) and Shbert method (Nešić et al., 213). Sch algorithms are interconnected with the plant via synchronised sampler and zero-order hold. Reslts from Khong et al. (213b) are smmarised in this section and then applied to the parameterised loop-shaping control problem in the next. 3.1 Optimisation Consider the optimisation problem: y := min Q(), (2) Ω where Q : Ω R m R is a Lipschitz continos fnction which takes its global minimm vale on C Ω, i.e. Q() = y for all C. Let Σ be a discrete-time extremm seeking algorithm for (2). The otpt seqence Σ generates to probe Q is denoted { k } k= and in the case of precise (i.e. noiseless) sampling, the collected measrements are y k = Q( k 1 ), k = 1, 2,.... Define also the seqence ȳ N := min y k. (3) k=1,...,n Let ˆδ be a non-negative real nmber. It follows from the above definition that the seqence {ȳ k } k=1 converges to the closed ˆδ-neighborhood (ˆδ-ball) of y if for all ɛ >, there exists infinitely many N N sch that N { Ω Q() y ˆδ + ɛ}. By the Lipschitz continity of Q, corresponding to the ˆδ above, there exists a δ > sch that the aforementioned condition holds if a sbseqence of { k } k= converges to the δ-neighborhood of C. That is, for all ɛ > and K N, there exists an N N sch that N > K and N C + (δ + ɛ) B, (4) where B denotes the closed nit ball in R m. In the presence of bonded additive pertrbations on the measrements as illstrated in Figre 1, i.e. y k = Q( k 1 ) + w k with w k ν for some ν >, the following assmption is important to establish convergence of the sampled-data extremm seeking scheme to be considered in Section

3 Q + w k Plant y k 1 Extremm Seeking Algorithm Σ y k ZOH (t) = k t [kt, (k + 1)T ) Sampler y k = y(kt ) Fig. 1. Extremm seeking algorithm with noisy otpt measrement. k 1 Extremm Seeking Algorithm Σ y k Assmption 3.1. Let δ be a small nmber which characterises the accracy of convergence as in (4). The discrete-time extremm seeking algorithm Σ satisfies the following: Given any µ > δ, there exists a ν > sch that if ŷ k Q(û k 1 ) ν for k = 1, 2,... and any seqence {û k } k= Ω, then for every ɛ > and K N, there exists an N N, N > K for which N C + (µ + ɛ) B. In other words, the a sbseqence of the otpt of Σ, { k } k= converges to a µ-neighborhood of the set C of global minimisers of Q. Remark 3.2. Amongst others, the DIRECT algorithm (Jones et al., 1993) is an example which satisfies Assmption 3.1 (Khong et al., 213a). Operating on a compact bondconstrained mlti-dimensional domain of search Ω := { R m i [a i, b i ] R, i = 1, 2,..., m}, it is intelligently balanced between local and global search. The trial points DIRECamples in the inpt space always form a dense sbset, whereby an otpt sbseqence of DIRECT converges to a global extremm. This algorithm will be sed for loop-shaping control design later in the paper. 3.2 Extremm seeking The main sampled-data extremm seeking framework based on Khong et al. (213b) is detailed in the following. Semi-global practical convergence to global extrema is established. Let { k } k= be a seqence of vectors in Ω and define the zero-order hold (ZOH) operation (t) := k for all t [kt, (k + 1)T ) (5) and k =, 1, 2,..., where T > denotes the sampling period or waiting time. Frthermore, let the state and otpt of a dynamical system in Definition 2.1 with respect to the inpt be respectively x and y and define the ideal periodic sampling operation x k := x(kt ); y k := y(kt ) for all k = 1, 2,.... (6) Figre 2 shows an extremm seeking scheme based on a sampled-data control law with period T. The following lemma on dynamical systems is needed to establish the main reslt of this section. Lemma 3.3. (Khong et al. (213a)). Given any dynamical system described in Definition 2.1 that satisfies Assmption 2.2, >, and ν >, there exists a T > sch that for any { k } k= Ω and x A(), y k Q( k 1 ) ν for all k = 1, 2,..., where y k is as in (6) with y being the otpt of the system for the inpt given by (5). Fig. 2. Sampled-data extremm seeking control. The main extremm seeking convergence reslt is stated next. The feedback configration in Figre 2 of a dynamical plant satisfying Definition 2.1 and Assmption 2.2 and an extremm seeking algorithm Σ satisfying Assmption 3.1, interconnected throgh a T -periodic sampler (6) and a synchronised zero-order hold (5), has the following convergence property. Theorem 3.4. Given any (, µ) sch that, µ > δ, where δ is given in Assmption 3.1, there exists a sampling/waiting period T > sch that for any x A(), a sbseqence of { k } k= converges to C + µ B, where C is the set of global minimisers for Q : Ω R m R, the steady-state inpt-otpt map of the plant, as in Assmption 2.2. Proof. By Assmption 3.1, there exists a ν > sch that if the inpt to Σ, ŷ satisfies ŷ k Q(û k 1 ) ν (7) for k = 1, 2,... and any {û k } k= Ω, then Σ generates an otpt { k } k= of which a sbseqence converges to C + µ B. Frthermore, Lemma 3.3 ensres the existence of a sampling period T > sch that the above-mentioned condition (7) holds for any initial plant s state x A(). 4. LOOP-SHAPING CONTROL DESIGN In this section, two approaches to parameterised loopshaping control synthesis problem are shown to satisfy Definition 2.1 and Assmption 2.2. Sampled-data extremm seeking methods in Section 3 are then applied to perform the task of loop-shaping. Tning gidelines with regards to the sampling period are provided to garantee global practical convergence to the desired loop shape. 4.1 Estimation based on implse response Let C() : y c c denote a SISO LTI controller with a transfer fnction that is continos on jr { }, and is Lipschitz continos in parameter Ω R m. C() can be distribted-parameter (Crtain and Zwart, 1995), or may admit a finite-dimensional state-space realisation: ẋ c = A c ()x c + B c ()y c, x c () = c = C c ()x c + D c ()y c, sch that eig(a c ()) jr = for all Ω. Let P : p y p be a SISO LTI plant that is possibly infinite-dimensional and admits a transfer fnction that is continos on jr { }. Consider now the retrn ratio or loop gain, L() := P C() : y c y p. 1253

4 In this paper, loop-shaping control design involves finding a Ω sch that L() has a desired shape. The loopshaping paradigm is a well-known method for synthesising a controller which gives rise to non-conservative closedloop robstness and performance; see (McFarlane and Glover, 1992; Vinnicombe, 21). In general, the magnitde of the loop gain is chosen to be large at freqencies where tracking performance or otpt distrbance rejection is important, i.e. sensitivity attenation is reqired, and small at freqencies where measrement noise or modelling ncertainty cold pose a problem, i.e. complementary sensitivity redction is reqired. These often occr in low freqency range for the former and high freqency range for the latter. A gentle slope of the loop is also reqired at crossover freqency to ensre internal stability by the Bode s phase formla (Doyle et al., 1992, Chapter 7). For any fixed Ω, the freqency response of the loop may be estimated by applying an implse y c to the system and taking the Forier transform F of its corresponding otpt/implse response y p, i.e. Let L()(jω) = (Fy p )(jω) = lim T x(t, ) := T T y p (t)e jωt dt. y p (t)e jωt dt T. The vale of erves as a waiting time: the larger T is, the closer x(t, ) will be to L(). Note that x() = x =. It follows that there exists a β KL sch that for any Ω, x(t, x, ) L() = x(t, x, ) L() β( L(), t) t. Let L d (jω) be a desired loop shape and is defined in the freqency range of interests ω [, W ]. Define for any fnction K that is continos on jr { }: K W := sp K(jω) L d (jω). ω [,W ] For the prpose of extremm seeking, let the plant inpt be and otpt be y(t) := x(t, x, y c ) W for t ; see Figre 3. As sch, Q() := lim x(t, x, y c ) W = sp ω [,W ] L()(jω) L d (jω) is a well-defined steady-state inpt-otpt map that is Lipschitz on Ω. It is also clear that the global minimm is achieved when L() = L d. It is straightforward to verify that the nonlinear, possibly distribted-parameter plant mapping from to y satisfies Assmption 2.2, as reqired. In view of this, the following corollary to Theorem 3.4 is in order. Corollary 4.1. Consider the feedback interconnection in Figre 2 of a dynamical plant y described by Figre 3 and an extremm seeking algorithm Σ satisfying Assmption 3.1. Given any µ > δ, where δ is the accracy of Σ as in Assmption 3.1, there exists a sampling/waiting period T > sch that a sbseqence of { k } k= Ω converges to + µ B, where satisfies L( ) = L d, a prescribed loop shape. Note that the period T in both Figres 2 and 3 are the same. 4.2 Estimation based on freqency sweep Now sppose one is only interested in estimating the magnitdes of the freqency response of L() at freqencies {ω 1, ω 2,..., ω N }. Then the inpt y c can be defined to take the form y c := a 1 sin(ω 1 t) + a 2 sin(ω 2 t) a N sin(ω N t), where a 1,..., a N >. That is, y c sweeps constantamplitde pre-tones/harmonics throgh the bandwidth of interest. By the sperposition principle of linear systems, it holds that y p = b 1 () sin(ω 1 t + φ 1 ) b N () sin(ω N t + φ N ), for some b 1 (),... b N () R and φ 1 (),..., φ N () R. Ths, L(jω i ) is given by b i ()/a i, for i = 1,..., N. Define l() := [b 1 ()/a 1,..., b N ()/a N ] T, (8) x(, ) :=, and { T x(t, ) := y } p(t)e jωit dt T y c(t)e jωit dt i = 1,..., N T >. It follows that there exists a β KL sch that for any Ω, x(t, x, ) l() = x(t, x, ) l() 2 β( l() 2, t) t. As before, let L d (jω) be a desired loop shape and define S := [ L d (jω 1 ),..., L d (jω N ) ] T. Let the targeted plant for extremm seeking with inpt have the otpt y(t) := x(t, x, ) S for t ; see Figre 4. Notice that Q() := lim x(t, x, y c ) S = l() S 2 is a well-defined steady-state inpt-otpt map that is Lipschitz on Ω. It achieves its global minimm when l() = S. As sch, the nonlinear plant mapping from to y flfills the conditions in Assmption 2.2. Similarly to the previos sbsection, one has the following corollary to Theorem 3.4. y c C() P y p T y p(t)e jωit dt T y c(t)e jωit dt x S y y c C() P y p T e jωt x dt W Fig. 3. Dynamical plant y for extremm seeking constrcted from implse response based estimation. y Fig. 4. Dynamical plant y for extremm seeking constrcted from freqency sweep based estimation. Corollary 4.2. Consider the feedback interconnection in Figre 2 of a dynamical plant y described by Figre 4 and an extremm seeking algorithm Σ satisfying Assmption 3.1. Given any µ > δ, where δ is 1254

5 Parameterised Controller Tner Loop Gain Estimator 15 L d Extremm Seeking Algorithm 1 r Signal Generator + e c C(s, ) P (s) y p Fig. 5. Parameterised controller tning sing an extremm seeking framework. the accracy of Σ as in Assmption 3.1, there exists a sampling/waiting period T > sch that a sbseqence of { k } k= Ω converges to + µ B, where satisfies L( ) = L d, a prescribed loop shape. 5. APPLICATION IN SELF-TUNING PID CONTROL In this section, an example in self-tning PID control design is shown, where the idea is to employ extremm seeking approach to tne PID parameters sch that the loop gain approaches a desired loop shape. The plant is LTI and of infinite dimension. A featre of the proposed scheme is that the models of controller and plant are not needed for the tning of controller parameters. A plant model is borrowed from Schei (1994), where it is a stable third-order minimm phase system with time-delay, given by 1 1s P (s) = (1 + 6s)(1 + 2s)(1 + 2s) e 1s. Note the plant is an infinite-dimensional system de to the time-delay term, e 1s. A PID controller is adopted ( C(s, ) = K p ) T i s + T d s (9) 1 + (T d /N f )s where it is parameterised by = [K p, T i, T d, N f ] T. Frthermore, the bonds of PID parameters are given in a compact set, Ω = { := [K p, T i, T d, N f ] T K p = [1, 5], T i = [1, 1], T d = [, 5], N f = [.1, 5]}. The feedback strctre of the plant and self-tning controller is depicted in Figre 5. The loop transfer fnction of the controlled system is L(s, ) = G(s)C(s, ). Sppose a desired loop shape is given by L d (s) =.33 s(s + 2), (1) and the system behavior within ω [.1,.1] Hz is of interest. Note there is no assmption made on the attainability of the desired loop shape, and exact matching to the desired loop shape may not be possible. Extremm seeking is sed to find controller parameters, Ω sch that L(s, ) be sfficiently close to L d (s) within the freqency range of interest withot sing models of plant and controller. A constant-amplitde pre-tones at freqencies {ω i = i/1 i = 1,..., 1} Hz is sed to probe the open loop transfer fnction, L. The magnitde of L is estimated Magnitde [db] Tre Magnitde = 3s = 5s = 7s = 9s = 1s = 2s Freqency [rad/s] Fig. 6. Tre and estimated magnitdes of L with different length of. Table 1. Controller parameters obtained with different waiting periods,. Waiting Period [s] K p T i T d N f sing (8), where a i are known and b i are calclated sing fast forier transform (FFT) algorithm (Cooley and Tkey, 1965). Figre 6 shows that the estimation accracy of L(s, ) improves as the data length sed for the estimation, is increased. The PID parameters are tned sing the proposed extremm seeking framework and the estimated loop shape, L, where the controller tnings are pdated every. The controller parameters obtained with different waiting periods are listed in Table 1. Note that the parameters for = 3 s are the same for = 7 s. However, Figre 7 shows that the smallest cost vale obtained by extremm seeking decreases with increasing. Comparing = 3 s and = 7 s, the decrease in cost is de to the improvement in L estimation. Figre 8 shows the loop L approaches the desired loop L d as the waiting period is increased from 3 s to 1 s, indicating that the longer waiting period improves the optimised reslt. Unit step responses obtained sing the parameters in Table 1 are shown in Figre 9, where a more responsive response is obtained for = 2 s. 6. CONCLUSIONS An online loop-shaping design of parameterised controller is presented, where extremm seeking is adopted to tne the parameters. One motivation of the proposed method is the need to fine-tne parameters of a fixed strctre controller online to accommodate for continos changes in the plant and environment. The control design problem is formlated and shown to be a special class in the global extremm seeking framework (Khong et al., 213b). The proposed method is applied to self-tning PID control, 1255

6 Minimm Cost Fond = 3s = 7s = 1s = 2s Nmber of Fnction Evalations Fig. 7. Minimm cost obtained verss fnction cont for different waiting period,. Magnitde [db] L d = 3s = 7s = 1s = 2s 1 2 Freqency [rad/s] Fig. 8. Loop shape for different waiting period,. y p = 3s = 7s = 1s = 2s Time [s] Fig. 9. Unit step responses for different waiting period,. where the magnitde plot of the compensated open-loop transfer fnction is close to the desired loop shape. A novelty of the proposed method is that the models of both plant and controller are not reqired dring the tning process. Admittedly, if the controller model is exploited, the tning may be accelerated. A more general design framework inclding controller model will be considered in the ftre. Additionally, alternative methods for the estimation of loop gain will be investigated. REFERENCES Ariyr, K.B. and Krstić, M. (23). Real-Time Optimization by Extremm Seeking Control. Wiley-Interscience. Cooley, J.W. and Tkey, J.W. (1965). An algorithm for the machine calclation of complex forier series. Mathematics of comptation, 19(9), Crtain, R.F. and Zwart, H.J. (1995). An Introdction to Infinite-Dimensional Linear Systems Theory. Texts in Applied Mathematics 21. Springer-Verlag. Doyle, J.C., Francis, B.A., and Tannenbam, A.R. (1992). Feedback control theory. Macmillan Pb. Co., New York. Jones, D.R., Pertnnen, C.D., and Stckman, B.E. (1993). Lipschitzian optimization withot the Lipschitz constant. Jornal of Optimization Theory and Applications, 79(1), Khalil, H.K. (22). Nonlinear Systems. Prentice Hall, 3rd edition. Khong, S.Z., Nešić, D., Manzie, C., and Tan, Y. (213a). Mltidimensional global extremm seeking via the DI- RECT optimisation algorithms. Atomatica, 49(7), Khong, S.Z., Nešić, D., Tan, Y., and Manzie, C. (213b). Unified frameworks for sampled-data extremm seeking control: global optimisation and mlti-nit systems. Atomatica, 49(9), Killingsworth, N.J. and Krstic, M. (26). PID tning sing extremm seeking: online, model-free performance optimization. IEEE Control Systems Magazine, 26(1), McFarlane, D.C. and Glover, K. (1992). A loop shaping design procedre sing H -synthesis. IEEE Trans. Atom. Contr., 37(6), Nešić, D., Ngyen, T., Tan, Y., and Manzie, C. (213). A non-gradient approach to global extremm seeking: an adaptation of the Shbert algorithm. Atomatica, 49(3), Relle, D. (1989). Elements of Differentiable Dynamics and Bifrcation Theory. Academic Press. Schei, T.S. (1994). Atomatic tning of PID controllers based on transfer fnction estimation. Atomatica, 3(12), Teel, A.R. and Popović, D. (21). Solving smooth and nonsmooth mltivariable extremm seeking problems by the methods of nonlinear programming. In Proceedings of American Control Conference, volme 3, Vinnicombe, G. (21). Uncertainty and Feedback H loop-shaping and the ν-gap metrics. Imperial College Press, London. Zhang, C. and Ordóñez, R. (211). Extremm-Seeking Control and Applications: A Nmerical Optimization- Based Approach. Advances in Indstrial Control. Spinger. 1256