Sabiliy and Convergence in Adapive Sysems * Margarea Sefanovic, Rengrong Wang, and Michael G. Safonov Universiy of Souhern California, Los Angeles, CA 989-563 Absrac - Sufficien condiions for adapive conrol o ensure sabiliy and convergence o a conroller ha is robusly sabilizing and performing are developed, provided ha such a conroller exiss in he candidae conroller pool. he resuls can be used o inerpre any cos-minimizing adapive scheme. An example of how a recenly developed adapive swiching mehod can fail o selec a sabilizing conroller is presened, and a correcion is proposed. Key words: adapive conrol, sabiliy, convergence, robusness, unfalsified conrol, model-free, learning I. INRODUCION Adapive conrol algorihms aim o achieve sabiliy and performance goals by using real-ime experimenal o change conroller parameers or, more generally, o swich among a given pool of candidae conrollers. A good adapive conrol algorihm mus have he abiliy o reliably deec when an acive conroller is failing o mee sabiliy and performance goals, else he algorihm canno be guaraneed o converge. ypically, adapive heories achieve convergence objecives by resricing aenion o plans assumed o saisfy assumpions, e.g., he well known bu difficul-o-saisfy sandard assumpions of adapive conrol []. he use of sandard assumpions has been widely criicized, and recen progress in adapive conrol has focused on swiching adapive conroller schemes ha eliminae he mos roublesome assumpions on he plan (e.g., [5], [6], [7], [8]). here are even some algorihms for which convergence can be assured wih essenially no assumpions on he plan, including a sochasic rial-anderror swiching mehod of Fu and Barmish [] and a mixed-sensiiviy unfalsified conrol algorihm of sao and Safonov [4]. hese algorihms are driven. hey have he abiliy o experimenally deec conrollers ha fail o mee goals wihou prior knowledge of he plan. If a candidae conroller is available ha mees performance and sabiliy goals, hese -driven swiching algorihms reliably converge o a conroller ha mees sabiliy and performance objecives. * Research suppored in par by AFOSR Gran F496-- - 3 Corresponding auhor: M. G. Safonov, msafonov@usc.edu. Email addresses: {msefano, rengronw, msafonov}@usc.edu. In his paper, we sudy he sabiliy and convergence of -driven adapive conrol sysems, wih a view owards idenifying and generalizing he properies ha disinguish hose adapive algorihms ha consisenly and reliably idenify conrollers ha achieve sabiliy and performance goals. We shall develop a model-free crierion for cosminimizaion based adapive algorihms o converge o a conroller ha sabilizes he sysem and achieves specified performance goal whenever such conroller exiss in he candidae pool. We adop he falsificaion paradigm proposed in [] and advanced in [], [4], [8] for deciding how he conrollers are seleced from he pool. A feaure of hese direc driven mehods is he inroducion of he concep of a ficiious reference signal ha plays a key role in eliminaing he burden of exhausive on-line search over he candidae conrollers ha was presen in he earlier direc swiching adapive algorihms [], [5]. In his regard, he ficiious reference signal is analogous o he plan-model idenificaion error signal of muli-model adapive swiching mehods like [6], [7]. he key idea behind he falsificaion paradigm as applied in [], [], [4] is o associae a -driven cos funcion wih each conroller model in he candidae pool. In a sense, any adapive algorihm can be associaed wih a cos funcion ha i minimizes. he very fac ha an algorihm chooses a conroller based on implies ha he algorihms orders conrollers based on. he ordering iself defines such a cos funcion. For example, recenly proposed swiching adapive schemes [7], [8], [] associae candidae conrollers wih candidae plan models, and order hese conrollers according o how closely is associaed plan model fis measured plan. he measure of model closeness o is he -driven cos funcion wih respec o which he muli-model adapive conrol (MMAC) mehods of [6], [7], [] are opimal. Convergence for MMAC algorihms is assured by assuming he rue plan is sufficienly close o he idenified model so ha i is wihin he robusness margin of is associaed conroller model. In he absence of he sufficien closeness assumpion on he plan, hese may no necessarily converge or even provide sabilizaion for he rue plan. In he presen paper, we shall derive planassumpion-free condiions under which sabiliy and performance are guaraneed. he paper is organized as follows. In Secion II, he fundamenal noions relaed o he problem we deal wih
are inroduced [], [], and some basic resuls from he sabiliy heory of adapive conrol are discussed. he problem formulaion is presened in Secion III. In Secion IV, our main resuls are saed giving plan-model-free sufficien condiions on he cos funcion for sabiliy and convergence of opimal -driven adapive mehods. An example of an opimal MMAC swiching algorihm is given in he Secion V. A weakness of he MMAC algorihm in failing o idenify and correc unsable behavior is demonsraed, and a proposed correcion is produced based on our Proposiion. he paper concludes wih some remarks in Secion VI. II. PRELIMINARIES A few noions from he behavioral heory of dynamical sysems are recalled nex [3]. We review some of he relevan noions for he problem of -driven discovery of conrollers ha fi conrol goals, as oulined in [], [3], [4] and [3]. A given phenomenon (plan, process) produces elemens (oucomes) ha reside in some se Z (universum). A subse B Z (behavior of he phenomenon) conains all possible oucomes. he mahemaical model of he phenomenon is he pair (Z, B). Se denoes an underlying se ha describes he evoluion of he oucomes in B (usually, he ime axis). We disinguish beween manifes variables (z manifes Z) ha describe explicily he behavior of he phenomenon, and laen (auxiliary) variables (z laen Z); e.g., plan inpu and oupu may serve as he manifes ( ( u, y) L L Z ). e e In his conex, we define he linear runcaion operaor P : Z Z as: zmanifes () ( Pz )( ) = oherwise his definiion differs slighly from he usual definiion of he runcaion operaor (cf. [4]) in ha he runcaion is performed wih respec o boh ime and signal vecor z. Measured se [], [3] conains he observed (measured) samples of he manifes plan : { z } {( y, u )} = B prue, where is he behavior of he rue plan. he acually available plan a ime is P ( z ) P (B ). p rue B prue Se K denoes a finie se of candidae conrollers. he ficiious reference signal r ( K, P z, ) is he reference signal ha would have exacly reproduced he measured signals P ( z ) had he conroller K been in he loop when he was colleced. Almos any adapive conrol algorihm associaes a suiably chosen cos funcion wih a paricular conroller ha minimizes his cos. In muliple-model/mulipleconroller swiching scheme, his funcion has a role of ordering candidae conrollers according o he chosen crierion. A -driven cos-minimizaion paradigm used here implies ha he ordering of he conrollers is based on he available plan. herefore, he cos (call i V) admis he following definiion: Definiion. he cos funcional V is a mapping: V: P Z K R + for he given conroller K, measured P z P Z and. he cos V represens he cos ha would be incurred had he conroller K been in he loop when Pz was recorded. Definiion. he rue cos V : K R { } as: rue + Vrue ( K): = sup V( K, P z, ) z B, is defined prue he rue cos represens, for each K, he maximum cos ha would be incurred if we had a chance o perform an infinie duraion experimen, for all possible experimenal. hus, V rue is an absrac noion, as i is no acually known a any finie ime. Noe: his definiion implies ha a any ime he curren unfalsified cos V is upper-bounded by he rue cos V rue. However, for some conroller K boh he rue cos and he unfalsified cos can have infinie values (his is he case when K is desabilizing); hus, i should be undersood ha V may no have a finie-valued bound when K is no sabilizing and unsable behaviors are excied. Le[ y, u ] represen he oupu signals of he supervisory feedback adapive sysem Σ: L e L e in Fig. -. Figure -: Supervisory feedback adapive conrol sysem Σ hroughou he paper, we make he assumpion ha all componens of he sysem under consideraion have zero inpu zero oupu propery. Definiion 3. (Sabiliy): A sysem wih inpu w and oupu z is said o be sable if w L, w, lim sup Reference inpu r () Algorihm Curren conroller K ˆ () K Disurbance sensor u() z w < ; if, in addiion, e Plan Noise sensor y()
sup w L e, w ( ) z w <, he sysem is said o be finie- gain sable; oherwise, i is said o be unsable. Specializing o he sysem in Fig. -, sabiliy means: [ ] lim sup yu, r <, r L, r. e Definiion 4. A robusly sabilizing and performing conroller K is a conroller ha sabilizes he given plan and minimizes he rue cos V rue. herefore, K = arg min ( V rue ( K )). Noe ha K is no necessarily unique. III. PROBLEM FORMULAION he problem we pursue in his sudy can be formulaed as follows: Derive he plan-assumpion-free condiions under which sabiliy of he adapive sysem and convergence of he adapive algorihm are guaraneed. Definiion 5. A -driven adapive conrol law is an algorihm ha selecs a each ime a conroller ˆK dependen on experimenal. Noe: here are differen ways of acually choosing a conroller (see e.g. [], [4]). he selecion algorihm used in his paper is he ε - cos minimizaion algorihm defined as follows. he algorihm oupus, a each ime insan, a conroller ˆK which is he acive conroller in he loop. Algorihm.. Iniialize: Le =, =; choose ε >. Le Kˆ. +. K be he firs conroller in he loop. ( ˆ,, ) > min V( K, Pz, ) If V K Pz + ε ( ) hen and Kˆ arg min V K, Pz, 3. Kˆ ˆ ; ˆ K reurn K; 4. go o.; (3-); ime insan is he ime of he las conroller swich. he swich occurs only when he curren unfalsified cos relaed o he currenly acive conroller exceeds he minimum (over all K) of he curren unfalsified cos by a leas ε. Here, ε serves o limi he number of swiches o a finie number, and so prevens he possibiliy of limi cycle ype of insabiliy ha may occur when here is a coninuous swiching beween wo or more sabilizing conrollers. I also ensures a non-zero dwell ime beween swiches. hroughou he res of he paper we will have he following sanding assumpion. I is much less resricive han he socalled sandard assumpions from he radiional adapive lieraure (e.g. knowledge of he plan relaive degree, high frequency gain, LI minimum phase plans ec. []) or even he assumpions made in he recen works on supervisory swiching MMAC mehods [6], [7], [8] (assumpion ha he real plan is sufficienly close o a model in he assumed model se). In fac, he following assumpion is inherenly presen in all oher adapive schemes, and i is minimal, provided ha we do no consider conrol laws such as diher conrol o be in he candidae se. Assumpion. he candidae conroller se K conains a leas one robusly sabilizing and performing conroller. he performance cos funcional V is chosen o have he following propery: Propery. (Monoone non-decreasing cos propery): For all, such ha : K K, z wih which K is consisen : V( K, P z, ) V( K, P z, ) Noe: When V is monoonically non-decreasing in ime, is opimal (minimal) value min V ( K, Pz, ) is monoonically non-decreasing in ime and uniformly bounded above for all z Z by V rue (K ) : P z P z min V( K,, ) min V( K,, ), > Definiion 6. (Unfalsified sabiliy): Given and measured [, ] y u we say ha he sabiliy of he sysem given in Fig. - is falsified if [ y, u ] r( K, z ) such ha lim sup =. r Oherwise, i is said o be unfalsified. Definiion 7. A sysem is said o be cos deecable if, whenever sabiliy of he sysem in Fig. - is falsified by z = y,u, hen ( K P ( ) ) lim V, z, =. Noe: he definiion says ha he unsable behavior associaed wih a non-sabilizing conroller K K leads o unboundedness of he cos funcion V. Definiion 8. (Sufficien Richness): We say he sysem inpu is sufficienly rich if K, lim max V ( K, P z, ) ( V rue ( K ) : = min rue ( )) V K z Σ K K Essenially, sufficien richness of he sysem inpu is necessary bu no sufficien o ensure cos convergence of an adapive conrol algorihm in he following sense:
lim Vrue ( K, P z, ) = Vrue ( K ), K K. A sufficienly rich inpu conains enough frequencies o excie he unsable dynamics of he sysem and hus increase he unfalsified cos V. IV. RESULS Le he Assumpion hold. Proposiion. Consider he feedback adapive conrol sysem from Fig. -. If he associaed cos funcion has he properies of cos-deecabiliy, monoone non-decrescence in ime, and uniform boundedness from above by he rue cos V : { } K R + for all plan, hen he rue swiching adapive conrol Algorihm will always converge wih finiely many conroller swiches and yield unfalsified sabiliy of he closed loop sysem saisfying (, ( ), ) [, ] V K P z V rue (K ) for all, where =. Moreover, if he sysem inpu is z y u sufficienly rich, he sequence of opimal unfalsified coss V( K ˆ, ) will converge o V rue (K ) ± ε. Proof. Le he curren conroller in he loop a ime be ˆK. Le z [,, y ]. Suppose he sabiliy of = r u B p rue he closed-loop sysem wih ˆK in he loop is falsified by he [ u, y ] ( ) ( r [ y u ] such ha lim sup, / r = ). Due o he cos-deecabiliy propery of V, lim V Kˆ, P [, u, y ] =. In paricular, for some ( ) >, V ( K ˆ, P [ u y ]) V rue K > ( ) + ε (due o (3-) in Algorihm ). Hence he conroller ˆK mus swich before ime and he unfalsified cos V( K ˆ, ) mus exceed min V( K, ) by a leas ε by he ime of he swich: V ( Kˆ, P [ u y ]) > min V ( K, P [ u y ]) + ε. If each conroller is swiched exacly or imes, hen we rivially have finie number of swiches (since K is finie). If a leas one conroller is swiched more han once (e.g. ˆK swiched a and laer, a ), hen due o Algorihm he difference in he minimal cos beween wo consecuive swiches mus be greaer han ε (recall monooniciy of he cos increase), V Kˆ, P [ u y ] > V Kˆ, P u y +. ( ) ( [ ]) ε Since min V( K, ) is bounded above by ( ) V rue K, he number of swiches o he same conroller is upperbounded by V rue ( K ) ε, which is finie. Since N card ( K) <, he overall number of swiches is upperbounded by ( ) ( N + ) Vrue K ε. Noe ha a any ime, a conroller swiched in he loop can remain here for an arbirarily long ime alhough i is differen from K. However, if he sysem inpu is sufficienly rich so as o increase he cos more han ε above he level V K ˆ, a he ime of he las swich, a ( ) swich o a new conroller ha minimizes he curren cos V ( ˆ K, ) will evenually occur a some ime >. According o Propery, he values of hese cos minima a any ime are monoone increasing and bounded above by V rue (K ). Sufficien richness will affec he cos o approach V (K ) ± ε. rue For finie ε, we always have guaraneed convergence o K RPS afer a finie number of seps. In pracice, i may suffice o use ε= so ha swiching and adapaion can occur coninuously. However, in his case he condiions of Proposiion are no longer saisfied and sabiliy of he adapive sysem is no longer guaraneed. V. EXAMPLE AND DISCUSSION Here, we presen an example ha shows how he adapive conrol mehod using fixed muliple models [9], [6], [] may fail o sabilize he plan if some of he condiions of Proposiion do no hold, even if here is a sabilizing conroller among he candidae conrollers. he swiching Algorihm wih a cos funcion obeying he condiions of Proposiion succeeds in finding a sabilizing conroller.. In adapive conrol mehod using fixed muliple models, here is a group of N candidae plan models P i, i {,...N}, wih corresponding candidae conrollers C i,i {,...N}, designed for he unknown plan W p (s). he C i s are designed so as o mee he conrol objecive of he corresponding candidae plan models. he candidae plan model, which bes represens he acual plan (has he leas cos value), is idenified a each insan and he corresponding conroller is swiched ino he loop. In he following example, he srucure of plan models and conrollers are he same as in [6] wih parameers (,,, ) β β α α for plan models and (,,, k θ θ θ ) for conrollers. wo candidae plan models and heir corresponding conrollers are designed so ha heir parameers are far from hose of he rue plan P * and is corresponding conroller C *. hese parameers are lised in
able. he conrolled plan in feedback wih he conroller is shown in Fig. 5-3. he conrol specificaion is assigned via he reference model W m (s) = /(s+3), while he unknown plan is W p (s) = /(s+5). he inpu is a sep signal. he simulaions are carried ou wih a dwell ime of. sec. All iniial condiions are zero. he cos funcion J () o be minimized is, as in [6]: λ ( ) J( ) = ei ( ) + exp e j I ( ) d, j =, j where e () I (5-) is he idenificaion error and λ =.5 (λ is a non-negaive forgeing facor ha deermines he weigh of pas ). Fig. 5- represens he on-line values of he cos funcion (5-) for boh idenifiers, when eiher conroller C or C is iniially in he loop. C is swiched ino he loop since i has smaller cos value han C from he very beginning. However, C is desabilizing, as can be confirmed by he analysis of he sabiliy margins lised in able, whereas C is sabilizing. he adapive conrol mehod in [6] based on minimizing he cos (5-) fails o pick he sabilizing conroller in his case. he cos (5-) for boh conrollers quickly blows up regardless of which conroller is in he loop iniially. o avoid choosing a desabilizing conroller, we use he swiching Algorihm wih he following cos funcion: l e i () l + exp( λ( l )) e i ( ) d J () = max,, l (, ) l i= exp( λ( l )) r i ( ) d (5 ) where λ ( exp ) r ( ) d. r, are he ficiious i e i i reference signal and he ficiious error defined in [9]. he corresponding unfalsified cos can be calculaed as shown in equaion (5-3) a he end of he paper, where he conroller Ki is given as Ki = ki θ k i i (i=, ), and ω m is he impulse response for he reference model W m (s). he unfalsified cos (5-3) saisfies he condiions of he Proposiion. We now use Algorihm o simulae he adapive sysem described above. A ime = one of he conrollers was seleced as he iniial one and pu in he loop. he sabilizing conroller C was quickly swiched ino he loop. he parameer ε is se o.. Fig.5- shows he simulaion resul of he unfalsified cos for boh conrollers: he cos of C is much smaller han ha of C (regardless of which conroller is iniially in he loop) and hus i will be swiched ino he loop. he sabilizing conroller C is successfully chosen. Figure 5-: Cos (5-) of C and C ; MMAC mehod r() Figure 5-: Cos (5-) of C and C ; Algorihm k Wp θ Figure 5-3: Feedback conrol sysem VI. CONCLUSION In his paper we sudied he problem of sabiliy and convergence in swiching adapive conrol. Noing ha every adapive scheme is opimal wih respec o some -driven conroller-ordering cos funcion, we have examined he quesion of finding sufficien condiions on he cos funcion o ensure sabiliy and convergence of he adapive conrol sysem given he minimal assumpion ha here is a leas one sabilizing conroller in he candidae se. Essenially our main conclusion is ha if he cos funcion is seleced so ha is opimal value V( Pz ) min V( K, Pz, ) is monoonically increasing, uniformly bounded above by Vrue (K ) : = min V ( K) <, and he cos rue deecabiliy holds, hen he robus sabiliy of he adapive y p
sysem is guaraneed whenever he candidae conroller pool conains a leas one sabilizing conroller. If, in addiion, sysem signals are sufficienly rich, convergence of he cos owards V rue (K ) is guaraneed. An example showed how a ypical MMAC swiching adapive scheme can fail o recognize and remove a desabilizing candidae conroller from he feedback loop, and ha his unsable behavior can be explained in erms of he failure of he model-error cos funcion associaed wih such MMAC schemes o saisfy he convergence condiions given by our sabiliy and convergence resuls in Secion IV. Based on hese resuls, a modificaion MMAC cos funcion is proposed and demonsraed o remedy he MMAC insabiliy problem. REFERENCES [] M. G. Safonov,. sao. he unfalsified conrol concep and learning, IEEE rans. Auomaic Conrol. 4(6): 843-847 June 997. [] P. Brugarolas, M.G. Safonov. A -driven approach o learning dynamical sysems. In Proc. Of CDC, pp. 46-465, Las Vegas, NV, Dec.. [3] J. C. Willems. Paradigms and puzzles in he heory of dynamical sysems. IEEE rans. Auomaic. Conrol, 36(3): 59-94, March 99. [4]. sao. Se heoreic Adapor Sysems. PhD hesis, Universiy of Souhern California, May 994. [5] B. Marensson. he order of any sabilizing regulaor is sufficien informaion for adapive sabilizaion. Sysem & Conrol Leers, 6: 87-9, July 985. [6] K.S. Narendra, J. Balakrishnan. Adapive conrol using muliple models. IEEE rans. Auomaic Conrol, 4():7-87, February 997. [7] P. Zhivoglyadov, R. H. Middleon, M. Fu. Localizaion based swiching adapive conrol for ime-varying discree-ime sysems. IEEE rans. Auomaic Conrol, 45 (4):75-755, April. [8] E. Mosca and. Agnoloni. Inference of candidae loop performance and filering for swiching supervisory conrol. Auomaica, 37(4):57-534, April. [9] A. Paul and M.G. Safonov. Model reference adapive conrol using muliple conrollers and swiching. Proc. IEEE Conf. on Decision and Conrol. Maui, HI, Dec. 9-, 3, o appear. [] K.S. Narendra and J. Balakrishnan. Improving ransien response of adapive conrol sysems using muliple models and swiching. IEEE rans. Auomaic Conrol, 39, pp.86-866, 994. [] K. S. Narendra and A.M. Annaswamy. Sable Adapive Sysems. NY: Prenice Hall, 989. [] M. Fu and B. R. Barmish. Adapive sabilizaion of linear sysems via swiching conrol", IEEE rans. Auomaic Conrol 3():97-3, December, 986. [3] M. G. Safonov and F. B. Cabral. Fiing conrollers o Sysems and Conrol Leers, 43(4):99-38, July. [4] M. G. Safonov. Sabiliy and robusness of mulivariable feedback sysems. Cambridge, Massachuses. MI Press, 98. y() l l y( ) y () l ω K i exp( λ( l )) y ( ) ω K m m i d u() l + u ( ) V( K, P y, u, ) max i = l (, ) l y ( ) exp( λ( l )) Ki d u ( ) (5-3) Parameers of Plan Models β β α α able : Parameers of plan, models and conrollers Parameers of Conrollers k θ θ θ Sabiliy Analysis of he Closed Loop Sysem Open Loop F W p θ ) ( P * - C * Sys * -/(s+5) 7.96 Inf P 4 C.5 - Sys /(s+5) Inf Inf GM (db) PM (deg) P -6 C 6 Sys -6/(s+5) -.58-33.5