Recursive Identification

Transcription

1 Chaper 8 Recursive Idenificaion Given a curren esimaed model and a new observaion, how should we updae his model in order o ake his new piece of informaion ino accoun? In many cases i is beneficial o have a model of he sysem available online while he sysem is in operaion. The model should hen be based on he observaions up ill he curren ime. A naive way o go ahead is o use all observaions up o o compue an esimae ˆ of he sysem parameers. In recursive idenificaion mehods, he parameer esimaes are compued recursively over ime: suppose we have an esimae ˆ a ieraion, hen recursive idenificaion aims o compue a new esimae ˆ by a simple modificaion of ˆ when a new observaion becomes available a ieraion. The counerpar o online mehods are he so-called o ine or bach mehods in which all he observaions are used simulaneously o esimae he model. Recursive mehods have he following general feaures: They are a cenral par in adapive sysems where he nex acion is based on he laes esimaed parameers. Typical examples are found in adapive conrol or adapive filering applicaions. Memory and compuaional requiremens a any imesep has o be modes. Specifically, one ofen requires ha boh are independen o he lengh of he hisory a any imesep. They are ofen applied o real-ime seings, where he rue underlying parameers are changing over ime (i.e. racking applicaions). They are ofen used for faul deecion sysems. Here one wans o deec when he observed signals or he underlying sysem di ers significanly from wha one would associae from a normal operaion modus. In general, he echniques go wih he same saisical properies as heir counerpars in bach seing. For example, he RLS gives consisen esimaes under he condiions as discussed in Secion 5.3. Tha is, he discussion on he recursive esimaors is ofen concerned wih compuaional issues. 23

2 8.. RECURSIVE LEAST SQUARES 8. Recursive Leas Squares Le us sar his secion wih perhaps he simples applicaion possible, neverheless inroducing ideas. Example 50 (RLS for Esimaing a Consan) Given he following sysem y = 0 + e, 8 =, 2,... (8.) In chaper 2, example we derive how he leas squares esimae of 0 using he firs observaions is given as he arihmeic (sample) mean, i.e. ˆ = y i. (8.2) i= Now i is no oo di ˆ = cul o rewrie his in a recursive form. X y i + y = ( )ˆ + y = ˆ + i= y ˆ. (8.3) This resul is quie appealing: he new esimae ˆ equals he previous esimae ˆ plus a small correcion erm. The correcion erm is proporional o he deviaion of he predicion ˆ and he observaion y. Moreover he correcion erm is weighed by he erm, which implies ha he magniude of he correcion will decrease in ime. Insead he esimae ˆ will become more reliable. In case a proper sochasic framework is assumed (see chaper 5, secion 3), he variance of ˆ becomes which can in urn be compued recursively as = =, (8.4) + = +. (8.5) In order o generalize he resul, we need he following well-known marix properies. Lemma 9 (Marix Inversion Lemma) Le Z 2 R d d be a posiive definie marix wih unique inverse Z, and le z 2 R d be any vecor, hen where Z + = Z + zz T. Z + =(Z + zz T ) = Z Z zz T Z T +z T Z z, (8.6) 24

3 8.. RECURSIVE LEAST SQUARES In words, he inverse of a marix wih a rank-one updae can be wrien in closed form using he inverse of he marix and a small correcion. roof: The proof is insrumenal. (Z + zz T ) Z Z zz T Z Z zz T Z +z T Z = I d Z z +z T Z z zz T Z zz T Z = I d +z T Z + z +z T Z z ( + zz z) zz T Z = I d + +z T Z z Now, noe ha (z T Z z) is a scalar, and hus zz T Z = I d + +z T Z (z T Z z) z as desired. + zz T Z (zz T ) (zz T Z ) Z zz T Z +z T Z z zz T Z zz T Z +z T Z z zz T Z zz T Z +z T Z z. (z T Z z)zz T Z +z T Z = I d, (8.7) z The previous example serves as a blueprin of he Recursive Leas Squares (RLS) algorihm, which we now will develop in full. Given a model for he observaions {(x,y )} R d given as y = T 0 x + e, 8 =, 2,..., (8.8) where 0 2 R d and he erms {e } are he corresponding residuals. Then chaper 2 learns us ha he LS soluion based on he observaions x i : i =,...,will be given as he soluion o he normal equaions x i x T i ˆ = y i x i. (8.9) i= Assume for now ha he soluion ˆ is unique, i.e. he marix R = i= x ix T i can be invered. Since rivially one has R = R x x T, (8.0) if follows ha ˆ = R = R i= X y i x i + y x i= = ˆ + R R ˆ + y x y x (x x T )ˆ = ˆ + R x y ˆ T x, (8.) and in summary 8 >< =(y x T ˆ ) K = R x >: ˆ = ˆ + K. (8.2) 25

4 8.. RECURSIVE LEAST SQUARES Here he erm will be inerpreed as he predicion error: i is he di erence beween he observed sample y and he prediced value x T ˆ.If is small, he esimae ˆ is good and should no be modified much. The marix K is inerpreed as he weighing or gain marix characerizing how much each elemen of he parameer vecor ˆ should be modified by. The RLS algorihm is compleed by circumvening he marix inversion of R in each imesep. Hereo, we can use he marix inversion Lemma. R = R R x x T R +x T R x. (8.3) Noe ha as such we subsiue he marix inversion by a simple scalar division. K = R x = R x R x (x T R x ) +x T R x = +x T R x R x. (8.4) Given iniial values R 0 and ˆ 0, he final RLS algorihm can as such be wrien as 8 =(y x T ˆ ) >< = x x T +x T x K = x = >: ˆ = ˆ + K, +x T x x (8.5) where we use = R for any. For e ciency reasons, one can We will come back o he imporan issue on how o choose he iniial values 0 and ˆ 0 in Subsecion Real-ime Idenificaion This subsecion presens some ideas which are useful in case he RLS algorihm is applied for racking ime-varying parameers. This is for example he case when he rue parameer vecor 0 varies over ime, and are as such denoed as { 0, }. This seing is referred o as he real-ime idenificaion seing. There are wo common approaches o modify he RLS algorihm o handle such case: (i) use of a forgeing facor (his subsecion); (ii) use of a Kalman filer as a parameer esimaor (nex subsecion). The forgeing facor approach sars from a slighly modified loss funcion V ( ) = s= s (y T x ) 2. (8.6) The Squared Loss funcion lying a he basis of RLS is recovered when =. If is se o some value slighly smaller han (say = 0.99 or = 0.95), one has ha for increasing pas observaions are discouned. The smaller go, he quicker informaion obained from previous daa will be forgoen, and hence he name. I is now sraighforward o re-derive he RLS based 26

5 8.. RECURSIVE LEAST SQUARES on (8.6), and he modified RLS becomes: 8 =(y x T ˆ ) >< = K = x = >: ˆ = ˆ + K. x x T +x T x +x T x x (8.7) Example 5 (Esimaor Windup) Ofen, some periods of he idenificaion experimen exhibi poor exciaion. This causes problems for he idenificaion algorihms. Consider he siuaion where ' =0in he RLS algorihm, hen (ˆ = ˆ =, (8.8) Noice ha ˆ remains consan during his period,... and increases exponenially wih ime when <. When he sysem is excied again (' 6=0), he esimaion gain K will be very large, and here will be an abrup change in he esimae, despie he fac ha he sysem has no changed. This e ec is referred o as esimaor windup. Since he sudy of Kalman filers will come back in some deail in laer chapers, we rea he Kalman filer inerpreaion as merely an example here. Example 52 (RLS as a Kalman Filer) A sochasic sae-space sysem akes he form ( X + = F X + V Y = H X + W 8 =, 2, 3,..., (8.9) where {X 2 R n } denoe he sochasic saes, {Y 2 R m } denoe he observed oucomes. {V 2 R n } denoe he process noise. {W 2 R m } denoe he observaion noise. {F 2 R n n } are called he sysem marices {H 2 R m n } Now i is easily seen ha he problem of ime-invarian RLS esimaion can be wrien as ( + = Y = x T + E 8 =, 2, 3,..., (8.20) 27

6 8.. RECURSIVE LEAST SQUARES where = = = =... is he unknown sae one wans o esimae based on observaions {Y }. Hence one can phrase he problem as a filering problem, where he Kalman filer provides he opimal soluion o under appropriae assumpions, evenually reducing o (8.5). The benefi of his is ha one can exend he model sraighforwardly by including unknown process noise erms {V }, modeling he drifing rue values as a random walk - approaching e ecively he real-ime idenificaion problem. Suppose {V,...,V,...} are sampled independenly from a Gaussian wih mean zero and covariance V 2 R n n, hen he Kalman filer would become 8 =(y x T ˆ ) >< = x x T + V +x T x (8.2) K = x >: ˆ = ˆ + K. Observe ha boh in case (8.7) as in (8.2) he basic RLS algorihm is modified such ha will no longer end o zero. In his way K also is prevened from decreasing o zero. The parameer esimae will herefore change coninuously Choosing Iniial Values The choice of he iniial values is paramoun in real life applicaion of RLS schemes. Close inspecion of he meaning of helps us here. In he Kalman filer inerpreaion of RLS plays he role of he covariance marix of he esimae ˆ, as such suggesing ha in case one is no a all cerain of a cerain choice of ˆ 0, one should ake a large 0 ; if one is fairly confiden in a cerain choice of ˆ 0, 0 should be aken small. If 0 is small, so will {K } >0 and he esimae {ˆ } will no change oo much from ˆ 0. If 0 would be large, ˆ will quickly jump away from ˆ 0. Wihou a priori knowledge, i is common pracice o ake he following iniial values (ˆ =0d (8.22) 0 = I d, wih I d = diag(,...,) 2 R d d he ideniy marix, and >0 a large number. The e ec on he choice of he iniial values (or he ransien behavior) can be derived algebraically. Consider he basic RLS algorihm (8.5). Then Now se Then z = R ˆ +x = R + x x T And hence ˆ ˆ = z = R 0 + R = R 0 + x x. (8.23) s= z = R ˆ. (8.24) +x y ˆ T x = z +x y = z 0 + x s x T s s= 28 R 0 ˆ 0 + x s y s. (8.25) s= x s y s. (8.26) s=

7 8.. RECURSIVE LEAST SQUARES So, if R 0 is small (i.e. 0 is large), hen ˆ is close o he o ine esimae = argmin (y s T x ) 2, (8.27) s= as seen by comparison of (8.26) wih he normal equaions associaed o (8.27) = x s x T s x s y s. (8.28) s= The mehods discussed in he above subsecions are appropriae o sysems ha are known o change slowly over ime. In such cases is chosen close o, or V is chosen as a small non-negaive posiive definie marix. If he sysem exhibis more likely from ime o ime some abrup changes of he parameers, echniques based on faul deecion migh be more suiable An ODE Analysis Simulaion no doub gives useful insigh. However, i is also clear ha i does no permi generally valid conclusions o be drawn, and herefore i is only a complemen o heory. The scope of a heoreical derivaion would in paricular be o sudy wheher he parameer esimaes ˆ converge as ends o infiniy. If so, o wha limi? And if possible also o esablish he limiing disribuion of ˆ. A successful approach considers he sequence {ˆ } =0,,2 as approximaing a coninuous vecor valued funcion { : R + R d }. This coninuos funcion evaluaed a a ime insan > 0is denoed as ( ), and he whole sequence is described as an Ordinary Di erenial Equaion. Such approach ypically adops a sochasic seing where {Y } is a sochasic process, and {X } is a vecor valued sochasic process, and boh have bounded firs- and second momens. Recall ha he minimal MMSE 2 R d is hen given as he soluion o E X X T = E [X Y ]. (8.29) Define again R = E X X T, and suppose his one is inverible. Define he funcional r (recall ha here is a funcion) as: r( ) =E X (Y X T ). (8.30) Now, consider he following ( ) = R r( ( )). If his ODE is solved numerically by an Euler mehod on discreizaion seps, 2,... one ges s= k k +( k k )R E[Y X T k ]. (8.32) Noe he similariy beween (8.32) and he algorihm (8.5), suggesing ha he soluions o he deerminisic ODE will be close in some sense. Specifically, consider he following recursion described by he algorihm ˆ = ˆ + R X (Y X T ˆ ). (8.33) wih ˆ 0 given. Then he pahs described by his discree recursion will be similar o he soluions { k } k using he imescale ( k k )=. The above saemens can be made quie precise as was done in [?]. The sudy of his ODE gives us new insigh in he RLS algorihm, including: 29

8 8.2. OTHER ALGORITHMS. The rajecories which solve he ODE are he expeced pahs of he algorihm. 2. Assume ha here is a posiive funcion V (, R) such ha along along he soluions of he ODE we have V ( ( ), R) apple 0. Then as, ( ) D c V ( ( ), R) =0, or o he boundary of he se of feasible soluions. In oher words, ( ) for go o he sable saionary poins of he ODE. Equivalenly, ˆ converge locally o a soluion in D c. 8.2 Oher Algorihms Since he problem of recursive idenificaion, adapive filering or online esimaion is so ubiquious, i comes as no surprise ha many di eren approaches exis. This secion reviews hree common variaions Recursive Insrumenal Variables Recall ha insrumenal variable echniques come ino he picure when he noise is known o be srongly colored, and a plain LSE is no consisen. An insrumenal variable esimaor uses random insrumens {Z } which are known o be independen o he noise of he sysem. Then we look for he parameers which mach his propery using he sample correlaions insead. Formally, consider he saisical sysem Y = x T 0 + D, (8.35) where {D } is colored noise, and x 2 R d, wih deerminisic bu unknown vecor 0. Suppose we have d-dimensional insrumens Z such ha E [Z D ]=0 d. (8.36) Tha is, he insrumens are orhogonal o he noise. Then he (bach) IV esimaor n is given as he soluion of 2 R d o nx Z (Y x T ) =0 d, (8.37) = which look similar o he normal equaions. If n = (Z x T ) were inverible, hen he soluion is unique and can be wrien as n = nx Z x T = nx = Z Y T, (8.38) Now we can use he echniques used for RLS o consruc a recursive mehod o esimae when he daa comes in. I is a simple example o derive he algorihm, which is given as 8 =(y x T ˆ ) >< = Z x T +x T Z (8.39) K = Z >: ˆ = ˆ + K. 30

9 8.2. OTHER ALGORITHMS The discussion on he behavior of RLS w.r.. iniial variables and forgeing facor remains valid Recursive redicion Error Mehod Recall ha a EM mehod bases inference on maximizing performance of he bes predicor corresponding o a model. Also his echnique is sraighforwardly o phrase in a recursive form. = argmin V ( ) = k= k k ( ), (8.40) where 0 < apple is ypically chosen as 0.99, 0.95, 0.9. As before ( ) denoes he predicion errors of corresponding o model parameers, ha is ( ) =y ŷ ( ) whereŷ ( ) is he opimal predicor a he h insance. Now, unlike he previous algorihms, no closed form soluion of (8.40) exiss in general, and one resors o numerical opimizaion ools. Bu here is an opporuniy here: i is no oo di cul o inegrae -say- a Gauss-Newon sep in he opimizer wih he online proocol. To see how his goes, consider again he second order Taylor decomposiion of he loss funcion. Les assume we have a fairly good esimae ˆ a he previous insance V ( ) =V (ˆ )+V 0 (ˆ ) T ( ˆ )+ 2 ( ˆ ) T V 00 (ˆ )( ˆ ). (8.4) Now, he challenge is o compue gradien V 0 and Hessian V 00 recursively. Deails can be found in he book (Södersröm, Soica, 989), bu are necessarily ied o he adaped model and are ofen approximaive in naure Recursive seudo-linear Leas Squares The following example expresses an ARMAX as a pseudo-linear model as follows. Example 53 (ARMAX) Given an ARMAX sysem A(q )y = B(q )u + C(q )e, (8.42) of orders n a,n b,n c. Then his sysem can almos be wrien as a LI model as follows y = ' T 0 + e, (8.43) where ( ' =( y,..., y na,u,...,u nb, ê,...,ê nc ) T 0 =(a,...,a na,b,...,b nb,c,...,c nc ), (8.44) where ê is he predicion error compued based on he model parameers ˆ.Theraionaleisha in case, ê is a good proxy o he predicion errors e based on he parameers. Then he Recursive arial Leas Squares algorihm implemens a RLS sraegy based on his linearized model. Indeed one can prove ha he resuling esimaes do converge if he sysem is obeys some regulariy condiions. Specifically, if he sysem is almos unsable he recursive esimaes are ofen unsable (and diverging) as well. In pracice, he resuling algorihm needs monioring of he resuling esimaes in order o deec such divergen behavior. 3

10 8.3. MODEL SELECTION Sochasic Approximaion The class of sochasic approximaion echniques ake a quie di eren perspecive on he recursive idenificaion problem. Here he parameer esimae ˆ obained previously is modified such ha i is beer suied for explaining he new sample relaed o (',y ). Formally, a new esimae ˆ is obained from he given ˆ and he sample (',y ) by solving for a given >0 he opimizaion problem ˆ = argmin J ( ) =( ˆ ) T ( ˆ )+ x T 2 y. (8.45) The opimal resul is hen given direcly as ˆ = ˆ x T y ', (8.46) obained by equaing he derivaive of J ( ) o zero. The algorihm is hen compleed by specificaion of he iniial esimae ˆ 0. This recursion gives hen wha is called he Leas Mean Squares (LMS) algorihm. This is he building sone of many implemenaions of adapive filering. The naming convenion sochasic approximaion is moivaed as follows. The correcion a insance is based on he gradien of a single poin (x,y ), and is a very noisy esimae of he overall gradien. A variaion of his algorihm is given by he recursion ˆ = ˆ kx k 2 + xt y, (8.47) wih > 0 small, and where ˆ 0 is given. This recursion is he basis of he Normalized LMS algorihm. The raionale is ha here each sample modifies he presen esimae proporional how close he esimae is o he working poin 0 d. 8.3 Model Selecion As in he bach seing, i is paramoun o be able o qualify and quanify how well our recursive algorihms succeeds in is ask. Bu he concepual and pracical ways o do urn ou o be enirely di eren. As i sands here is no comprehensive heoreical framework for his quesion, bu some insigh is gained in he following example. Example 54 (redicing Random noise) As seen, a lo of fancy mahemaics can be brough in o form complex recursive schemes, bu a he end of he day he mehods implemened need merely may good predicions. I helps o reason abou his objecive by considering he predicion of random whie noise: by consrucion his is impossible o do beer han ŷ =0(why?). A mehod rying o fi a complex model o such daa will necessarily do worse han his simple predicor, and he example is ofen used as a validiy check of a new mehod. Excep for he radiional consideraions of bias and variance of a model, and he saisical uncerainy associaed wih esimaing parameers, oher issues include he following: Iniializaion of he parameers. If he iniial guess of he parameers is no adequae, he recursive algorihm migh ake much samples before correcing his (ransien e ec). Forgeing Facor. The choice of a forgeing facor makes a rade-o beween flexibiliy and accuracy. 32

11 8.3. MODEL SELECTION Window. If he window used for esimaing hen one mus decide on how many samples are used for esimaing a a cerain insance. Sabiliy of he esimae. If he algorihm a hand is no well-uned o he ask a hand, i may display diverging esimaes. This is clearly undesirable, and some algorihms go wih guaranees ha no such unsable behavior can occur. Gain. A ypical parameer which needs be uned concerns he size of he updae made a a new sample. If he gain is oo low, a resuling algorihm will no converge fasly. If he gain is oo large, one may risk unsable behavior. In order o check weher a recursive idenificaion is well-uned for a cerain applicaion, i is insrumenal o monior closely he online behavior of he mehod, and o make appropriae graphical illusraions of he mehod. 33