Recursive Prediction Error Identification of Nonlinear State Space Models

Transcription

1 Recursive Preicio Error Ieificaio of Noliear ae pace Moels orbjör Wigre ysems a Corol, Deparme of Iformaio echology, Uppsala Uiversiy, PO Bo 337, E-7505 Uppsala, WEDEN. orbjor.wigre@i.uu.se Key Wors - Ieificaio; o-liear sysems; sae-space moels; preicio error mehos; scalig; samplig. Absrac A recursive preicio error algorihm for ieificaio of sysems escribe by oliear oriary iffereial equaio (ODE moels is presee. he moel is a MIMO ODE moel, parameerize wih coefficies of a muli-variable polyomial ha escribes oe compoe of he righ ha sie fucio of he ODE. I is eplaie why such a parameerizaio is a ey o obai a well efie algorihm, ha oes o suffer from sigulariies a over-parameerizaio problems. Furhermore, i is prove ha he selece moel ca also hale sysems wih more complicae righ ha sie srucure, by ieificaio of a ipu-oupu equivale sysem i he cooriae sysem of he selece saes. he liear oupu measuremes ca be corrupe by zero mea isurbaces ha are correlae bewee measuremes a over ime. he isurbace correlaio mari is esimae olie a ee o be ow beforeha. he algorihm is applie o live aa from a sysem cosisig of wo cascae as wih free oules. I is illusrae ha he ieificaio algorihm is capable of proucig a highly accurae oliear moel of he sysem, espie he fac ha he righ ha srucure of he sysem has wo orivial oliear compoes. A ovel echique base o scalig of he samplig perio ha sigificaly improves he umerical properies of he algorihm is also isclose.

2 . INRODUCION ysem ieificaio base o liear moels is oay well esablishe i research a i pracice, a fac ha maifess iself i wiely use sofware pacages lie he MALAB ysem Ieificaio oolbo. May mehos have bee eee o eal also wih oliear moels. Algorihms a available heoreical resuls are however more scaere i he oliear fiel. here are some srog reasos for his. Firs, i is more ifficul o fi moels wih a wie valiiy, alhough geeral framewors ca be formulae (Ljug, 997. Oe approach eoe gray-bo moelig combies physical moelig, umerical iegraio a parameer esimaio for esimaio of physical parameers i iffereial equaio moels, oriary (Bohli, 994 a parial (Fuquis, 994. Algorihms base o geeral sae space moels are also iscusse i Ljug a Gla (004. I oher siuaios blac bo ieificaio, wihou irec physical moel iegraio, is applie. May moels a mehos have bee propose, see e.g. jöberg e al., (997 for a survey. he mos simple mehos are perhaps he bloc-oriee algorihms ha are base o cascae liear yamic a saic oliear blocs (Billigs a Fahouri, 98. he Hammersei a he Wieer moel (Weswic a Verhage, 996 base algorihms belog o his class. Oher commo approaches for oliear sysem ieificaio iclue he use of eural ewors (Che a Billigs, 99; jöberg a Ljug, 99, fuzzy moels a he o-liear yamic moels base o ifferece equaios of e.g. NARMAX (Che a Billigs, 989, NARX a NFIR ype, cf. Ljug, (997. he fuameal mehos base o Wieer a Volerra series (Wieer, 958; cheze, 980 shoul also be meioe here. Give he moel srucure, he choice of algorihm (opimizaio algorihm is ofe more complicae ha i he liear case, where leas squares echiques ca ofe be applie. For oliear moels leas squares moelig ca also someimes be use, a he he recommeaio is o ry his approach firs (Ljug, 997. I gray-bo moelig i is o ucommo o e up wih moels where he impora parameers eer oliearly, e.g. ue o uerlyig physical priciples (Bohli, 994; Esam a me, 987. All quaiies eee for a leas squares formulaio may o

3 be measurable eiher, a siuaio ha is ofe hale by he iroucio of oupu error moels. Oher algorihmic issue ha ees o be hale wih some care iclue regularizaio (jöberg a Ljug, 99; Ljug, 997 a scalig (Esam a me, 987. Ipu sigal properies require for eciaio of oliear sysems are also more complicae ha i he liear case (Wigre, 993, 003. A hir reaso for he ifficuly of oliear sysem ieificaio is ha fuameal properies, e.g. sabiliy, is ramaically more ifficul o hale whe he moel yamics is oliear. his affecs he performace a usefuless of algorihms as well as he heoreical aalysis. Furher complicaios o he heoreical sie ivolve saisical properies, lie saioariy, whose valiiy is crucial i aemps o apply heoreical aalysis. Available resuls i his fiel are ofe base o eesios from liear sysem ieificaio, (Ljug,977ab; Ljug a öersröm 983. here are also geeral resuls lie Ljug, (975 available, bu here applicaio ofe require ha fuameal properies lie sabiliy a saisical saioariy are eiher verifie or assume. Naurally, a mai river for he evelopme of algorihms for ieificaio of oliear sysems is he large umber of ecoomically impora applicaios. his iclues power sysem compoes (Esam a me, 987, coiuous pulp igesers (Fuqvis, 994, iesel geeraors (Billigs e al., 988, rum boilers (Åsröm a Elu, 975 a ph-corol sysems (Pajue, 99, jus o meio a few. Wih he above bacgrou he coribuios of he prese wor ca be sae. he repor eplois a resrice parameerizaio of a oliear sae space moel, where oly oe compoe of he righ ha sie of he ODE is parameerize. I is eplaie ha such a resrice parameerizaio is a ey o obai a oliear blac bo ieificaio problem ha reuces he ris for over-parameerizaio. he remaiig compoes of he ODE are selece as a series of iegraors, iegraig he parameerize compoe. his choice of sae is irecly applicable o all muliple ipu sigle oupu (MIO oliear ODEs formulae i ipu-oupu form. Physical moelig of elecrical, mechaical a chemical sysems ofe resuls i such moels. However, whe physical moelig is performe for he purpose of corol i is ofe he case ha processes ha are 3

4 pars of he pla are moele iepeely, cf. he cascae as of secio 5. I such a case a bloc righ ha sie of a oliear sae space ODE is aurally obaie i.e. here are several compoes of he ODE ha have oliear, orivial righ ha sies, cf. Ljug a Gla (004. A firs glace i hece seems ha he propose resrice parameerizaio may o be geeral eough i pracice. he firs mai coribuio of he repor proves ha he propose resrice moel srucure is locally applicable for escripio of sae space ODE moels wih a geeral righ ha sie srucure. I oher wors, eve if e.g. physical moelig suggess a complicae righ ha sie srucure of a oliear ODE, he propose parameerizaio of a sigle compoe of he ODE ca be epece o provie useful ieificaio resuls, albei i aoher cooriae sysem ha he oe use for physical moelig. he seco coribuio is he algorihm, which is of recursive preicio error (RPEM ype. I is base o a iscreize versio of he above oupu error ODE moel. Discreizaio is performe by a Euler ifferece scheme. he avaage wih his is ha a irec coecio o he coiuous ime physical parameers is reaie, wheever such a relaio is a ha. he parameerizaio of he righ ha sie compoe is base o a muli-variable polyomial i he saes a ipus. he formulaio of he scheme allows for muliple liear oupu measuremes, i.e. he approach ca be use for recursive ieificaio of MIMO oliear sae space moels. Measureme oise levels a he oise correlaio mari are esimae o lie a ee o o be ow beforeha. A hir coribuio is cosiue by a ovel scheme uilizig scalig of he samplig perio whe ruig he algorihm. he scalig algorihms eplois he fac ha he samplig perio appears eplicily a liearly i propose algorihm. he operaio of he scheme is iscusse a i is show ha i resuls i a epoeial (i he moel orer scalig of he saes of he propose moel. his is aracive, sice algorihmic problems ofe origiae from poor scalig of he geerae sigals i he ieificaio algorihm. he scheme resuls i sigifica performace improvemes i eperimeal rus. 4

5 Avaages as compare o eisig echiques iclues he use of a smaller umber of parameers ha wha is eee whe some oher blac bo moels lie Volerra series epasios a eural ewors are applie. his is ue o he srucure impose by he propose blac bo parameerizaio. he srucure also maes over-parameerizaio less liely a reuces he ee for regularizaio ha is a ecessiy whe e.g. eural ewors are applie o sysem ieificaio (jöberg a Ljg, 99. his is impora, sice parameerizaios for oliear sysems are ofe ifficul o aalyze a sice i is ofe o ow whe a cerai parameerizaio will resul i a sigular ieificaio problem. ecoary beefis shoul iclue compleiy avaages as well as possibly beer accuracy as sae by he parsimoy priciple. he algorihm is also applicable o sae space moels wih a geeral righ ha sie srucure a ca also be irecly applie o-lie. Noe ha coiuous ime oliear sae space moels is he basis for may oliear coroller esig mehos. I is hece avaageous ha he propose moel a algorihm provie esimaes ha are easy o rasform o oliear MIMO coiuous ime sae space moels. Disavaages iclue he ris o coverge o local sub-opimal miimum pois, a ris ha always ee o be assesse whe oupu error moelig is use. Furhermore, srucural prior iformaio cao be uilize sice a blac bo approach is use. he repor is orgaize as follows. ecio preses he resrice moel srucure a proves resuls abou is geeraliy. ecio 3 erives he algorihm while secio 4 iscusses scalig.. ecio 5 procees wih a iscussio of he ieificaio of a highly oliear ouble a process where he algorihm is applie o measure aa. he coclusios appear i secio 6. Coiios o he moel a he algorihm are irouce i he e whe eee a hey are eoe by C, C, a so o.. MODEL RUCURAL APEC o moivae he propose moel srucure, some properies of oliear ODE moels are firs aresse. wo mai possibiliies eis for he formulaio of oliear ODE moels. Eiher a ipuoupu approach ca be ae, or a geeral oliear sae space moel ca be use. As iscusse 5

6 below he wo approaches iffer whe he ipu-oupu moel is wrie i sae space form i ha oly oe righ ha sie oliear fucio ees o be parameerize, as compare o for he geeral sae space moel. A firs glace i hece may seem ha he ipu-oupu base moel is less geeral a ha he geeral oliear sae space moel is he oe o base algorihms o. However, here also appears o be avaages associae wih he oliear ipu-oupu moel. Firs, a reucio of he umber of parameers seems o be possible. More imporaly, a resricio of he geeral moel srucure appears o be a ecessiy i orer o avoi over-parameerizaio a o have a well pose ieificaio problem. his secio aalyses hese issues a provies heoreical moivaio for he propose resrice sae space moel srucure. he sarig poi for he erivaio of he moel ha forms he basis for he evelopme of he repor is he :h orer oriary iffereial equaio ( ( ( ( = f,...,, u,..., u,..., u,..., u,. ( he iffereial equaio is parameerize i erms of he uow parameer vecor, cosisig of compoes (bol face characers eoe vecors a marices. he epee variable is eoe by (, while he ipu vecor is give by ( ( ( ( ( ( ( = (. ( u u u u u Coiuous ime is eoe by. Differeiaio j imes is eoe by he superscrip ( j i he repor. I orer o wrie ( i sae space form he followig selecio of saes is mae his resuls i he sae space moel ( i i =, i =,...,. ( 3 ( ( =. ( 4 ( ( ( f u u u u,,,,,,,,,, ice he observe quaiy of ieres of ( is, he measureme equaio is 6

7 ( = ( 0 0 y where y ( eoes he measure quaiy. ( 5 Obviously, he sysem ( 4 ca escribe arbirary ODEs of he form (, so i is a oliear moel ha ca escribe a relaively wie class of oliear sysems. here are, however, relae sae space moels ha o o have he srucure of ( 4, raher he more geeral srucure ( f u u,,,,,, ( = f u u,,,,,, ( f,,, u,, u,, ( 6 applies, where he vecor u( represes all ipu sigal compoes o he sysem. he wo mai problems aresse i his secio are he P How are ( 4 a ( 6 relae o over-parameerizaio? P How geeral is ( 4 as compare o he geeral escripio ( 6? Noe ha whe aressig he above problems, i is geerally assume ha he objecive is o obai a blac bo moel, i.e. o srucure is assume for he righ ha sie fucios of he compoes ( 4 a ( 6. he problem P ca be aalyze by cosieraio of he liear sae space moel a a a u = y= ( b b b ( 7 ha correspos o he sricly proper rasfer fucio 7

8 ( = ( Ys bs + b s+ b s a s a s a Us. ( ice ( 7 is a miimal realizaio i case here is o pole zero cacellaio i ( 8, i is clear ha blac bo parameerizaios base o ( 6 woul, i he liear special case, easily resul i overparameerizaio. As a eample, a blac bo muli-variable polyomial parameerizaio base o ( 6 woul i he liear special case have a sysem mari fille wih iepee parameers, as compare o he parameers of ( 7 eee o escribe he yamic moes. he yamics of he resrice moel ( 4 o he oher ha, becomes ieical wih he corollable caoical form ( 7 i case of a muli-variable polyomial parameerizaio. Hece, i ca be coclue ha he moel srucure ( 4 shoul be use raher ha ( 6 i orer o avoi over-parameerizaio. o aalyze he problem P, he remaiig pars of his secio ses ou o prove ha ( 4 ca be locally use o escribe he yamics of ( 6, by applyig a rasformaio o aoher cooriae sysem. he iea of he proof is o rewrie ( 6 as he ipu-oupu moel (, which i ur ca be wrie i he sae space form ( 4. he proof compues iffereiae versios of he las compoe of ( 6, followe by a applicaio of he implici fucio heorem. o begi, he las compoe of ( 6 is assume o be he observe quaiy of ieres, i.e. he epee variable o be use i ( shall be. his meas ha he followig resricio ees o be impose C he sysem ( 6 is muliple ipu sigle oupu wih observe sigal. he las equaio of ( 6 is he iffereiae wih respec o. A firs iffereiaio resuls i f ( f f f f f = ( u u u u ( f,...,, u, u,..., u, u,,...,. ( 9 Afer iffereiaio seps, he followig relaios are obaie 8

9 ( f,,, u,, u,,, ( ( ( = f u u u u,,,,,,,,,,, ( ( f,, u,, u,, u,, u,,, ( ( f (,...,, u,..., u,..., u,..., u,,..., ( ( f,...,, u,..., u,..., u,..., u,,...,. ( 0 I orer o prove ha ( 0 ca be wrie as ( 4, i is ow sufficie o wrie he las equaio of ( 0 as (, which i ur ca be wrie as ( 4 usig ( 3. owars ha e, he sae variables,..., ee o be elimiae from he las compoe of ( 0. he e sep of he proof herefore maes use of he implici fucio heorem (Khalil, 996. he implici fucio heorem is give by m Lemma : [Implici fucio heorem] Assume ha f:r R R is coiuously iffereiable a each poi ( y z of a ope se P R R m. Le ( y0 z0 which f( y0, z0 = 0a for which he Jacobia mari [ f y]( y0 z0 eighborhoos U R of y 0 a V R m be a poi i P for, is osigular. he here eis of z 0 such ha for each z V he equaio a uique soluio y U. Moreover, his soluio ca be give as y g( z iffereiable a z= z 0. fyz, = 0has =, where g is coiuously Noe ha he oaio y a z oes o ecessarily refer o he measureme i Lemma, hey jus eoe variables use i he formulaio of he resul i Khalil, (996.. he iea is ow o solve he oliear sysem of equaios obaie from he firs equaios of ( 0 for.... he quaiies releva o he applicaio of Lemma are ( + ( + i i = f = f ( R R R ( (... : y =... R ( 9

10 z = ( + ( + ( ( ( = ( u u u u i i R ( 3 By ispecio, o apply he implici fucio heorem a o obai a coiuously iffereiable ODE wrie as ( 4, he followig coiios o ( 6 ee o be irouce: C he ipu sigal is u( is imes coiuously iffereiable. C3 f ( u u,...,,,...,, is imes coiuously iffereiable, for all u u R + +. C4 f ( u u,...,,,...,,, i =,...,, are imes coiuously iffereiable, for all i i u u R + + i. i C5 [ f y]( y z, efie by ( is osigular for ( y0 z0 0 0 efie by ( a ( 3. Remar : Coiuous iffereiabiliy is assume o hol globally i C, C3 a C4. Relae resricios i lie wih Lemma are possible, bu woul complicae he preseaio a are herefore o iclue. he moels of his repor are polyomial, a hece global iffereiabiliy applies. ( ( I follows from he cosrucio of u u u u,...,,,...,...,,...,,,..., a from C, C3 a C4 ha f,...,, u,... u,..., u,..., u,,...,... is coiuously iffereiable. For pois ( y0 z0 f ( ( where C5 is fulfille, here are herefore eighborhoos U R + + i i = ( of y 0 a V R of z 0 such ha for each z V he equaio ( ( ( (,...,, u,... u,..., u,..., u,,..., (... f = 0 ( 4 has he coiuously iffereiable soluio 0

11 ( g u u u u,,,,,,,,,,,, = ( ( ( g u u u u. ( 5,,,,,,,,,,,, g ( (,,,, u,, u (, u,, u (,,, Whe he compoes of ( 5 are isere io he las compoe of ( 0, he resul ca be epresse as he followig ODE ( ( ( ( ( = f g,...,, u,... u,..., u,..., u,,...,... ( 6 ( ( ( ( ( (, g,...,, u,..., u,..., u,..., u,,...,,, u,..., u,..., u,..., u,,..., which is of he form (. Noe ha sice he compoes of ( 5 are coiuously iffereiable, i follows by cosrucio of ( 0, a by C, C3 a C4, ha he righ ha sie of ( 6 is coiuously iffereiable. his proves heorem : Assume ha C, C, C3 a C4 hol a cosier he ODE ( f u u,,,,,, ( = f u u,,,,,, ( f,,, u,, u,. he, for all pois where C5 hols, here are eighborhoos U R of (... a + + i i = V R ( ( ( ( of u u u u as well as a cooriae rasformaio ha rasforms he ODE o he form ( ( =. ( ( ( f u u u u,,,,,,,,,, Remar : As sae above physical moelig ofe resuls i ODE:s of he form ( 6. he implicaio of heorem is ha such sysems ca ayway be locally moele by ieificaio algorihms base o he srucure ( 4, as i he remaier of his repor.

12 Remar 3: I case he las sae equaio of ( 6 (he mai observe quaiy epes o a ipu sigal, he cosrucio of he sae rasformaio of heorem irouces erivaives of his ipu sigal, a fac ha may be pracically roublesome. However, i case he las sae equaio of ( 6 oes o coai ay epeece of he ipu, o erivaives of ipus are irouce i he cosrucios leaig o heorem, cf. ( 9 - ( 0. I such a siuaio o erivaives of ipus are eee i he parameerizaio ( 9 - ( provie ha he remaiig sae equaios of ( 6 oes o coai ay erivaives of ipu sigals. 3. RECURIVE PREDICION ERROR ALGORIHM I his secio a recursive preicio error meho (RPEM of oupu error ype is erive, base o he moel ( 4. he erivaio follows he saar approach of (Ljug a öersröm, 983. he oupu error approach is eee sice applicaio of leas squares echiques irecly o ( 4 woul require ha he complee sae vecor woul be measurable, which is a quie resricive assumpio. 3. Parameerizaio a Discreizaio he recursive ieificaio algorihm is assume o operae o iscree ime measure ipu a oupu sigals, obaie from he sysem of ieres. he ipu sigal vecor is give by (, a he measure oupu sigal is assume o be ( = y, (... y, (. ( 7 y m m m p he ipu sigal erivaives coaie i ( ha are someimes eee (cf. Remar 3 may be measure or geerae by umerical iffereiaio. I orer o ieify he sysem, he moel ( 4 is use, ogeher wih a ow oupu equaio, i.e. he followig moel is use ( ( = ( ( ( f,,, u,, u,, u,, u, ( 8 y c c = y c c p. p p

13 Remar 4: I he basic muliple ipu sigle oupu seig, efie by C, where he oupu sigal correspos o he epee variable, i follows ha = p =, c = a c i = 0, i. his is o a algorihmic requireme hough, a muliple ipu muliple oupu moels ca be use, allowig for use of auiliary measuremes of erivaives of he oupu i he ieificaio process. As sae above, he coefficies of he oupu relaio are o esimae. he reaso is ha he cascae srucure of he yamics a he oupu sigal relaio woul he resul i a sigular problem sice oly he prouc of he saic small sigal gai maers from a ipu-oupu poi of view, cf. Wigre (993. he moel ( 8 is parameerize i coiuous ime. he followig polyomial ( parameerizaio is irouce for ( f,...,, u,..., u,..., u,..., u, I I I I I u u u ( ( f ( u u u u,...,,,...,,...,,...,, = i = 0 i = 0i u = 0 i = 0 u i = 0 u I u i i i i ( i i u u u u i... i i u... i... i... i u u u u u = ϕ ( u i =0 u u u,. ( 9 where = u u u u u X I I... I... I I I... I u ( 0 I ϕ = u I... u u u... u u I u... I... I I u u u... (... ( ( u...( u I I I I u u u u u. ( a where capial I is use o eoe he upper limi i he correspoig sum of ( 9, i.e. he egree of he variable i quesio. 3

14 I orer o formulae a iscree ime moel he coiuous ime ODE moel ( 8 ees o be iscreize. his is oe wih a Euler forwar iscreizaio scheme wih samplig perio s. he resul is he followig iscree ime moel ( +, (, (, ( ( ( + = +,,, ( ( u u ( u ( u +,, ϕ,,,,,,,,,,, ( y(, c c (, y(, = = = C(, yp(, cp c p (,. ( 3. Graie he graie of he oupu preicor of ( is of ceral imporace i he RPEM. he graie is ψ (, (, (, y = = C. ( 3 he erivaive of he sae wih respec o he parameer vecor follows from a iffereiaio of he sae equaio of (, eacly as i Ljug a öersröm, (983. ( +, (, (, ( = +,. ϕ( (,, u( (, ( ( u( + ϕ,, ( 4 Ieraio of ( 4 he gives he graie afer muliplicaio accorig o ( 3. he compuaio of he las facor of he las compoe of ( 4 remais o be efie. his follows by sraighforwar iffereiaio of (. he eails are oulie i he algorihm ( 8. he oaio may loo comple, however whe coig he algorihm he srucure of he uerlyig sums of ( 9 ca be aurally eploie i a loop srucure ha efficiely compues all require quaiies. 3.3 Algorihm I orer o erive a recursive preicio error ieificaio algorihm he followig crierio is irouce, cf. Ljug a öersröm, (983, pp

15 [ ] V( = E ε (, Λ (, ε (, + log e Λ (,. ( 5 I ( 5, E[ ] eoes he epecaio operaor, Λ(, eoes he (uow covariace mari of he measureme isurbace a ε(, eoes he preicio error. he covariace is epee of he uow parameer vecor sice i is esimae from he preicio errors obaie urig he ieificaio ru. his is he reaso for he seco erm of ( 5. I case of Gaussia preicio errors he crierio equals he maimum lielihoo crierio, cf. Ljug a öersröm, (983. he preicio error, ε(,, is give by (, = (, (, ε y y. m ( 6 he miimizaio of ( 5 is he performe wih he sochasic Gauss-Newo meho, cf. Ljug a öersröm (983, pp ha algorihm requires firs a seco erivaives of he crierio ( 5. he firs erivaive follows from ( 3 - ( 6. he seco erivaive follows by iffereiaio of ( 5 wice, uilizig ( 3 a ( 4, a by iroucio of he coveioal Gauss-Newo approimaios. he resulig upaig equaios for a Λ(, he become. cf. Ljug a öersröm, (983 ( = ( + µ ( ( = Λ( + µ ( Λ (, Λ (, ( (, ( ( R ψ ε ( ε ( ( ε, (, ( Λ( ( 7 I is show i Ljug a öersröm, (983, pp , ha his choice of upaig irecio for Λ leas o a miimizaio of ( 6. I ( 7, µ ( is he gai sequece a R( is he approimaio of he seco erivaive of he crierio of ( 5. he quaiies ε ( ψ ( (,,, ca be compue from ( 6 a ( 4, respecively, by replaceme of wih he ruig esimae (. he same proceure ca be use for Λ. he resulig algorihm is o recursive hough, sice all available aa ees o be processe by he yamic moel, for fie (, i orer o geerae ε, (, ψ, ( a Λ (,. However, eacly as i 5

16 Ljug a öersröm, (983, his issue ca be resolve by usig he ruig esimae also i (, o obai approimaios of y, (, ε, (, ψ, ( a Λ (,. hese approimaios are geerally accurae wheever he aapaio gai is sufficiely small a he sysem is epoeially sable. Before saig he complee algorihm i is oe ha a projecio algorihm is eee i orer o eep he moel sable, as iicae by he braces i he -recursio i ( 8. he projecio algorihm is furher iscusse i he e subsecio. he fial resul is ow he followig recursive algorihm ( = y ( y( ε m Λ ( = Λ( + µ ( ( ε( ε ( Λ( R ( = R( + µ ( ( ψ( Λ ( ψ ( R( [ = + µ ( ( ( ψ( Λ ( (] ε R D M I ( ( ( ( ( ( ( ( u I I u u ϕ u = u u u I ( ( I ( I I I I ( ( ( ( u... u u u...( u I ( ( u u u u u ( + ( ( + = ( + ( + ( ϕ ( + y y( + = y p ( + c c = c c p p ( ( ( ( ( + ( + ( ( ( ϕ ( ( ( ( ( ( ( ( ( ( ( u I I u I u I i u u u = 0... i i i..., i i =,..., 6

17 7 ( ( ( ϕ ϕ ϕ = ( ( ( ( ( ( ( ( ( ( ϕ ϕ = + + ψ c c c c p p + = + +. ( Projecio a Iiializaio I he liear case, he applicaio of projecio algorihms (Ljug a öersröm, 983 is a saar approach o eep he moel asympoically sable. he reaso is ha sabiliy properies ca be easily erace from he pole locaios. I he geeral oliear case reae here, o such sraighforwar meho oes eis. his is o reaso o avoi he problem hough, here is raher a srog ee for sabiliy moiorig mehos i he oliear sysem ieificaio fiel. Oe (srige approach coul be o apply Lyapuov sabiliy heory o he moel. A rawbac wih his approach is he fac ha Lyapuov sabiliy aalysis ca be resricive a eve icoclusive, proviig resuls o isabiliy ha may o provie he require sharpess. A aleraive coul be o rely i liearizaio echiques i combiaio wih a assumpio o slow aapaio. he rawbac wih his approach is ha he sigal level variaio may have o be ep relaively low, so as o o ivaliae he liearizaio assumpio oo much. his woul, i ur, lea o ifficulies i he esimaio of he o-lieariies of he sysem, sice he oliear effecs are o reflece very much i he measure sigals, cf. Wigre (003. he projecio algorihm applie i ( 8 is base o a liearizaio of he moel. he liearize sysem mari becomes

18 ( = I + he moel se is efie accorig o { ( ϕ( ( D ( M = eig < δ }, δ > 0. ( 30 ice he sysem is o liear, δ may be selece wih some margi o he liear sabiliy limi. he followig projecio algorihm is he applie, i.e. [ ( ] DM ( ( D = ( ( D M M. ( 3 he upaig is hece soppe whe he resul is ousie he moel se, cf. Wigre, (994. As always whe oupu error ieificaio algorihms are use, iiializaio is of ceral imporace i orer o avoi covergece o sub-opimal miimum pois of he crierio. For oliear moels his is eve more impora, cosierig he more complicae yamics. he quaiies ha require iiial values iclue ( µ, (, Λ(, ( R a (. Oe eperiece from he his wor, is ha i is avaageous for he algorihm o maiai a relaively low aapaio gai iiially. his allows he search irecio o buil up from aa wihou eesive iiial parameer ecursios perhaps owars a usable moel. he choice of he gai sequece µ ( / is hece a very impora oe. he iiial values for Λ( a R( are boh closely ie o he choice of gai sequece a heses choices are iscusse below. he gai sequece of he algorihm is selece accorig o ( ( µ + = µ µ + µ 0 0 =. + µ µ( µ ( ( 3 his suppors he limiaio of he iiial aapaio gai, by selecio of a high eough µ. A he same ime he beefis of he coveioal epoeial ecayig rasie (Ljug a öersröm, 8

19 983 ca be reaie. ypical values ca be µ 0 = , µ = 00 ogeher wih µ ( 0 = 0. Noe ha he iiial value for Λ( shoul o sigificaly ueresimae he covariace mari of he preicio errors sice his es o icrease he gai of he parameer recursio. A similar argume applies o R(.Useful iiial parameer esimaes ca ofe be obaie by applyig liear ieificaio mehos. If his is o possible, he sraegy of secio 5 ca be applie, i.e. o iiiae ( wih parameers correspoig o asympoically sable liear yamics, well isie he moel se. 4. CALING calig of esimae quaiies is a saar echique i opimizaio heory (Lueberger, 984. I he lieraure o liear ieificaio scalig has o achieve a ceral saus hough. he reaso is perhaps he srog posiio of leas squares echiques ha are o so sesiive o problems wih slow covergece a local miimum pois, ha may be cause by poor scalig i oliear opimizaio problems. I oliear ieificaio scalig probably eserves a more ceral saus. Eample : Cosier ieificaio of he followig muli-variable polyomial oliear fucio f, = α + β where he effec of he oliear erm β o f(, is assume o be abou he same as he effec of he liear erm α 0. I case he eperime coiio woul be such ha woul be roughly a facor of 00 greaer ha, i follows ha he esimae β ees o be abou 0 4 α o eplai he aa. I case he achievable relaive accuracy of α a β woul be similar, i follows ha he compoes of he covariace mari woul iffer by a facor of0 8, i.e. coiioig is liely o be poor. I case graie algorihms are applie covergece is liely o be slow sice he covergece spee is egaively affece by he eigevalue sprea of he problem, cf. Lueberger, (984. Far from he miimum poi slow covergece is also liely o hamper more avace mehos. he reaso is ha he quaraic covergece ha ca be prove for avace Newo search mehos is oly vali close o he miimum poi of he crierio. I case of poor 9

20 scalig a coiioig he crierio fucio is liely o have arrow valleys where he algorihm may well for log perios of ime. Coveioal liear scalig applie o oliear sysem ieificaio irouces a osigular liear rasformaio of he ieifie parameers as ~ ~ = = ( 33 he rasformaio is he eploie i he crierio fucio ( 5 a a miimizaio of he crierio is performe wih respec o ~ isea of wih respec o, i.e. ~ (, ~ (, ~ log e (, ~ ~ = argmi E ε, Λ ε + Λ. ( 34 ~ he eails of he algorihmic cosequeces o ( 8 are o oulie here. I pracice iagoal mari scalig is usually all he prior owlege allows, cf. Lueberger, (984, Esam a me, (987. he applicaio of ( 33 requires some owlege of he epece rage of he iffere parameers, somehig ha may be ifficul o obai for blac bo moels. Furher, referrig o Eample, he origi of he scalig problems may very well be he relaive size of he sae sigals ha are geerae i he algorihm. I was eperimeally oice ha scalig problems i ( 8 are ofe highly relae o he selecio of he samplig perio. he samplig perio of course ees o be shor eough urig measureme, i orer o capure he esseial yamics. However, sice he samplig perio appears eplicily i he algorihm ( 8 i is sraighforwar o apply he algorihm wih aoher, scale value of he samplig perio. Eve if a scale value of he samplig perio is applie, he algorihm sill aemps o miimize he crierio, hereby obaiig oher miimizig parameer values ha whe he rue samplig perio is use. Whe esig he iea eperimeally, ramaic improvemes coul be observe i he algorihmic behavior. Covergece spees coul be improve a iiial values ha lea o ivergece a isabiliy coul be mae o wor well. o eplai his, irouce he followig moels ( cf. ( ha iffer oly i he applie samplig perio (he superscrip iscrimiaes bewee ( 35 a ( 36 0

21 ( +, ( +, ( +, (, (, (, (, (, ( = + f u u (,,.,,,, (, ( 35 ( +, ( +, ( +, (, (, (, (, (, ( = +. ( 36 f u u (,,.,,,, (, Noe ha he origial samplig perio mus be reaie i all ime argume, so as o refer o he correc measureme imes. Irouce he followig assumpios C6 he measureme y correspos o he saes (, a (, of ( 35 a ( 36. C7 he algorihm coverges o a eac escripio of he ipu -oupu properies of he sysem for ( 35 a ( 36, i.e. y (, (, = = 0,. I follows from C6 a C7 ha (, (, = = ( ( + s, ( + (,, ( ( ( ( s +, = +,, ( 37 Applyig C7 oce more resuls i ( = (,,,. ( 38 he argumeaio ca he be repeae sarig wih ( 38. he resul is 3( = 3 (,,,. ( 39 Repeiio of his process imes, a eploiig he fac ha he relaios are vali for all proves heorem : Cosier he wo moels

22 ( +, ( +, ( +, (, (, (, (, (, ( = + f u u (,,.,,,, (, ( +, ( +, ( +, (, (, (, (, (, ( = + f u u (,,.,,,, (, where is he measureme samplig perio a where is he scale samplig ierval applie whe ruig he algorihm ( 8. Provie ha C6 a C7 hols i he follows ha = (, (, i i i, i=,,. Remar 5: he essece of his resul is ha he scalig of he samplig perio resuls i sae variable scalig accorig o heorem, provie ha he esimae moel coverges o a goo eough ipu oupu moel. ice powers of he sae variables appear i he muli-variable polyomial use i he repor, such scalig ca be epece o have a sigifica effec o he crierio miimizaio problem. 5. EXPERIMEN I his secio a process cosisig of a wo cascae as wih free oules fe by a pump is suie. he laboraory equipme is locae i he corol laboraory a Uppsala Uiversiy a i is isplaye i Fig.. he ipu sigal o he process is he volage applie o he pump a he oupu sigals cosis of measuremes of he waer levels of he wo as. he process is corolle from a PC equippe wih MALAB a MALAB ierfaces o he A/D a D/A coverers ha provie commas o a measuremes from he process. he process is relaively slow wih ime cosas slighly less ha oe miue. he ipu o he process for he ieificaio eperime is geerae off lie i MALAB. A simple ime loop i MALAB oupus he ipu sigal o he pump a measures he levels of he as. he laboraory process is suiable for physical moelig. Applicaio of Beroulli s priciple a coservaio of mass resuls i

23 h a g h A A u + h =. a g a g h + h A A ( 40 Here h a h eoe he levels of he upper a he lower a, respecively. he correspoig areas of he as are A a A while he efflue areas are eoe a a a. he graviy is eoe by g, he volage o ipu flow coversio cosa by a he applie volage o he pump by u. I ca be oe ha he moel is highly oliear, ha he srucure correspos o ( 6 raher ha o ( 4, a ha here is o eplici ipu epeece i he equaio escribig he level of he lower a. Hece he resul of heorem a Remar 3 ca be illusrae i a real siuaio. he scalig scheme of heorem will also be illusrae. I he followig eample he ipu sigal is he volage applie o he pump a oupu sigal is efie o be he level of he lower a, i.e. ( 8 is use for ieificaio of a seco orer oliear sysem. he waer level measuremes from he upper a are o use. Eample : 500 samples of aa from he ouble a process were collece. he samplig perio was 5 s. I orer o ecie he sysem i frequecy a i ampliue a PRB owsample 30 imes, moifie by muliplicaio wih a uiformly isribue raom variable, was use as ipu sigal, cf. Wigre, (003. he ipu sigal varie he applie corol volage o he pump bewee 0 a.4 V. he applie ipu sigal appears i Fig. ogeher wih he measure waer level of he lower a. Ieificaio was he performe wih he algorihm ( 8. Noe ha he observe quaiy equals he firs sae vecor compoe i ( 8, corary o he formulaio i ( 40. Liear measuremes were assume, i.e. he followig measureme equaio was use y =. ( 4 he oliear sae relaio he ees o compesae for he complee ipu-oupu olieariy. he moel orers of he righ ha sie fucio were selece as I = I = I =, i.e. u 3

24 =( ϕ =( u u u u. ( 4 he ieificaio algorihm was iiialize wih R ( 0 = 30I a wih ( 0 0 ( / 0 = 0. Furhermore, µ 0 = 0999., µ 300 =.., = a Λ 0 = 0000., µ 0 = 5 were use. he sabiliy limi of he projecio algorihm was se o δ = he iiial parameer vecor was selece as = ( ( 43 his correspos o asympoically sable coiuous ime liear yamics. Iiial eperimes reveale a eecy o e up a he bouary of he moel se. Eperimeig race he problem o he ime scalig of he origial sysem, a hece scalig of he samplig perio was applie as escribe i secio 4. he values =5.0 s a =0.5 s were use, i.e. he algorihm was applie wih a scale samplig perio of 0.5 s. he scalig resolve he problems iicae above. A he e of he recursive ieificaio ru, he followig parameers were obaie, correspoig o he scale samplig perio. ( 500 = ( 44 he parameers obaie a he e of he ru were use o simulae he oupu of he sysem. he resul appears i Fig. 3 a i shows ha a quie accurae moel is obaie. he evoluio of he parameer esimaes urig he ru appears i Fig. 4. As a compariso o Fig., he liear moel ha is obaie whe he parameers correspoig o oliear erms are se o zero, was simulae. he resul appears i Fig. 5 a his figure iicaes ha he oliear coribuio o he oupu sigal of he moel is omia. o valiae he moel he mea resiual aalysis meho was applie (Wigre, 003. ha meho sors resiuals i pre-eermie iervals, correspoig o he ierval of he measure oupu sigal a he same ime isa. his provies a esimae of he moelig accuracy as a fucio of he sigal levels. he resuls appears i Fig. 5, which iicaes ha he accuracy is beer for high sigal levels. A ispecio of Fig. 3 suppors his fac. A he e of he ru 4

25 Λ( 500 = 404. was obaie. I ca be coclue ha ha he propose meho is capable of accurae moelig of he oliear ouble a laboraory process. his verifies he algorihm, heorem as well as heorem. 6. CONCLUION he repor has iscusse recursive ieificaio base o oliear blac bo sae space moels. I was firs eplaie why oliear sae space moels where he righ ha sies of more ha oe compoe of he ODE moel is parameerize, may easily lea o over-parameerizaio. uch moel srucures are herefore usuiable whe blac bo parameerizaios of oliear sae space moels are use. he recursive preicio error ieificaio algorihm propose i he repor was herefore base o a muli-variable polyomial parameerizaio of oe sigle righ ha sie compoe of he oliear ODE. May sysems are however physically moele by oliear ODE moels wih (srucure righ ha sies wih more ha oe orivial compoe. he applicabiliy of he propose RPEM o he ieificaio of such sysems was herefore aalyze. I was prove ha a geeral sae space moel wih more ha oe orivial righ ha sie compoe ca be (locally rasforme o he cooriae sysem of he moel use by he propose RPEM. he algorihm is hece applicable o a quie geeral class of oliear sysems. he meho was applie successfully o measure aa from a laboraory process cosisig of wo cascae as wih free oules, hereby verifyig all mai resuls of he repor eperimeally. Pracically impora algorihmic aspecs iclue iiializaio sice he oupu error formulaio suffers from a ris of covergece o a sub-opimal miimum poi of he crierio fucio. o improve algorihmic performace scalig ure ou o be a impora echique. I paricular scalig of he samplig perio applie i he algorihm was show o be a efficie meho o improve he umerical performace of he algorihm. I was show ha whe he algorihm coverges, he meho scales he saes of he moel accorig o he quoie bewee he scale a rue samplig perios. ice poor scalig bewee he saes is a severe problem whe powers of he saes are eploie i he oliear moel, he meho sigificaly improves he operaio of he 5

26 propose algorihm. Coveioal scalig of he assume parameer values ca of course also be applie. Furher suies of scalig is recommee for he propose a relae mehos. Oher ieresig research subjecs iclue he esig of projecio algorihms for oliear sysems as well as a closer suy of covergece a ieifiabiliy properies of his a relae mehos. REFERENCE Åsröm, K. J. a Elu, K (975. A simple o-liear rum boiler moel. I. J. Cor.,, Billigs,. A. a. Y. Fahouri (98. Ieificaio of sysems coaiig liear yamic a saic oliear elemes. Auomaica, 8, 5-6. Billigs,. A., M. B. Fazil, J. L. ulley a P. M. Johso (988. Ieificaio of a o-liear ifferece equaio moel of a iusrial iesel geeraor. Mech. ys. igal Processig,, Bohli,, (994. A case suy of grey bo ieificaio, Auomaica, 30, pp Che.. a. A. Billigs (989. Represeaio of oliear sysems: he NARMAX moel. I. J. Cor., 49, Che, a. A. Billigs (99. Neural ewors for oliear yamic sysem moellig a ieificaio. I. J. Corol, 56, Esam, L. a. me (987. Parameer esimaio i yamic sysems wih applicaio o power egieerig. echical Liceiae hesisis, UPEC 8747R, Uppsala Uiversiy, Uppsala, wee. Fuquis, J. (994. O moelig a ieificaio of a coiuous pulp igeser. Proc. YID 994, Copehage, Demar. Khalil, H. K. (996. Noliear ysems - seco eiio. Preice Hall, Upper ale River, NJ. Ljug, L. (975. heorems for he asympoic aalysis of recursive sochasic algorihms. Repor 75, Deparme of Auomaic Corol, Lu Isiue of echology, Lu, wee. Ljug, L. (977a. O posiive real rasfer fucios a he covergece of some recursive schemes. IEEE ras. Au. Cor., AC-, Ljug, L. (977b. Aalysis of recursive sochasic algorihms. IEEE ras. Au. Cor., AC-,

27 Ljug, L. (997. No-liear blac bo moels i sysem ieificaio. Proc. IFAC ymposium o Avace Corol of Chemical Processes, ADCHEM 97, Baff, Caaa, -3. Ljug L.a. Gla (004. Moellbygge och imulerig, : e. uelieraur, Lu, wee (i weish. Ljug, L. a. öersröm (983. heory a Pracice of Recursive Ieificaio. M. I.. Press, Cambrige, MA. Lueberger, D. G. (984. Liear a Noliear Programmig, : E.. Aiso-Wesley, Reaig, MA. Pajue, G. A. (99. Aapive corol of Wieer ype oliear sysems. Auomaica, 8, cheze, M. (980. he Volerra a Wieer heories of Noliear ysems. Joh Wiley, New Yor, NY. jöberg, J. a L. Ljug (99. Overraiig, regularizaio, a searchig for miimum i eural ewors. Proc. ymp. O Aapive sysems i Corol a igal Processig. Greoble, wizerla. jöberg, J., Q. Zhag, L. Ljug, A. Beveise, B. Delyo, P. Y. Gloreec, H. Hjalmarsso a A. Juisy. Noliear blac-bo moelig i sysem ieificaio: A uifie approach. Auomaica, 3, pp Weswic, D. a M. Verhage (996. Ieifyig MIMO Wieer sysems usig subspace moel ieificaio mehos. igal Processig, 5, Wieer, N. (958. Noliear Problems i Raom heory. he echology Press M. I.., a Joh Wiley a os, New Yor, NY. Wigre,. (993. Recursive Preicio Error Ieificaio Usig he Noliear Wieer Moel. Auomaica, 9, Wigre,. (994. Covergece aalysis of recursive ieificaio algorihms base o he oliear Wieer moel. IEEE ras. Au. Cor., 39, Wigre,. (003. User choices a moel valiaio i sysem ieificaio usig oliear Wieer moels. Proc. YID 003, Roeram, he Neherlas, Augus 7-9,

28 Figures a Capios Figure : Phoo of he cascae a laboraory process. 0 y( ime [s] 3 u( ime [s] Figure : he ipu sigal (boom a he oupu sigal, i.e. he measure waer level of he lower a (op. 8

29 0 y( a y m ( ime [s] 3 u( ime [s] Figure 3: he oupu aa ploe ogeher wih he simulae moel, usig parameers obaie a he e of he ru. he lower subplo shows he ipu sigal parameers ime [s] Figure 4: Evoluio of he parameer esimaes. 9

30 0 y( a y m ( ime [s] 3 u( ime [s] Figure 5: he aa a he simulae moel oupu, obaie wih liear parameers. his is obaie by seig he fourh, sih, seveh a eighh compoes of he parameer vecor equal o zero. 0.5 mea resiual umber of samples y( y( Figure 6: he resul of valiaio wih he mea resiual aalysis meho. 30