SINCE most practical systems are continuous-time nonlinear

Transcription

1 Inrnaional Journal o Compur Scinc and Elcronics Enginring IJCSEE Volum, Issu 3 3 ISSN 3-4X; EISSN 3-48 Sparabl Las-squars Approach or Gaussian Procss Modl Idniicaion Using Firly Algorihm Tomohiro Hachino, Hioshi Takaa, Shigru Nakayama, Siji Fukushima, and Yasuaka Igarashi Absrac This papr prsns a sparabl las-squars LS approach combining h linar LS mhod wih irly algorihm FA o rain a Gaussian procss GP prior modl or nonlinar sysm idniicaion. Th hyprparamr vcor o h GP prior covarianc is sarchd or by FA, whil h sysm paramr vcor o h objciv sysm and h wighing paramr vcor o h GP prior man ar simad by h linar LS mhod. Th nonlinar uncion o h objciv sysm is simad as h prdiciv man uncion o h GP, and h conidnc masur o h simad nonlinar uncion is valuad by h prdiciv covarianc o h GP. Th proposd idniicaion mhod is applid o modling o a simpliid lcric powr sysm on numrical simulaion. Kywords Firly algorihm, Gaussian procss modl, idniicaion, nonlinar sysm, sparabl las-squars approach. I. INTRODUCTION SINCE mos pracical sysms ar coninuous-im nonlinar sysms, h dvlopmn o accura idniicaion algorihm o such sysms is a ky problm or prcis analysis or conrol dsign. For his problm, many paramric idniicaion mhods hav bn xploid using nural nwork modl [], orhogonal las-squars LS simaor [], radial basis uncion modl [3], [4], and so orh. Howvr, hs modls nd many wighing paramrs o dscrib h nonlinariy o h objciv sysms. Morovr, any conidnc masurs o h simad nonlinar uncions ar no obaind in such approachs. In rcn yars, h Gaussian procss GP modl has rcivd much anion or nonlinar sysm idniicaion [] [7]. Th GP modl was originally uilid or h rgrssion Tomohiro Hachino is wih h Dparmn o Elcrical and Elcronics Enginring, Kagoshima Univrsiy, Kagoshima, 89-6 Japan phon and ax: ; -mail: hachino@.kagoshima-u.ac.jp. Hioshi Takaa is wih h Dparmn o Elcrical and Elcronics Enginring, Kagoshima Univrsiy, Kagoshima, 89-6 Japan -mail: akaa@.kagoshima-u.ac.jp. Shigru Nakayama is wih h Dparmn o Inormaion Scinc and Biomdical Enginring, Kagoshima Univrsiy, Kagoshima, 89-6 Japan -mail: shignaka@ib.kagoshima-u.ac.jp. Siji Fukushima is wih h Dparmn o Elcrical and Elcronics Enginring, Kagoshima Univrsiy, Kagoshima, 89-6 Japan -mail: ukushima@.kagoshima-u.ac.jp. Yasuaka Igarashi is wih h Dparmn o Elcrical and Elcronics Enginring, Kagoshima Univrsiy, Kagoshima, 89-6 Japan -mail: igarashi@.kagoshima-u.ac.jp. problm by O Hagan [8] and has bn uilid or rgrssion or classiicaion problm [9] []. Th GP modl is a nonparamric modl and is naurally ino Baysian ramwork. Sinc i has wr paramrs han paramric modls such as h nural nwork modl, w can dscrib h nonlinariy o h objciv sysm in a w paramrs. Morovr, h GP givs us no only h man uncion bu also h covarianc uncion. In his papr, w dal wih a nonparamric idniicaion o coninuous-im nonlinar sysms using h GP modl. Th GP prior modl drivd by applying h dlayd sa variabl ilr has o b approprialy raind basd on h idniicaion daa. Gnrally his raining bcoms nonlinar opimiaion problm. In his papr, h sparabl LS approach combining h linar LS mhod wih irly algorihm FA is proposd o rain h GP prior modl. Th hyprparamr vcor o h GP prior covarianc is sarchd or by FA, whil h sysm paramr vcor o h objciv sysm and h wighing paramr vcor o h GP prior man ar simad by h linar LS mhod. FA is an opimiaion chniqu inspird by an inllign bhavior o irly swarms []. In FA, or any wo lashing irlis, h lss brighr irly movs oward h brighr on according o h aracivnss. Th aracivnss is proporional o h ligh innsiy obsrvd by h parnr and monoonically dcrass as h disanc bwn wo irlis incrass, owing o h invrs squar law and h absorpion propry o ligh. FA consiss o only h basic arihmic opraions and dos no rquir complicad coding and gnic opraions such as crossovrs and muaions o gnic algorihm GA. Morovr, h prormanc and compuaional cos o FA ar shown o b br han hos o ohr populaion-basd algorihms such as GA and paricl swarm opimiaion PSO [], [3]. Ths advanags suggs ha h us o FA incrass icincy whn h GP prior modl or idniicaion is raind. This papr is organid as ollows. In scion II, h problm is ormulad. In scion III, h GP prior modl or h idniicaion is drivd. In scion IV, h sparabl LS approach using FA is prsnd or raining h GP prior modl. In scion V, h nonlinar uncion wih h conidnc masur is simad rom h GP posrior disribuion. In scion VI, numrical simulaion or a simpliid lcric powr sysm is carrid ou o illusra h civnss o h proposd idniicaion mhod. Finally som conclusions ar givn in 44

2 Inrnaional Journal o Compur Scinc and Elcronics Enginring IJCSEE Volum, Issu 3 3 ISSN 3-4X; EISSN 3-48 scion VII. II. STATEMENT OF THE PROBLEM Considr a singl-inpu, singl-oupu, coninuous-im nonlinar sysm dscribd by a i p n i x = + i= i =n,n,,nα j= j =m,m,,m β b j p m j u a =, n m = [ p n n x,p n n x,,p n nα x, p m m u,p m m u,,p m m β u ] T y =x+ u and x ar h ru inpu and oupu signals, rspcivly. y is h noisy oupu ha is corrupd by h masurmn nois. is an unknown nonlinar uncion, which is assumd o b saionary and smooh. p dnos h dirnial opraor. n n i i =,,,α m and m j j =,,,β ar assumd o b known. Th purpos o his papr is o idniy h paramrs {a i } and {b j } o h linar rms and h nonlinar uncion wih h conidnc masur, rom h ru inpu and noisy oupu daa in h GP ramwork. III. GP PRIOR MODEL FOR IDENTIFICATION Equaion can b rwrin as p n y =w a i p n i y + i= i =n,n,,nα j= j =m,m,,m β b j p m j u+ε w =[p n n y,p n n y,,p n nα y, p m m u,p m m u,,p m m β u] T ε is an rror causd by h masurmn nois. Muliplying boh sids o by h dlayd sa variabl ilr F p [] yilds p n y =w a i p n i y + i= i =n,n,,nα j= j =m,m,,m β b j p m j u +ε u =F pu, y =F py, and w = F pw ar h ilrd signals, and ε is assumd o b ro man Gaussian nois wih varianc σ n. Puing =,,, N ino 3 yilds 3 y = v + Gθ l 4 y =[p n y,p n y,,p n y N ] T v =[w + ε,w + ε,,w N + ε N ] T θ l =[a,,a i,,a n,b,,b j,,b m ] T G =[g, g,, g N ] T g =[ p n y,, p n i y,, y, p m u,,p m j u,,u ] T A GP is a Gaussian random uncion and is complly dscribd by is man uncion and covarianc uncion. W can rgard i as a collcion o random variabls wih a join mulivariabl Gaussian disribuion. Thror, h uncion valus can b rprsnd by h GP: Nmw, Σw, w 6 =[w,w,,w N ] T w =[w, w,, w N ] w is h inpu o h uncion, mw is h man uncion vcor, and Σw, w is h covarianc marix. Th man uncion is on rprsnd by a polynomial rgrssion []. In his papr, h man uncion is xprssd by h irs ordr polynomial, i.., a linar combinaion o h inpu variabl: mw = w T θ m θ m = [ θ n,θ n,,θ nα,θ m,θ m,,θ mβ ] T 8 θ m is h unknown paramr vcor or h man uncion. Thus, h man uncion vcor mw is dscribd as ollows: 7 mw =w T θ m 9 Th covarianc Σ pq = sw p, w q is an lmn o h covarianc marix Σ, which is a uncion o w p and w q. Undr h assumpion ha h nonlinar uncion is saionary and smooh, h ollowing Gaussian krnl is uilid in his papr: Σ pq = sw p, w q = σy xp w p w q l dnos h Euclidan norm. Equaion mans ha h covarianc o h oupus o h nonlinar uncion dpnds only on h disanc bwn h inpus w p and w q. A high corrlaion bwn h oupus o h nonlinar uncion occurs or h inpus ha ar clos o ach ohr. Th ovrall varianc o h random uncion can b conrolld by σ y, and h characrisic lngh scal o h procss can b changd by l. 44

3 Inrnaional Journal o Compur Scinc and Elcronics Enginring IJCSEE Volum, Issu 3 3 ISSN 3-4X; EISSN 3-48 From 6, h vcor v o h noisy uncion valus in 4 can b wrin as v Nmw, Kw, w Kw, w =Σw, w+σ ni N I N : N N idniy marix and θ c = [σ y,l,σ n ] T is calld h hyprparamr vcor. From 4 and, h GP modl or h idniicaion is drivd as y Nmw+Gθ l, Kw, w 3 In h ollowing, Kw, w is wrin as K or simpliciy. IV. SEPARABLE LS APPROACH BY FA A h irs sag o h idniicaion, h GP prior modl is raind by opimiing h unknown paramr vcor θ = [θm, T θl T, θt c ] T. This raining is carrid ou by maximiing h log marginal liklihood o h idniicaion daa: J = logpy w, G, θ = log K y Zθ ml T K y Zθ ml N logπ 4 Z =[ w T... G ] θ ml =[θ T m, θ T l ]T Alhough his problm is a nonlinar opimiaion on, w can spara h linar opimiaion par and h nonlinar opimiaion par. Th parial drivaiv o 4 wih rspc o h paramr vcor θ ml is as ollows: J = Z T K y Z T K Zθ ml 6 θ ml No ha i h candidas o h hyprparamr vcor θ c o h covarianc uncion ar givn, h candidas o h covarianc marix K can b consrucd. Thror, h paramr vcor θ ml can b simad by h linar LS mhod rom 6: θ ml =Z T K Z Z T K y 7 Howvr, vn i h paramr vcor θ ml is known, h opimiaion wih rspc o θ c is a complicad nonlinar problm and migh sur rom h local opima problm. Thror, in his papr, w propos h sparabl LS approach combining h linar LS mhod wih FA. Only X = θ c =[σ y,l,σ n ] T is rprsnd wih h posiions o irlis in h sarch spac and is sarchd or by FA. Th daild raining algorihm is as ollows: sp : Iniialiaion Gnra an iniial populaion o Q irlis wih random posiions X [i] i =,,,Q. S h iraion counr l o. sp : Consrucion o h covarianc marix Consruc Q candidas o h covarianc marix K [i] using X [i] i =,,,Q. sp 3: Esimaion o θ ml Esima Q candidas o θ ml[i] i =,,,Q: θ ml[i] =Z T K [i] Z Z T K [i] y 8 sp 4: Ligh innsiy calculaion Calcula h ligh innsiy I i o ach irly rom h log marginal liklihood o h idniicaion daa: I i X [i] = log K [i] y Zθ ml[i] T K [i] y Zθ ml[i] N logπ 9 sp : Soring o h irlis Sor h irlis in ascnding ordr o hir ligh innsiis and ind h currn bs posiion: X l bs = X [Q] sp 6: Movmn o h irlis I I i X [i] I j X [j], mov a irly i a posiion X [i] oward a brighr irly j a posiion X [j] by X [i] = X [i] + β xp γrij X [j] X [i] +α l rand r ij is h Euclidan disanc bwn X [i] and X [j], β is h aracivnss a r ij =, γ is h mdia absorpion coicin, α l is h randomiaion paramr, and rand is uniormly disribud random numbr wih ampliud in h rang [.,.]. β = β xp γrij is h aracivnss bwn h irlis i and j. sp 7: Rpiion S h iraion counr o l = l + and go o sp unil h prspciid iraion numbr l max. sp 8: Drminaion o h GP prior modl Drmin h vcor ˆX = ˆθc = [ˆσ y, ˆl, ˆσ n ] T and h corrsponding paramr vcor ˆθ ml =[ˆθ m, T ˆθ l T]T using h bs posiion X lmax bs o irly. Consruc h subopimal prior man uncion and prior covarianc uncion: mw = w T ˆθm sw p, w q = ˆσ y xp w p w q ˆl kw p, w q = sw p, w q + ˆσ nδ pq, 3 sw p, w q is an lmn o covarianc marix Σ, kw p, w q is an lmn o covarianc marix K, and δ pq is h Kronckr dla, which is i p = q and ohrwis. 44

4 Inrnaional Journal o Compur Scinc and Elcronics Enginring IJCSEE Volum, Issu 3 3 ISSN 3-4X; EISSN x. u x. u.. Fig. Tru nonlinar uncion Fig. 3 Absolu rror bwn ru and simad nonlinar uncions - - n io g r c n id n o c - % x. u x. u.. Fig. Esimad nonlinar uncion Fig. 4 9.% conidnc rgion V. ESTIMATION OF THE NONLINEAR FUNCTION For a nw inpu w and h corrsponding uncion w, w hav h ollowing join Gaussian disribuion: [ ] [ ] y mw+g ˆθl w N mw, [ K Σw, w ] Σw, w, sw, w 4 From h ormula or condiioning a join Gaussian disribuion [4], h posrior disribuion or spciic s daa is w w, G, y, w N ˆw, ˆσ h man uncion ˆ is givn as ˆw = mw +Σw, wk y mw G ˆθ l 6 which is usd as h simad nonlinar uncion o h objciv sysm. And is covarianc ˆσ is valuad as ˆσ =sw, w Σw, wk Σw, w 7 which is usd or h conidnc masur o h simad nonlinar uncion. VI. ILLUSTRATIVE EXAMPLE Considr an lcric powr sysm [] dscribd by ẍ+a ẋ = = P M + P in M = P m M + u sin x+p in M =[x,u] T y =x+ 8 x =δ phas angl u =ΔE d incrmn o xciaion volag, M inria coicin D damping coicin P gnraor oupu powr, P in urbin oupu 443

5 ¹ ¹ Inrnaional Journal o Compur Scinc and Elcronics Enginring IJCSEE Volum, Issu 3 3 ISSN 3-4X; EISSN x ^ x mhod CPU: InlR CorTM i7-64m.8gh. W can conirm ha h proposd mhod givs mor accura modl o h objciv lcric powr sysm wih smallr compuaional burdn. x, x Fig. Tru oupu and oupu by h simad modl powr. In numrical xampl, M = D =.6, Pm =. P in =.8 and a = D/ M =. ar s. Th masurmn nois is whi Gaussian nois, nois-o-signal raio is abou.%. Th numbr o inpu and oupu daa or idniicaion is akn o b N = 8. Th hird-ordr Burworh ilr wih h cuo rquncy ω c =rad/s is uilid as a dlayd sa variabl ilr. Th sing paramrs o FA ar chosn as ollows: i irly si: Q = ii aracivnss a r ij =: β =. iii mdia absorpion coicin: γ =. iv randomiaion paramr: α l =..97 l v maximum iraion numbr: l max = Th hyprparamr vcor o h covarianc uncion has bn drmind by FA as ˆθc = [ˆσ y, ˆl, ˆσ n ] T = [4.87,.343,.8] T. Esima o h paramr in h linar rm is â =.96, which is vry clos o h ru valu a =.. Th ru nonlinar uncion, h simad nonlinar uncion ˆ, h absolu rror bwn and ˆ, and h doubl sandard dviaion conidnc inrval 9.% conidnc rgion around h simad nonlinar uncion ar shown in Figs. 4, rspcivly, h hick curvs dpic h rajcoris o h idniicaion daa. Clarly h simad nonlinar uncion ˆ is shown o b vry clos o h ru nonlinar uncion on h daa rgion. Th conidnc rgion o h simad nonlinar uncion grows as gos away rom h daa rgion. On h ohr hand, h conidnc rgion o h simad nonlinar uncion is vry small on h daa rgion. Fig. shows h ru oupu x and h oupu ˆx by h simad modl, h oupus wr gnrad by h inpus or validaion. This igur indicas ha ˆx machs x considrably. For comparison, h PSO-basd GP PSOGP mhod [7] has bn carrid ou or his idniicaion problm. Th man squars rror o h oupu N k= x k ˆx k /N is.6 4 or h proposd mhod and.77 4 or PSOGP mhod. Th compuaional im o h raining is 4.7 s or h proposd mhod and 33.4 s or PSOGP VII. CONCLUSIONS In his papr, w hav proposd a sparabl LS approach combining h linar LS mhod wih FA o rain a GP prior modl or nonlinar sysm idniicaion. Th GP prior modl is raind by maximiing h log marginal liklihood o h idniicaion daa. Th proposd idniicaion mhod is cagorid ino h nonparamric idniicaion and dos no nd h drminaion o h modl srucur. Sinc FA is simpl and has a high ponial or global opimiaion, h proposd raining algorihm maks sysm idniicaion b mor icin. Simulaion rsuls show ha h proposd mhod can b succssully applid o modling o h lcric powr sysm. REFERENCES [] C. Z. Jin, K. Wada, K. Hirasawa, and J. Muraa, Idniicaion o nonlinar coninuous sysms by using nural nwork compnsaor in Japans, IEEJ Trans. C, Vol. 4, No., pp. 9 6, 994. [] K. M. Tsang and S. A. Billings, Idniicaion o coninuous im nonlinar sysms using dlayd sa variabl ilrs, Inrnaional Journal o Conrol, Vol. 6, No., pp. 9 8, 994. [3] G. P. Liu and V. Kadirkamanahan, Sabl squnial idniicaion o coninuous nonlinar dynamical sysms by growing radial basis uncion nworks, Inrnaional Journal o Conrol, Vol. 6, No., pp. 3 69, 996. [4] T. Hachino, I. Karub, Y. Minari, and H. Takaa, Coninuous-im idniicaion o nonlinar sysms using radial basis uncion nwork modl and gnic algorihm, Proc. o h h IFAC Symposium on Sysm Idniicaion, Vol., pp ,. [] J. Kocijan, A. Girard, B. Banko, and R. Murray-Smih, Dynamic sysms idniicaion wih Gaussian procsss, Mahmaical and Compur Modlling o Dynamical Sysms, Vol., No. 4, pp. 4 44,. [6] T. Hachino and H. Takaa, Idniicaion o coninuous-im nonlinar sysms by using a Gaussian procss modl, IEEJ Trans. on Elcrical and Elcronic Enginring, Vol. 3, No. 6, pp. 6 68, 8. [7] T. Hachino, S. Yonda, and H. Takaa, Gaussian procss modls raind by paricl swarm opimiaion or coninuous-im nonlinar sysm idniicaion, Proc. o Inrnaional Symposium on Nonlinar Thory and Is Applicaions, pp. 76 8,. [8] A. O Hagan, Curv iing and opimal dsign or prdicion wih discussion, Journal o h Royal Saisical Sociy B, Vol. 4, pp. 4, 978. [9] C. K. I. Williams, Prdicion wih Gaussian procsss: rom Linar rgrssion o linar prdicion and byond, in Larning and Inrnc in Graphical Modls, Kluwr Acadmic Prss, pp. 99 6, 998. [] M. Sgr, Gaussian procsss or machin larning, Inrnaional Journal o Nural Sysms, Vol. 4, No., pp. 38, 4. [] C. E. Rasmussn and C. K. I. Williams, Gaussian Procsss or Machin Larning, MIT Prss, 6. [] X. S. Yang, Firly algorihms or mulimodal opimiaion, in Sochasic Algorihms: Foundaions and Applicaions, SAGA 9, Lcur Nos in Compur Scincs, Vol. 79, pp , 9. [3] S. K. Pal, C. S. Rai, and A. P. Singh, Comparaiv sudy o irly algorihm and paricl swarm opimiaion or noisy non-linar opimiaion problms, Inrnaional Journal o Inllign Sysms and Applicaions, Vol. 4, No., pp. 7,. [4] R. von Miss, Mahmaical Thory o Probabiliy and Saisics, Acadmic Prss, 964. [] H. Takaa, An auomaic choosing conrol or nonlinar sysms, Proc. o h 3h IEEE CDC, pp ,