13. Parametric and Non-Parametric Uncertainties, Radial Basis Functions and Neural Network Approximations

Transcription

1 Lecture 7 3. Parametrc ad No-Parametrc Ucertates, Radal Bass Fuctos ad Neural Network Approxmatos he parameter estmato algorthms descrbed prevous sectos were based o the assumpto that the system ucertates F ( xu, ) ca be represeted exactly by a lear combato of kow bass fuctos Φ ( x, u) ad ukow costat parameters Θ. hs lear combato Θ Φ ( x, u) represets the system parametrc ucertaty. Ofte practce, the truth model (, ) (, ) x& = f x u + F x u Kow Ukow (3.) cotas umodelled dyamcs. Also, the system wll be typcally subject to measuremet ose, uexpected falures, evrometal dsturbaces, etc. hese evets costtute the so-called o-parametrc ucertates. Because of the latter, a more realstc model of the system ucertates ca be wrtte the form: (, ) =Θ Φ (, ) + ε (, ) F x u x u x u Ucertaty Parametrc No-Parametrc (3.) I (3.), the ukow fucto F( x, u ) s approxmated by lear combato of kow bass fuctos Φ ( xu, ) = ( ϕ K ϕ N ) ad ukow costat parameters Θ. he approxmato error ε ( x) represets the o-parametrc ucertaty, whle the ukow costat parameters Θ deotes the parametrc ucertaty the system dyamcs. I order to characterze the latter, oe eeds to be able to fd a good set of bass fuctos x, u x x, u Φ such that the approxmato error ε becomes small o a compact doma. Polyomals, Fourer seres expasos, sples ad feedforward eural etworks ca be used to approxmate fuctos o compact domas. Defto 3. A Radal Bass Fucto (RBF) s defed as a Gaussa: (, ) c ( x x ) W( x x ) c c x xc W ϕ xx = e = e (3.3) 75

2 I (3.3), x R s the put, xc R s the ceter, ad W W ϕ x, xc = > s a postve-defte = ϕ x to abbrevate symmetrc matrx of weghts. Most ofte, we wll wrte ad deote a RBF whch s cetered at the th ceter. Remark 3. Other deftos of a RBF are avalable. Ofte lterature, a RBF s defed as ϕ = ϕ( x x c W ), where deotes the usual Eucldea weghted orm. I addto, t W s requred that ϕ () s tegrable o R ad ϕ ( x) dx. Bascally, ths type of RBF depeds oly o the weghted dstace r = x xc betwee ts curret put x ad the W ceter x c. he Gaussa RBF (3.3) s a example of ths type of actvato fucto. Others clude: multquadrcs verse multquadrcs heorem 3. (Mcchell, 986) Let ϕ ϕ( r) x Let { } whose (, ) R ϕ r = r + c, c> ϕ =, c> ( r) ( r + c ) = be the Gaussa, the multquadrcs, or the verse multquadrcs fucto. N = be a set of dstct pots R. he the ( N N) th j elemet s ϕj = ϕ( x x j ), s osgular. terpolato matrx Φ, Referece: C. A. Mcchell, Iterpolato of scattered data: Dstace matrces ad codtoally postve defte fuctos, Costructve Approxmato, vol., pp. -, 986. Defto 3. Artfcal Feedforward Neural Networks (NN) are mult-put-mult-output systems composed of may ter-coected olear processg elemets (euros) operatg parallel. 76

3 Fgures 3. ad 3. show sketches of two feedforward NN-s. 4 Network Iput 5 Network Output Iput Node 3 6 Output Frst Hdde Layer Secod Hdde Layer Fgure 3.: Feedforward Neural Network wth hdde layers ad 6 euros Fgure 3.: Feedforward Neural Network wth hdde layer ad 5 euros As see from the examples, a artfcal feedforward eural etwork cossts of euros ad ther coectos. A block-dagram of a euro s show below. 77

4 Fgure 3.3: Artfcal Neuro Block-Dagram Neuros, the basc processg elemets of NN-s, have two ma compoets: a weghted summer a olear actvato fucto he actvato fuctos of terest are Radal Bass Fuctos. Defto 3.3 A feedforward RBF NN s defed as: ( x C W ) ( x) ϕ ϕ NN ( x) θ b ( θ b) M = M + = =Θ Φ x 443 ϕ N ( x) ϕ ( x C Θ N W ) N 443 Φ ( x) (3.4) N + m where ( θ ) Θ= b R s the vector of weghts, receptve feld, W = W > s the orm weghtg matrx, b ( ) N + x ϕ x ϕn x R C R s the ceter of the th m R s the NN bas, ad Φ = K s the regressor vector, whose compoets are the bass actvato fuctos ϕ ( x) ϕ( x C ) = ad the uty fucto. W Remark 3. I practcal applcatos, the symmetrc postve-defte matrx W (3.4) s chose to be dagoal ad the form: W =, =, K, N σ where σ represets the wdth of the th Gaussa fucto, that s: ϕ x = e x C σ compoet of the regressor vector becomes the th Φ x (3.4). Moreover, the compoets of the regressor are costructed usg a sotropc Gaussa fucto 78

5 ϕ x = e N x C d whose stadard devato (.e., wdth) σ s fxed accordg to the spread of the ceters C, N s the umber of ceters, ad d s the mum dstace betwee the chose ceters. I ths case, the stadard devato σ of all the sotropc Gaussa RBF compoets s fxed at σ = d N hs formula esures that the dvdual RBF-s are ot too peaked or too flat. Both of these two extreme codtos should be avoded. Feedforward NN-s are capable of approxmatg geerc classes of fuctos, cludg cotuous ad tegrable oes, o compact domas, ad to wth ay tolerace. hs property of the feedforward NN-s s ofte referred to as the Uversal Approxmato, whle the NN-s themselves are ofte called the uversal approxmators. Uversal Approxmato heorem for RBF NN-s, (Park ad Sadberg, 99) Let ϕ ( x): R R be a tegrable bouded cotuous fucto ad assume that R ϕ ( x) dx he for ay cotuous fucto f ( x ) ad ay r, ε > there s a RBF NN wth N euros, a set of ceters { } N C =, ad a commo wdth σ > such that N ˆ x C f ( x) = θϕ =Θ Φ x = σ 443 ϕ ( x) f ( x) NN( x) = ( f ( x) NN( x) ) dx ε O N L = x r Refereces: Park, J., ad Sadberg, I.W., Uversal approxmato usg radal-bass-fucto etworks, Neural Computato, vol. 3, o., pp

6 Scarsell, F., so, A.C., Uversal approxmato usg feedforward eural etworks: A survey of some exstg methods, ad some ew results, Neural Networks, vol., No., pp. 5-37, 998. Hut, K.J., Sbarbaro, D., Zbkowsk, R., Gawthrop, P.J., Neural Networks for Cotrol Systems A Survey, Automatca, vol. 8, No. 6., pp. 83-, Robust System ID usg Dyamc Models ad Projecto Operator Assumpto 4. Cotrol put u ad correspodg trajectores of the system (3.) evolve o compact x, u doma: ( m, : ) X U = x R u R x x u u (4.) where x, u are kow postve costats. Usg the uversal approxmato property of RNF NN-s, the represetato (3.) for the ucerta fucto F( x, u ) (3.) ca ow be theoretcally justfed. Formally speakg, for ay tolerace ϕ x, u, ad a ε >, there must exst N RBF odes matrx of costat (ukow) coeffcets Θ, such that for all ( x, u) X U : (, ) (, ) ε (, ) ε F x u Θ Φ x u = x u (4.) Cosequetly, the system dyamcs takes the form: where ( x, u) x& = f x, u +Θ Φ x, u +ε x, u (4.3) ε represets bouded o-parametrc ucertaty. he goal of terest s to perform system ID the presece of the o-parametrc ucertates. Remark 4. If the system ID laws are chose as (.7), that s f ˆ & Θ= ΓΦ x, u e P (4.4) 8

7 the the tme dervatve of the Lyapuov fucto caddate (.) becomes: (, ) e Pe trace V e ΔΘ = + ΔΘ Γ ΔΘ (4.5) (, ) ΔΘ = + ε (, ) V & e e Qe e P x u (4.6) he d term (4.6) s due to the o-parametrc ucertaty ε. A upper boud for V & ca be computed as: V & e, ΔΘ λ Q e + e P ε = e λ Q e P ε (4.7) m m From (4.7) t follows that the tme dervatve s egatve outsde of the compact set P ε = = λm ( Q) E e R : e r (4.8) Cosequetly, the predcto error et eters the compact set E wth fte tme ad remas the set for all future tme. hus, the predcto error sgal e s uformly bouded. However, othg ca be sad about boudedess of the parameter estmato errors ΔΘ () t. I fact, whle the predcto error stays bouded, the parameter estmato errors ca grow ubouded. hs s caused by the presece of the o-parametrc ucertaty ε. Fgure 4. llustrates the udesrable effects due to the presece of the o-parametrc ucertates the system dyamcs. e r ΔΘ Fgure 4.: Udesrable Effects of No-Parametrc Ucertates I order to prevet the estmated parameters from growg, the Projecto Operator wll be troduced to the estmato laws. 8

8 5. he Projecto Operator Defto 5.: Subset Ω R s covex f [ x y Ω R ] λ x + ( λ) [ y = z Ω], λ, (5.) Relato (5.) states that f two pots belog to the covex subset Ω the all the pots o the coectg le also belog to Ω. Defto 5.: Fucto f : R R s covex f ( λ x + ( λ) y) λ f ( x) + ( λ) f ( y), λ f (5.) Iequalty (5.) s llustrated Fgure 5.. It states that the graph of a covex fucto must be located below the straght le, whch coects ay two correspodg fucto values. f(x) f(y) f(z) x z y Fgure 5.: A Covex Fucto Lemma 5. Let f ( θ ): R R be a covex fucto. he for ay costat δ > the subset Ω δ = { θ R f ( θ ) δ } Proof: Let θ θ Ω ay λ herefore ( θ ) δ s covex.,. he ( θ ) δ δ f λθ θ f ad f ( θ ) δ. Sce ( x) ( λ) θ λ f ( θ ) + ( λ) f ( θ ) λ δ + ( λ) δ = δ 3 3 δ δ f, ad cosequetly, θ Ωδ whch completes the proof. f s covex the for 8