ECE 636: Systems identification

Transcription

1 ECE 636: Sysems ideificio Lecures 7 8 Predicio error mehods

2 Se spce models Coiuous ime lier se spce spce model: x ( = Ax( + Bu( + w( y( = Cx( + υ( A:, B: m, C: Discree ime lier se spce model: x( + = A( θ x( + B( θ u( + w( u( y( = C( θ x( + υ( G(q - A:, B: m, C: E{ ww } = R ( θ, E{ υυ } = R ( θ, E{ wυ } = R ( θ Alogy wih geerl LI model srucure: y Gq u H q e E dig ( = (, θ ( + (, θ (, { ee } = ( Λ( θ Gq (, θ = C( θ( qi A( θ B( θ AR model s SS model: y( + y( y( = e( x( = [ y( y(... y( ] + x( + = A( θ x( + B( θ u( + w( y( = C( θ x( + υ( A = B = [ ] C = [ ] u( = υ( = 0 w( = e( e( H(q - y( +

3 Model se: he se of ll cdide models cosidered for give prolem (e.g., ARX, ARMAX. Oce his is defied eleme of his se is seleced (model order selecio, eg e.g. for ARMAX models : selecio of,, c. Selecio of well suied model se d model order depeds geerlly o: flexiiliy, prsimoy, compuiol complexiy, properies of he cos fucio rue sysem y ( = G 0 ( q u ( + H 0 ( q e 0 ( Ee { 0 ( } = λ 0 Model y( G( q =, θ u( + H( q, θ( e Ee { ( } = λ ( θ Defie he se: D = { θ G0( q = G( q, θ, H0( q = H( q, θ, λ0 = λ ( θ} Uiqueess: D cois excly oe eleme he sysem is uiquely defied y he rue prmeer vecor θ=θ θ 0 Exmple ARMAX sysem d model A ( q y( = B ( q u( + C ( q e (, E{ e (} = λ Aq ( y ( = B ( q u ( + Cq ( (, e Ee { (} = λ For uique soluio he quiy * should e zero, where * = mi( 0, 0, c c0 Sysem ideifile: D o empy d he prmeer esime coverges o eleme of his se sympoiclly, i.e.: θˆ D, Prmeer ideifile: Sysem ideifile d D cois oly oe eleme

4 Les squres for LI sysems For ARX d FIR models we c formule s lier regressio: y ( + y ( y ( = u ( u ( + e ( Aq ( y ( = Bq ( u ( + e ( y ( = φ ( θ θ = [ ] FIR models: φ( = [ y(... y( u(... u( ] y Bq u e ( = ( ( + ( [ ] φ( = u(... u( θ =... Les squres esime: θ ˆ LS = ( ( ( y( φ φ = φ = For rue sysem of he sme srucure: A 0( q y( = B0( q u( + e0( or y ( = B0( q u ( + e0 ( we hve: θ ˆ LS θ 0 = ( ( ( e0( φ φ = φ = For cosise esimes ( E{ φ( φ (} o sigulr: for persisely exciig ipus of order his holds. E{ φ( e0 (} = 0 which holds if e 0 ( is whie oise or Ε{e 0 (}=0 d here re o erms y(-. he les squres mehod yields cosise esimes oly if hese codiios re sisfied. Moreover, if he rue sysem is e.g.armax we co ge cosise esimes. I geerl, he lower he SR is, he more is we will hve i our esimes if hese codiios re violed

5 Predicio error mehods Soluio: predicio error mehods or isrumel vrile mehods (ex Bsic cocep of predicio error mehods: Selechehe prmeer vecor so h he predicio errors ε (, θ = y( yˆ (, θ re smll for more geerlized model srucures I geerl, PEM mehods miimize cos fucio of he form: V ( θ = ( e F (, θ = ef (, θ = L( q ε (, θ e( w.r.. θ where ( ef (, θ is moooic, e.g. for ARX models les squres cos fucio: Lq ( = H(q - ( ε = ε For he geerl LI model srucure: y ( Gq ( =, θ u ( + H( q, θ(, e E{ ee } = digλ ( i ( θ u( We wish o fid lier oe sep predicor of he form G(q - y( + y ˆ(, θ = L( q, θ( u + L( q, θ( e L(0, θ = 0, L(0, θ = 0 herefore, he geerl PEM procedure is: Selecio of he predicor fucio: Usully we use he opiml me squre error predicor > > E { y ( yˆ (, θ } for which we ge: yˆ(, θ = H ( q, θ G( q, θ u( + [ H ( q, θ] y( d ε (, θ = H ( q, θ y( H ( q, θ G( q, θ u( = e( Selecio of model se: Prmerizio of G,H s fucio of θ (AR, ARMAX ec Selecio of cos fucio: Usully squre cos fucio θˆ = rg mi V θ ( θ Deermiio of θ h miimizes he cos fucio: I geerl, o lier opimizio prolem

6 Predicio error mehods C we oi lyic soluio for θ ˆ = rg mi V ( θ θ? Yes, if he predicor is lier wih respec o θ d we use les squressqures cos fucio, ie i.e. Τ y ˆ(, θ = φ ( θ d Τ V ( θ = [ y( y(, θ ] = y( ( φ θ = = he opiml oe sep predicors for ARX, FIR models sisfy his s: ˆ(, B ARX y θ = ( q u ( + [ Aq ( ] y( = y(... y( + u ( u( FIR y ˆ(, θ = Bq ( ( u = u ( u ( φ ( = [ y (... y ( u (... u ( ] φ ARX FIR ( = [ u (... u ( ] However, for ARMAX models he opiml me squre predicor is: y ˆ(, Bq ( ( u [ Aq ( ]( y [ Cq ( ](, θ = + + herefore y ( = φ (, θ θ φ(, θ = [ y (... y ( u (... u ( ε(, θ... ε(, θ] θ = [ c... c ] c ε θ c his is o lier regressio prolem ymore! I he firs cse (ARX/FIR we ge glol miimum I he secod cse (ARMAX we hve o resor o ierive opimizio mehods d we my hve locl miim, slow covergece, compuiolly complex procedures. We lso eed o iiilize he prmeer vecor θ wih iiil vlue h my ifluece he fil resul

7 Predicio error mehods How do we fid he soluio of θˆ = rg mi V θ ( θ? Ierive opimizio mehods Firs we hve o iiilize, i.e. selec vlue for Defie ε (, θ ψ(, θ = θ d lso he firs d secod grdie of he cos fucio w.r.. he prmeer vecor θ, i.e.: ' ' V ( θ V( θ = V( θ V( θ = i θ '' ' V ( θ ( θ = ( θ ( θ == V V V ij θi θ j For squred crierio fucio, i.e.: we hve: = i V ' V ( θ = ε (, θ ψ (, θ ˆθ (0 ( θ = ε (, θ = '' V ( θ = ψ(, θ ψ (, θ + (, θ he simples ierive mehod is grdie/seepes desce, wherey he upde rule is give y: ˆ( k+ ˆ( k ' ( k ˆ( k ( k ˆ( k θ = θ αkv( θ = θ αk ε(, θ ψ (, θ ( ε (, θ ε = = θθ α k is lerig cos which deermies he upde size ech sep Simple upde rule, u geerlly slow covergece = θ θ

8 Predicio error mehods We c opimize he lerig cos α k s follows: Secod order ylor expsio for he cos fucio roud θ (k : ˆ ( ( k ' ˆ ( k ˆ ( k ˆ ( k '' ˆ ( k ( ( ( ( ( ( ( ˆ ( k V θ V θ + V θ θ θ + θ θ V θ θ θ Se θ=θ (k+ ˆ( k+ ˆ( k ' ˆ( k ˆ( k+ ˆ( k ˆ( k+ ˆ( k '' ˆ( k ˆ( k+ ˆ( k V( θ V( θ + V( θ ( θ θ + ( θ θ V( θ ( θ θ ( d susiue ˆ( k+ ˆ ( k ' ( k θ = θ αkv( θ ˆ( k+ ˆ( k ' ˆ( k ˆ( k ' ( '' ( ' ( ˆ k ˆ k ˆ k V( θ V( θ αk V( θ + αk( θ θ ( V( θ V( θ V( θ Miimizig w.r.. α k yields he opiml lerig cos: ' ˆ ( k V ( θ α = k ' ˆ ( '' ˆ ( ' ˆ ( ( V ( θ k V ( θ k V ( θ k We c lso miimize ( direcly w.r.. θ(k+. his yields he ewo Rphso upde rule: ( k+ ( k '' ˆ( k ' ˆ( k θ = θ V( θ V( θ Fser covergece, u requires he compuio of he iverse mrix of he secod derivives of he cos fucio wih respec o he model prmeers. his mrix is lso clled he Hessi mrix. Is ij h eleme is: '' V ( θ V( θ = [ V( θ ] = ij ij θi θ j I geerl we c hve lerig cos differe h α k =,i.e. : ( k+ ( k '' ( k ' ( k = α k V V θ ˆ θ ˆ ( ˆ ( ˆ θ θ

9 Predicio error mehods I order o void hvig o clcule secod order derivives, modified upde rule ermed he Guss ewo mehod my e used for qudric cos fucios. ( θ = ε (, θ = I resuls from lierly pproximig he error roud θ (k : ˆ ( k ˆ( k ε (, θ ˆ( k ˆ( k ˆ( k ˆ( k ε(, θ = ε(, θ + ( θ θ = ε(, θ ψ (, θ ( θ θ θ Miimizig he sum of squres of his quiy wr w.r.. θ (k+ ˆ ( k + θ = rg mi θ ε (, θ V = his is lier regressio prolem of he sdrd form wih regressor vecor ψ Τ d regressio ˆ ( k coefficies ( θ θ. he LS soluio is: θ ˆ( k+ ˆ( k ˆ( k ˆ( k ˆ( k ˆ( k = θ + αk (, (, (, e(, ψ θ ψ θ = ψ θ θ = Slower covergece h ewo Rphso, simpler implemeio. Iiilizio: My ifluece he resuls cosiderly! Prolem depede. For exmple, we c re geerl srucure (ARMAX i sdrd les squres sese, oi esime of he prmeers d use his s he iiil esime i he ierive mehods ove.

10 Predicio error mehods Exmple ARMAX model Aq ( y ( = Bq ( u ( + Cq ( e ( Cq e Aq y Bq u ( (, θ = ( ( ( ( y ( + y ( y ( = u ( u ( + + e ( + ce ( c e ( c c Differeiio wih respec o i e (, θ e (, θ Cq ( = y ( i = y ( i i i C( q Differeiio wih respec o i,c i e (, θ e (, θ Cq u i u i ( = ( = ( i i C( q e (, θ e (, θ e i C q e i ( i, θ + C ( 0 ( i, c = i c = i C( q θ Afer compuig he sigls ove, defied s: F F F y ( = y(, u ( = u(, e ( = e( C ( q C ( q C ( q we hve: ψ(, θ = y F (... y F ( F (... F ( F (... F ( u u e e c

11 We c ow clcule he quiy: ' V ( θ = e(, θ ψ (, θ = Predicio error mehods which is vecor wih dimesio + + c. Is firs elemes re: ' F V ( θ = e(, θ y ( i, i =,,..., i = he remiig elemes re: V = e u j i= + + j = ' F ( θ (, θ (,,,...,,,,..., i = = = = ' F V ( θ e(, θ e ( j, i,,..., c, j,,..., i c = d we c e.g., use he Guss ewo mehod o clcule he upde rule

12 Predicio error mehods covergece Wh is he ehvior of he PEM esimes sympoiclly, i i.e. for?? Assumpios: he ipu oupu d re siory processes he ipu sigl is persisely exciig he mrix i V V ( θ is o sigulr les loclly ll roud he miim i of V ( θ he rsfer fucios Gq (, θ, Hq (, θ re smooh (differeile fucios of θ Relively wek codiios (hey ypiclly hold i prcice For ergodic d siory sigls he sum V ( (, coverges sympoiclly, i.e.: θ = ε θ ( (, V θ V θ s = oe: I he geerl cse we hve muliple oupu sysem we c cosider cos fucios of he form: V( θ = h( R( θ where h moooiclly icresig d R ( θ = ε(, θ ε (, θ = is he smple covrice mrix, e.g. we c cosider cos fucio of he form hr ( ( θ = rr ( ( θ (r: rce, i.e. he sum of he digol elemes, i oher words he vrice of he residuls correspodig o differe oupus I his cse lso, due o ergodiciy he cos fucio coverges sympoiclly: V( θ = h( R( θ h( R ( θ = V ( θ, s I c e show h he PEM esime θ ˆ coverges o miimum poi of V ( θ for his is vlid eve if he se D = { θ G0( q = G( q, θ, H0( q = H( q, θ, λ0 = λ ( θ} is empy, i.e. he model dlco descrie he rue sysem perfecly! Impor resul, s i gurees covergece eve i his cse: we oi pproximio h miimizes he residul vrice!

13 Cosisecy d sympoic lysis of PEM esimes Assume ow h D is o empy, i.e. here exiss vecor θ 0 such h he rue sysem c e wrie s: y ( Gq ( =, θ0 u ( + Hq (, θ0 e (, E{ e} = λ ( θ0 he PEM esime θˆθ is cosise for ope loop sysems i.e.: e: θ ˆ θ 0, s or equivlely θ ˆ D I oher words he sysem is sysem ideifile if he ssumpios i he previous slide hold If ddiiolly D cois oly oe eleme: prmeer ideifile Asympoic disriuio of he esimes Assumig h he sysem is prmeer ideifile he sympoic disriuio of he esimes is orml d specificlly: ( θ ˆ θ 0 (0, P s where he sympoic covrice mrix P evlued θ 0 is give y: Τ { E (, (, } 0 0 = λ P = λ ψ θ ψ θ ε (, θ ψ(, θ = θ I prcice we c esime his mrix from our oservios s: λ ( θ = E{e } 0 Τ = ˆ ˆ P = λ ψ(, θ ψ (, θ ˆ λ = ε (, θ =

14 Cosisecy d sympoic lysis of PEM esimes Exmple: Cosider he ARX model A( q y( = B( q u( + e( = φ Τ ( θ+ e( Τ I his cse we hve (lier regressio ε (, θ = y( φ ( θ ε (, θ ψ(, θ = = φ( θ herefore P = λ [ E{ φ( φ (}], which we hve see efore How does his resul compre o lier regressio prolem wih sic idepede vriles (regressors? I he dymic cse: he esime θˆθ is cosise, i.e.: e: θ ˆ θ 0, s d sympoiclly disriued s follows: ( θ ˆ θ 0 (0, P s P = λ [ E{ φ( φ (}] = λ φ( φ ( = Sic cse: For fiie legh d he esime θˆ is uised E{ θˆ } = θ0 d follows orml disriuio: ˆ ( θ θ0 (0, λ φ( φ ( =