WEIGHTED LEAST SQUARES ESTIMATION FOR THE NONLINEAR OBSERVATION EQUATIONS MODELS. T m. i= observational error

Transcription

1 WEIGHED LEAS SUARES ESIMAION FOR HE NONLINEAR OBSERVAION EUAIONS MODELS Unknowns: [ L ], Osrvals: [ ] Mathatical odl (nonlinar): i fi (,, K, n ), i,, K, n. n L n> Osrvations: i i + i, i,, K, n, i osrvational rror Osrvation quations odl in atri for: f ( ), + f ( ) + Wightd last squars principl: P in P stric ( P P) positiv dfinit ( z Pz>, z ) wight atri f ( ) φ P [ f ( )] P[ f ( )] in Solution: φ [ f ( )] P[ f ( )] ( ) ( ) [ f ( )] P [ f ( )] [ ( )] [ ( )] [ ( )] f P f f [ f ( )] P{ f ( )} [ f ( )] φ f ( ) [ f ( )] P ( ) f Nonlinar noral quations: ( ) P[ f ( )] Solution nurical thods. quations in unknowns WEIGHED LEAS SUARES ESIMAION FOR HE LINEAR (OBERVAION EUAIONS) MODEL Spcial cas of linar odl: f ( ) A+ c: f A h nonlinar noral quations co th linar noral quations: A P[ A c] (Linar) noral quations: A PA A P( c ) or N u Solution: ( ) ( ) A PA A P c or N u. h noral quations can also drivd dirctl: + A+ c+

2 A c φ P ( A c) P( A c ) in φ ( ) ( A c) P( A) : A P( A c) : ( A PA) A P( c ) LINEARIZAION h nonlinar odl f ( ) rlating osrvations to unknowns can linarizd alor pansion around so known approiat valus [ K ] n and oitting scond and highr ordr trs. Fro f f ( ) f (,, K ) f (,, K, ) (,, K, )( ), i,,..., n, i i i i + j j j j w hav in atri for f f ( ) f ( ) + ( )( ) + Aδ whr f f( ),(approiat osrvals) A ( ), (dsign atri), δ, (rducd unknowns). Rplacing f( ) + Aδ in th original odl f ( ) +, w otain th linarizd odl Aδ + whr f ( ) ar th rducd osrvations. φ P ( Aδ ) P( Aδ ) in: φ A P( Aδ ) δ Noral quations: A PAδ A P or N u Solution: δ ( ) A PA A P or N u.

3 ALERNAIVE FORMS OF NOLINEAR MODELS AND HEIR LEAS SUARES SOLUIONS Starting point: osrvation quations odl. Stochastic odl: +, E{ }, E{ } σ Dtrinistic or functional odl f ( ) n quations with < n unknowns Eliination of unknowns: It is possil to liinat q< unknowns and rain with q unknowns and n q quations Eapl: f, ( n q) ( n q) ( q) q n q f, q ( q) q f,, f, q q ( q) q : ( n q) ( n q) ( q) q Solv th q quations f(, ) for th q unknowns : ψ(, ) Rplac in (, ) f : f (, ψ(, )) Gnral for aftr liination: h(, ) s quations in a nw rducd st of unknowns. Not: th diffrnc f s rains th sa as in th osrvation quations whr f n f dgrs of frdo (css inforation) Mid quations odl (also calld condition quations with unknowns): +, h(, ) A godtic variant: rplac to otain th Gauss-Hlrt odl: h(, ) h% (, ) Condition quations odl: Eapl: f ( n ) ( n ) f n Spcial cas: Eliination of all th unknowns! ( ), f ( ) f ( ) f ( ) ( n ) ( n ) ( ) Solv th quations f ( ) for th unknowns: ψ( ) and rplac in ( ) f : f ( ψ( ) ) Gnral for aftr liination: g( ) s n quations with no unknowns. Condition quations odl: +, g( ) A godtic variant (also calld condition quations): g( ) g% ( ).

4 Non-linar last squars stiation for th id quations odl: P ( ) P( ) in sujct to h(, ) Lagrangan: Φ ( ) P( ) k h(, ) Φ ( ) P k (, ) : P( ) (, ) k Φ k (, ) : (, ) k Φ h(, ) : h(, ) k St A (, ) (, ), B (, ) (, ) s s n Non-linar solution sst: P( ) B (, ) k, A (, ) k, h(, ) n+ + s quations in n+ + s unknowns (, n, k ). n s Non-linar last squars stiation for th condition quations odl: P ( ) P( ) in sujct to g( ) Lagrangan: Φ ( ) P( ) k g( ) Φ g g ( ) P k ( ) : P( ) ( ) k Φ g( ) : g( ) k g St: B( ) ( ) Non-linar solution sst: P( ) B ( ) k, g( ) n+ s quations in n+ s unknowns (, n k ). s

5 Wightd last squars solution for th linarizd id quations odl: Linarization using approiat valus, ( in gnral) h(, ) h(, ) + (, )( ) + (, )( ) whr: h h(, ), A (, ) Last squars solution: (, ), B +, δ δ Aδ + Bδ + h :, δ., δ (rducd osrvations) P ( ) P( ) in sujct to Aδ + Bδ + h Lagrangan: Φ ( δ ) P( δ ) k ( Aδ + Bδ + h ) Φ δ k A : A k Φ ( δ ) P k B : P( δ ) B k δ Φ ( Aδ + Bδ + h) k Aδ + Bδ + h : P( δ ) B k : δ P B k Rplac δ P B k in Aδ + Bδ + h : Aδ + B BP B k+ h Solv for k: k ( BP B ) ( Aδ + B+ h ) M ( Aδ + w ), M BP B, w B+ h Rplac in A k : A k A M ( Aδ + w) A M Aδ + A M w Solv for δ : δ ( A M A) A M w k M ( Aδ + B+ h ): δ ( ) P B k P B M Aδ + w Solution: M BP B, N A M A δ N A M w δ P B M ( Aδ + w) P B M [ I AN A M ] w +δ, + δ ( ) P B M I AN A M w Misclosur: ( h, ) h(, ) + (, )( ) h+ B w Iprovnt itration: If ( i ), ( i ) is th solution in th i itration stp, w us ( i) ( i), as approiat valus in th nt i itration stp. h itration starts with an aritrar ut clos ( i) ( i) to th unknowns valu () and ().

6 Wightd last squars solution for th linarizd condition quations odl: Linarization using approiat valus ( in gnral) g g( ) g( ) + ( )( ) g+ B( ) Bδ + g : Bδ + g g whr B ( ) g g( ), δ, δ, (rducd osrvations) Last squars solution: P ( ) P( ) in sujct to Bδ + g Lagrangan: ( ) Φ δ P( δ ) k ( Bδ + g ) Φ ( δ ) P k B : P( δ ) B k : δ δ P B k Φ ( Bδ + g ) k Bδ + g : Rplac δ P B k in Bδ + g : B BP B k+ g Solv for k: ( ) ( ) k BP B B+ g M ( B+ g ), M BP B Rplac k M ( B+ g ) in δ P B k: δ P B M ( B+ g ) g Misclosur: ( w g ) g( ) + ( )( ) g + B δ P B M w, + δ P B M w Error stiat: δ Iprovnt itration: If ( i ) is th solution in th i itration stp, w us ( i) ( i) as approiat valus in th nt i itration stp. h itration starts with ().

7 PROBABILISIC ESIMAION IN HE LINEAR MODEL HE GAUSS-MARKOV HEOREM Suppos that in th linar odl A+ w assu that th rrors ar rando varials with zro an ( E{ } ) and covarianc atri known up to a scalar function, i.. { E } σ, with known and σ unknown. W ar gnrall intrstd in otaining not onl optial stiats of th unknowns, ut also optial stiats q of an quantit q a, which is a linar function of th unknowns. W sk BLUE stiats (E) which ar linar (L) functions of th data q d + γ uniforl uniasd (U), i.. th satisf E{ q } q, and st (B) in th sns of having iniu an squar rror E{( q q) in. q d, γ W thus sk to dtrin th valus of d and γ which ak q d + γ a BLUE of E{ } E{ A+ } A+ E{ } A, E{ q } d E{ } + γ d A + γ th uniasdnss condition givs q a. Noting that E{ q } d A+ γ q a, γ. d A a, h scond condition rstricts th stiat to th for q d. h first on a th quantitis d A poss a rstriction on q a for which an uniasd stiat can found. A quantit q can uniasdl stiatd if and onl if it is of th for q a d A and upon rcognizing that A ar th osrvals w arriv at th conclusion that: A quantit q can otain an uniasd stiat onl if it can prssd as a function q d of th osrvals. In statistics an stial quantit is dfind on for which an uniasd an uniasd stiat ists. his a lad to th wrong conclusion that stiailit is a proailistic notion. W hav shown that a quantit is stial if and onl if it can prssd as a function of th osrvals. his condition is indpndnt of th ffct of rrors on th osrvations +, and sinc th rrors ar th onl proailistic factor in our odl, stiailit is a purl dtrinistic notion. h an squar rror (with γ ) cos E{( q q) } E{( d a ) } E{[ d ( A+ ) a ) } E{( d A+ d a ) } q E{[ d + ( d A a ) ] } E{( d ) + ( d A a ) d + [( d A a ) ] } E{ d d+ ( d A a ) d + [( d A a ) ] } d E{ } d+ ( d A a ) d E{ } + [( d A a ) ] σ d d+ [( d A a ) ]

8 o iniiz σ d d+ [( d A a ) ] undr th raining uniasdnss condition q (quivalntl A d a ) w for th Lagrangan d A a σ d d [ ( A d a)] k ( A d a ) Φ + Solution: Φ + [ ( )] d σ d A d a A k A : σ + ( ) d A A d a Ak Φ ( A d a) : k A d a. In viw of th scond th first cos σ d Ak, so that dσ Ak. Rplacing this in th scond w gt σ A Ak a and thus k σ ( A A) a. With this valu of k, dσ Ak cos d A( A A) a and th dsird stiat q d cos q ( ) a A A A. his BLUE stiat can applid to an unknown k sipl prssing it as k ik, whr i k is th k colun of th idntit atri I [ ilikl i]. hus rplacing a i k w hav i ( A A) A and coining all stiats k k ( ) A A A. Coparing this to th last squar stiat ( A PA) A P, th two stiats coincid for w arriv to th clratd Gauss-Markov thor. P. hus Gauss-Markov thor: h BLUE stiats of th unknowns in a linar odl A+, ~ (, σ ), coincid with th last squars stiat, whn th wight atri is qual to th invrs of th covarianc atri of th rrors, cpt fro an (aritrar) scalar factor. Evr stial function of th unknowns q a has q a as BLUE stiat. Rark: h Gauss-Markov thor can asil tndd to th linarizd condition and id quations od. For condition quations Bδ + g w sk BLUES q d + γ of q δ. For id quations Aδ + Bδ + h w sk BLUES q d + γ of q a δ + δ. Miniization of th an squar rror ( varianc) undr th conditions drivd fro th unifor uniasdnss condition (for vr δ, or for vr δ and δ, rspctivl) lads to th sa solution as in last squars with th wight atri rplacd P.

9 ACCURACY OF ESIMAES Osrvation quations: Covarianc propagation: ( ) ( ) ( ) ( ) ( ) A PA A P A PA A P A+ A PA A PA+ A PA A P ( ) + A PA A P + N A P Covarianc factor atri: ( ) N A P N A P N A PP PAN N NN N, Covarianc atri: C σ N Error stiat: ( ) I AN A P ( )( + ) ( ) + ( ) A AN A P I AN A P A A AN A PA I AN A P St: H I AN A P: H. Eas to prov: H H, H PHP H I AN A P n N A PA n I n tr tr n tr( ) tr( ) tr tr( ) tr( ) P H PH PHP PH PH PH PH PH E P E PH σ P PH σ H σ n { } tr( { } ) tr( ) tr ( ) σ P is an uniasd stiat of σ : n E{ σ } σ Estiatd covarianc atri: C σ N Mid quations: Covarianc propagation: δ : δ +, Aδ + Bδ + h : Bδ + h Aδ δ N A M ( B+ h ) N A M ( Bδ + B+ h ) N A M ( Aδ + B ) N A M Aδ N A M B δ N A M B Covarianc factor atri: δ ( N A M B) ( N A M B) N A M BP B M AN N A M MM AN N A M AN N N δ St: H I M AN A. Eas to prov: H H, M H HM, HM A

10 δ + P B M ( AN A M I)( B+ h ) δ + + P B ( M AN A I) M ( Bδ + B+ h ) P B HM ( B B h) P B HM ( B A ) δ + δ + + δ + δ δ + P B HM B+ P B HM Aδ δ + ( I P B HM B) St: G I P B HM B. Eas to prov: G G, P G GP, δ δ + G GG GP G GGP GP δ P P B ( M M AN A M ) BP δ Error stiat: δ : + δ, δ + δ δ G ( I G) P ( I G ) P( I G) S tr( S) tr( S ) E P E S σ P S σ P S { } tr( { } ) tr( ) tr( ) P G GP : G PGP, I G I G ( ) P S P I G P I G P I PGP P I G I G I G ( ) ( ) ( ) ( ) ( ) ( ) H I M AN A n N A M A n N N n I n tr tr n tr( ) tr( ) tr( ) tr I G P B HM B, I G P B HM B HM BP B H n tr( ) tr( ) tr( ) tr E P σ P S σ I G σ n { } tr( ) tr( ) ( ) σ P n is an uniasd stiat of σ ( E{ σ } σ ) Covarianc atrics: C Cδ σ N, C δ σ C P P B ( M M AN A M ) BP Estiatd covarianc atrics: C Cδ σ N, C ( ) δ σ C P P B M M AN A M BP Condition Equations: Covarianc propagation: + δ +

11 δ P B M ( B+ g ) δ + P B M [ B( δ + ) + g ] δ+ P B M ( Bδ+ g ) P B M B δ+ ( I P B M B) sinc Bδ + g Covarianc factor atri: I P B M B I P B M B I P B M B P I B M BP δ ( ) ( ) ( ) ( ) P P B M BP P B M BP + P B M ( BP B ) M BP P P B M BP ( ) P P B M BP B BB B Covarianc atri: C σ, stiat σ P, s E{ σ } σ Estiat of covarianc atri: C σ. Not: σ is an uniasd stiat of σ caus: δ P B M ( B+ g ) P B M ( Bδ + B+ g ) P B M ( Bδ g ) P B M B P B M B, sinc Bδ + g. + + P ( P B M B) P( P B M B) B M BP B M B B M B B M B B M B tr( ) tr( ) E{ σ } E{ P } E{tr( B M B )} tr( B M BE{ }) σ tr( B M BP ) s s s s tr( s σ M BP B ) σ tr( M M) σ tr I s σ σ. s s s s

12 SUMMARY: DEERMINISIC ESIMAION Osrvation quations odl: f ( ), + f ( ) + f P in : Nonlinar noral quations: ( ) P[ f ( )] Linarization: f ( ) + Aδ + f f ( ), A ( ) (, δ ) P in : Noral quations: A PAδ A P or N u δ ( ) A PA A P or N u, δ A. Mid quations odl (condition quations with unknowns): +, h(, ) P in : Non-linar solution sst: P( ) B (, ) k, A (, ) k, h(, ) A(, ) (, ), B(, ) (, ) Rlatd: Gauss-Hlrt odl: h(, ) h% (, ) Do not us! Linarization: h(, ) Aδ + Bδ + h h h(, ), A (, ) (, ), B, δ., δ P in : δ w B+ h h(, h ), N A M w M BP B, N A M A [ ] δ P B M I AN A M w, δ Condition quations odl: +, g( ) g P in : Non-linar solution sst: P( ) B ( ) k, g( ) whr: B( ) ( ) Linarization: g( ) Bδ + g g B ( ) g g( ), δ, P in : δ P B M w, δ ( w g + B g( ), M BP B )

13 SOCHASIC ESIMAION Stochastic odl: A+, E { }, C { }, E σ σ unknown BLUE stiats of L Linar: q a q d + γ U uniforl Uniasd: E{ q } q for vr a d A q a d A d q is stial ( ists q with E{ q } q) q is a function of th osrvals q d B Bst iniu an squar rror: E{( q q) } in q in sujct to a d A: q a, whr q ( ) A A A Gauss-Markov thor: For stial functions q a (i.. such that th ar functions q d of th osrvals) th BLUE stiats coincid with th last squars stiats if PλC for an λ>. Osrvation quations: Accuracis of stiats Covarianc factor atri: δ N ( A A ), Covarianc atri: C σ N σ P, ( E{ σ } σ. Estiatd covarianc atri: n C σ N Mid quations: Covarianc factor atrics: ( A M A) N δ P P B ( M M AN A M ) BP δ σ P n ( E{ σ } σ ) Covarianc atrics: C Cδ σ N, C δ σ C P P B ( M M AN A M ) BP Estiatd covarianc atrics: C Cδ σ N, C ( ) δ σ C P P B M M AN A M BP Condition Equations: Covarianc factor atri: ( ) P P B M BP B BB B σ P, ( E{ σ } σ ) s Covarianc atri: C σ, Estiat of covarianc atri: C σ.