Estimation of a andom Vaiabl Obsv and stimat. ˆ is an stimat of. ζ : outcom Estimation ul ˆ Sampl Spac Eampl: : Pson s Hight, : Wight. : Ailin Company s Stock Pic, : Cud Oil Pic. Cost of Estimation Eo Fo a givn pai of valus = and =, th stimat is ( ) and th stimation o is = ( ). Lt C b th cost function. W want to minimiz th avag cost C = C() f (, ) d =, C () f ( ) f () d. f ( ) is fd to as th a postioi pobability distibution.
Cost Functions Squad Cost Whn C = pobability distibution, th bst stimation ul is th conditional man of th a postioi [ ] ms = E = = f( ) Th stimat is fd to as th minimum man-squad o (MMSE) stimat. Absolut Cost Whn C =, th bst stimation ul is th mdian of th a postioi pobability distibution abs Unifom Cost f ( ) = f ( ). abs 0 if Δ Whn C =, if >Δ th bst stimation ul is th maimum a postioi pobability f ( ) = ma f ( ) unf Not Whn th a postioi pobability distibution is Gaussian, th th stimats nd up at a sam valu.
3 Minimum Man-Squad Eo (MMSE) Estimat Obsv = and stimat by. = is a function of ; w will us o Fo givn = and =, = C. intchangably. Th avag cost is C = C f (, ) d, { } = f (, ) d { }, = f ( ) f ( ) d { ˆ } = d f () f ( ) Sinc f ( ) 0, C is minimizd whn th inn intgal is minimizd Wit th inn intgal as W qui ( ˆ) [ ] f( ) E ˆ [ ] ˆ I = f ( ) f ( ) + f ( ) d I ˆ = 0, which is achvd whn = E = = f ( ). Th MMSE stimat, ˆ pobability distibution = = + ms, is th conditional man of th a postioi
4 Absolut Eo Cost = Fo givn =, = and, C.. Th avag cost is C = f (, ) d, = d f () f ( ) Sinc f ( ) 0, C is minimizd whn th inn intgal is minimizd. Wit th inn intgal as I = f( ) + f( ) Using th Libnitz's ul, d d d I( ) = ( ) f ( ˆ ) ˆ ˆ ˆ f ˆ f ˆ + + d d + ˆ ( ) f ( ) ( ) f ( ) f ( ) ˆ ˆ = f ( ) f ( ) ( a) Stting q. a to zo lads to f ( ) = f ( ). is th mdian of th a postioi pobability distibution. abs
5 Unifom Eo Cost Fo givn =, = and, C 0 if Δ = if >Δ.. Th avag cost is C = C() f (, ) d, d f () C() f ( ) +Δ = d f () f ( ) Δ Sinc f ( ) 0, C is minimizd whn th inn tm is minimizd. = It is minimizd whn w maimiz +Δ Δ f ( ). Fo a small valu of Δ, th intgal is maimizd whn f ( ) = ma f ( ). unf Th bst stimation ul is th maimum a postioi pobability.
6 Eampl Thow a dic. Suppos and a dtmind accoding to th following tabl. ζ - - 0 3 4 0 4 4 Thn th th stimats a: 0 4 ˆms 0 0 3 abs 0 -<= <= 0 4 0 - -, unf o, o 3
7 Eampl Man Squa Eo Estimat ˆ ms V ( ) ( V ) = + V. Assum ~ N 0,, V ~ N 0,, and and V a indpndnt. Find th minimum man squa o stimat of givn. Solution: ms = E[ = ] = f( ) f f, (, ) f f f fv = = = f () f () f () What is f ( )? Sinc = + V and and V a indpndnt Gaussian, f () = N 0, +. V Thfo f ( ) = = + V π π π π V + V V ( ) ( + V ) V ( ) + ( + ) V V ( )
8 Th ponnt is ( ) + V V ( + ) V ( + V ) V + V + + V V = = + V V + V ( 3) Fom q. and 3, f( ) = π V + V + V V + V ~, N V V + + V ( ) Fom, [ ] = E = = f ( ) ms = + V ( 4) Eq.4 says ms, whn >> 0, whn V << V
9 Estimating by a Constant Somtims it is difficult to find th conditional man. W sttl fo a sub-optimal solution. As th simplst cas, consid a fid stimato: ( ) = a fo som constant a, gadlss of th valu of. Still w want to minimiz th man-squad o. MSE is minimizd whn a =, and th sulting man-squad o is = VA. Poof. = a f (, ) d, = a f ( ) = a a + d a da = Stting th divativ to zo lads to a=. Whn a =, = f ( ) =
0 Estimating by a Lina Function. Lina MSE Estimat W want, fo givn, = a+ b fo som constants a and b. Still w want to minimiz th MSE. Thn = ρ, ( ) + and th sulting MSE is ( ρ, ) = Not. Th lina stimat is sam as th conditional man whn and a jointly Gaussian. In anoth wod, th lina stimat is indd th minimum MSE stimat whn and a jointly Gaussian. Poof. = d a+ b f (, ), = d a + b + + ab a b f (, ) = a + b + + ab a b, d d a b b a a = + b = + Stting th divativs to zo, w hav a = ρ, b= a
Homwok (, ) = Show th sulting MSE is ρ Poptis of th Lina MSE Estimat ) Unbiasd Estimat E ˆ = ) Othogonality Pincipl: E [ o obsvd ] = 0 E ˆ = 0 ; that is, 3) Th sulting MSE o is qual to [ ] = E o to b stimatd = E ˆ Homwok Pov th abov poptis. gssion Lin Th lina MSE stimato is fd to as th gssion lin.
Eampl Consid th following pimnt. Div th gssion lin that is th lina stimat of givn, and th sulting MSE. ζ - - 0 3 4 0 4 9 ˆ ρ = 0.565, = 0.39 0.5, =.9858, 4 3 lina data 0-0 4 6 8 0 optimal (conditional mna) - Lina (lina) -3 (souc: ms tabl) Homwok Mak an cl fil to find th lina MSE stinat.