Introducton Modelng Data Gven a et of obervaton, we wh to ft a mathematcal model Model deend on adutable arameter traght lne: m + c n Polnomal: a + a + a + L+ a n Choce of model deend uon roblem Aroach Degn a fgure-of-mert (or ftne) functon Meaure agreement between model and data mall value mean good ft Adut the model arameter to mnme the ftne functon Gve the bet-ft arameter Qualt of Ft If obervaton no, we can never ft eactl We wh to etmate he qualt of the ft of the model he uncertant n the etmate of the bet ft arameter Probabltc Aroach Concentrate on D outut for now Inut obervaton: Model form: { }.. { }.. f ( ; + : Model arameter : Redual noe term Dtrbuton () Lkelhood Probablt of obervaton gven the model arameter O ( f ( ; ) Prob. of all data ( f ( ; ) he lkelhood of the data gven the arameter
Model arameter etmaton We would lke the arameter whch are mot lkel gven the data data) data) data) unknown, but fed Mame Pror PDF of aram Eamle: Lne fttng Ft traght lne to data {, } m + c f ( ; { m, c}) Aume normal error on wth.d. ( ) ( ;, ) ( data m, ( ( m + ) Eamle: Lne fttng ( data m, log data m, ( ( m + ) log ( ( m + ) log data m, ( ( m + ) + cont { m, c} arg mn F( m, ( ( m + ) Lne Fttng { m, c} arg mn F( m, Let df dm df dc, ( ( m + ) ( ( ( m + ) ( m c etc m c ( ( m + ) m m Uncertant he arameter that mame data onl gve u a ngle value o nformaton about uncertant Often we need the full df data) data) data) Hard to comute! Dtrbuton of arameter ( data) Z Z data) a normalaton contant Z d Can be comuted eactl for ome cae Otherwe can be aromated b amlng where necear Z data ) are random amle from
Mean Mean of ), µ ) ˆ ( data) d d Z ˆ data ) Z Z data ) Eamle: Lne Fttng log data m, ( ( m + ) + cont m c If cont K ( ˆ) K( ˆ ) + cont K data) ( : ˆ, ) Etmatng Parameter PDF Error Proagaton ot alwa ea to comute dtrbuton eactl Common aromaton Error roagaton Monte-Carlo etmate Ueful when we have an analtc form for the comutng arameter from the data F({, }) F ({, }) For ntance, for lne fttng m c If vare b Error Proagaton F ({, }) then hu Gauan noe on caue noe on vare b aro. wth varance wth varance F F Monte-Carlo Method Ued when no analtc functon avalable Algorthm: For k..m Add noe to each orgnal meaurement Etmate otmal arameter, k Parameter PDF aromated b PDF of { k } otal varance of F 3
.5.45.4.35.3.5..5..5 3 4 5 Qualt of Ft How well doe the model actuall ft the data? Qualt of Ft How well doe the model actuall ft the data? Probabl not a ver good ft Deend on uncertant on meaurement Probabl not a ver good ft Deend on uncertant on meaurement Qualt of Ft Model form: f ( ; + ( f ( ; ) Aume normal error on wth.d. ( f ( ; ) log + cont If each Ch-quared Dtrbuton drawn from a zero mean, unt normal PDF, then the PDF of M known a χ ( M : ) M ha a mean of and a varance of Ch quare PDF chdf(m,) n MatLab 3 4 5 6 7 8 9 Ch-quared Dtrbuton Ch-quare and the Gamma Functon For large, χ (M:) (M:, ) A Gauan wth mean and varance.45.4.35.3.5..5..5 5 Ch-quared ormal 5 Gamma Functon Incomlete Gamma Functon P( a, ) Γ( a) CDF of Ch-quare: Γ( z) a t t z t t t e dt t e dt M P ( χ < M ) P(, ) χ n! Γ( + n) 4 6 8 4 6 8 4
Qualt of Ft ( f ( ; ) log + cont Margnalaton ) ( M ( f ( ; ) χ wth d.o.f., ) Let If P P M M < If then P P(χ < M ) (CDF of χ at M ).99 then model a M > reaonable ft.9999 the model a oor ft chcdf(m,) n MatLab, ) ( ) ), ) d ( ) ) ) d ( ), ) d Eamle: Lne fttng If we are onl ntereted n the gradent, m m, c data) ( m, c : ( mˆ, cˆ), ) m data) ( m, c : ( mˆ, cˆ), ) dc m data) ( m; mˆ, ) m m General Lnear Model Conder model of form M f ( ) w f ( ) + ( :, ) Lne fttng : M, f (),f () Polnomal degree d fttng : M d +, f () General Lnear Model Data: {, } Contruct M Degn Matr D { D } D f ( ) / Otmal weght w gven b oluton to Dw r r / General Lnear Model Perform VD on D D UWV wˆ VW Covarance of r I U w VW U IUW w VW V r V 5
Model electon uoe we have everal oble model Eg lnear, quadratc, cubc etc How do we elect the bet? P(data aram) alone nuffcent More comle model gve lower redual Akake Informaton Crtera mle method of model electon For each model fnd otmal arameter,, and comute AIC log + d d the number of ndeendent arameter elect model gvng the mallet AIC Baean Model electon uoe M ndee varou oble model We eek the mot robable model to ft to ome data We thu wh to fnd M to mame P( data M ) P( M ) P ( M data) P( data) If all model equall lkel, mame P( data M ) contant Baean Model electon uoe model M ha arameter, P ( data M ) data, M ) M ) d (he `Evdence for model M) Unfortunatel, th often moble to comute analtcall Varou aromaton rooed ee the lterature 6