It's Only Fitting. Fitting model to data parameterizing model estimating unknown parameters in the model

Size: px

Start display at page:

Download "It's Only Fitting. Fitting model to data parameterizing model estimating unknown parameters in the model"

Barry Fox
6 years ago
Views:

1 It's Only Fitting Fitting model to data parameterizing model estimating unknown parameters in the model Likelihood: an example Cohort of 8! individuals observe survivors at times >œ 1, 2, 3,..., : 8", 8,..., 8 Suppose we want to fit the demographic survival model. The probability of this particular outcome under the model is the joint probability mass function: Pr ÒR" œ 8", R œ 8,..., R œ 8 R! œ 8! Ó œ :8 ˆ 8 :8 ˆ 8 â:8 ˆ 8 "! "."

2 œ 8>." Š : Ð".:Ñ 8 > > >." > For instance, suppose the numbers of survivors, starting at >œ0, are: 100, 93, 80, 69, 61, 56, 52, 45, 43, 40, 36. Then PrÒR" œ 93, R œ 80,..., R œ 36 R œ 100Ó œ! ˆ 100 ˆ : Ð".:Ñ 80 : Ð".:Ñ... ˆ 40 : Ð".:Ñ œp: a b R. A. Fisher (1922): use as as estimate of : the value that mazimizes P: a b.

3 P: a b: likelihood function (joint pmf or pdf, evaluated at the data) s: : maximum likelihood (ML) estimate of : Likelihoods are usually products, so in maximizing it is usually easier to work with log-likelihoods: ln PÐ:Ñ œ! 8>." ln Š 3! 8 ln : 8 > > 3! Ð8.8 Ñ ln : >." > Set. ln P a: bî.: œ 0 and solve for :: s:œ! 8! 8 > >." For the example data, s:œ0.8998

4 Hypothesis tests and confidence intervals Suppose of : :! is the true (but unknown) value log-likelihood ratio quantity: K œ.òln PÐ: Ñ. ln PÐ:ÑÓ s Math-stat theorem (Wilks 1938): K converges in distribution to a chi-square distribution with 1 df (same type of convergence as CLT)!

5 Result allows for a statistical hypothesis test: H:! :œ:! :Á: H: "! Calculate K, reject H! in favor of H " if K exceeds the "!!Ð".! Ñth percentile,! Ð"Ñ, of a chi-square (1 df) distribution, for an approximate size! test. Result also allows an approximate 100(1.! Ñ % confidence interval for : (set of all values of :! for which H! would not be rejected): Ö: À PÐ:Ñ s! ln. ln PÐ:! Ñ Ÿ! Ð"ÑÎ The 95th percentile of a chi-square(1) distribution is about!þ!& Ð"Ñ œ Finding the endpoinds of the confidence interval usually requires numerical rootfinding. For the data, the 95% CI for : is (0.8750, )

6 Likelihoods for other models NLAR model: \ œ 2Ð\ Ñ 3 I > >." > Recall that the one-step conditional pdf is :B ˆ B œ > >.". " a B.2B a 5 bb > >." a5 1b exp. Suppose B!, B",..., B are the (transformed) observations of population abundances. ): vector of unknown parameters in 2aB : variance of noise, I 5 > Likelihood function: PÐ)5, Ñ œ :ÐB l B Ñ > >." >." b

7 Log-likelihood: ln PÐ, Ñ œ " ln :ÐB l B Ñ )5 > >." œ. ÐÎÑ ln Ð 1Ñ. ÐÎÑ ln Ð5 Ñ. c"îð Ñd! cb. 2ÐB Ñd 5 > >." ML estimates s), 5 s are the values of ) and 5 that maximize P or ln P. ex. environmental stochastic Ricker model (\ œ ln R ): > > \ \ œ \ 3 +.,/ >." > >." 3 I> Data: 8!, 8",..., 8 Transformed data: B0 œ ln 80, B1 œ ln 81,..., B œ ln 8 ) œ c+,, d w

8 ln PÐ+,,, Ñ œ. ÐÎÑ ln Ð 1Ñ. ÐÎÑ ln Ð5 Ñ 5. c"îð 5 Ñd! ab. B. + 3,/ B >." b > >." Multivariate NLAR model (such as LPA): \ œ 2Ð\ Ñ 3 I > >." > :ÐB> l B>." Ñ " 5. œ kdk. a 1b w." exp c.ðb.2 ÑD ÐB.2 ÑÎ d > >." > >." 5 : number of state variables in \ > 2 œ 2aB b >." >." Lilelihood is a product of multivariate normal pdf's log-likelihood is: ln PÐ)D, Ñ œ. Ð5ÎÑ ln Ð 1Ñ. ÐÎÑ ln kdk. Ð"ÎÑ! w." ÐB. 2 Ñ D ÐB. 2 Ñ > >." > >."

9 Properties of ML estimates: asymptotically unbiased E Š s) p ) as p_ consistent PrŠ ¹ s ). ) ¹ K % p 1 as p_ asymptotically efficient VŠ )s p theoretical minimum as p_ asymptotically normal: distribution of s ) converges to a normal distribution as p_

10 Conditional least squares estimation Conditional expected value: E ˆ \ \ œ B œ 2aB b > >." >." >." ) Conditional sum of squares: UÐ) Ñ œ! cb. 2ÐB ) Ñd > >." Conditional least squares (CLS) estimates of the parameters in ) jointly minimize UÐ) Ñ

11 Properties of CLS estimates: asymptotically unbiased consistent asymptotically robust to mis-specification of distribution of I > (not effficient)

12 Estimating D (ML or CLS) Fitted model yields series of residuals (or residual vectors): / œ B.2ÐB s > > >." Ñ s >." for >œ 1, 2,...,, where 2ÐB Ñis the skeleton evaluated at the ML or CLS estimate of ) Let Hœ c/, /,..., / d ( 5 ) then " D s w œhhî

13 Confidence intervals, hypothesis tests Profile likelihood (ML estimates) = ) œ / = is a parameter vector of interest / vector of nuisance parameters is a Pa=/D,, b: likelihood function PŠ =/, s, Ds : likelihood function = = maximized for fixed = ( profile likelihood function of = ) PŠ =/D sss,, : maximized likelihood function (unrestricted)

14 Approx 100(1.! Ñ% confidence region for = is the set of all values of = for which K œ. ln P Š =/, s, D s. ln PŠ =/D s, s, s where = = Ÿ! a( b ( is the number of parameters in = Bootstrapping (ML or CLS estimates) /", /,..., /: residuals (or residual vectors) /", /,..., /: random sample with replacement from the centered residuals (i.e. subtract the average from each of /", /,..., / to center them around zero)

15 Generate a bootstrapped data set B, B, B,..., B (simulated data set from estimated model) using B œ 2ÐB s Ñ 3 / > >." >! " Fit the model to B!, B", B,..., B, obtaining bootstrapped parameter estimates ), D Repeat 2000 or so times (using residuals & estimates from original fitted model), obtaining 2000 sets of bootstrapped parameter estimates (or test statistics, or functions of parameter estimates such as eigenvalues or Lyaponov exponents)

16 Resulting 2000 values constitute a good (statistically consistent) estimate of the variability of the estimates e.g., 2.5th and 97.5th percentiles of the bootstrapped estimates of a parameter are a valid approximate 95% confidence interval for the parameter

17 Model evaluation Goodness of fit If the model fits well, then the residuals / ", /,..., / should behave approximately like independent, identically distributed normal (or multivariate normal) variates Evaluating model fit revolves around the residuals e.g. autocorrelation tests, residual plots to look for unchanging variance, normal probability plots, etc. R-square : proportion of variability in data accounted for by fitted model Validation (predictive ability of the model) Ideally, collect new data, independently from the data used to fit the model (possibly with experimental, model-suggested perturbations!)

Chances are good Modeling stochastic forces in ecology

Chances are good Modeling stochastic forces in ecology Ecologists have a love-hate relationship with mathematical models The current popularity in population ecology of models so abstract that it is difficult