Treatment and analysis of data Applied statistics Lecture 4: Estimation

Transcription

1 Treatment and analysis of data Applied statistics Lecture 4: Estimation Topics covered: Hierarchy of estimation methods Modelling of data The likelihood function The Maximum Likelihood Estimate (MLE) Confidence interval from the likelihood Confidence regions Least squares estimation Properties of the likelihood function (More on MLE in next lecture) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 1 Estimation distribution, parameters estimation population probability theory statistics, inference sample data Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 2

2 Hierarchy of standard estimation methods (incomplete and not very strict!) no prior normality homoscedasticity linearity single parameter Bayesian estimation Maximum likelihood estimation Generalized least-squares method Ordinary least-squares method Linear regression Sample mean Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 3 Probabilistic modelling of data Except for simple cases of descriptive statistics, it is necessary to model the data in a probabilistic sense in order to do estimation. The observation process is modelled as a random experiment, whose outcome is the observed data. The data are a realization of a (multivariate) random variable X, whose pdf depends in some known way on the parameters we want to estimate. A typical model may be something like this: physical model of the phenomenon, depending on the parameters θ 1 model of the observation process, depending on θ 2 random realization data x observable quantities pdf of the random variable X, f ( x θ 1, θ 2 ) θ 1 are the parameters we are interested in, θ 2 are nuisance parameters. Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 4

3 Example: Stellar parameters from photometry (highly simplified) We want to estimate physical parameters of a group of stars, such as their effective tempertures (T eff ), gravities (g) and metallicities (Z), from photometric measurements of the stellar light in selected spectral passbands. The stellar parameters T eff, g, Z (and possibly other relevant quantities) constitute the unknown parameter vector θ 1. Stellar atmosphere theory permits to compute what the stellar spectrum should be for given stellar parameters, S ( λ θ 1 ). The photometric measurement in a certain passband k essentially estimates the stellar flux received at Earth between the wavelength limits of the passband, I k λ max 1, 2 k θ1 λ min k ( θ θ ) = c S( λ ) dλ where [λmin k, λmax k ] is the wavelength coverage of passband k and c k its sensitivity. The nuisance parameters θ 2 consist of (all or part of) λmin k, λmax k, c k for all the passbands. The observational data may be modelled as photon counts ~ Pois[I k (θ 1, θ 2 )]. Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 5 k The likelihood function Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 6

4 The maximum likelihood estimate (MLE) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 7 Example: The mean of a normal distribution from one data point (1) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 8

5 Example: The mean of a normal distribution from one data point (2) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 9 A less trivial example (1) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 10

6 A less trivial example (2) stated confidence interval 0.54±0.11 Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 11 Example: MLE for a Poisson variable (1) no good! OK Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 12

7 Example: MLE for a Poisson variable (2) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 13 Example: MLE for a Poisson variable (3) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 14

8 Example: The mean and s.d. from several data points (1) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 15 Example: The mean and s.d. from several data points (2) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 16

9 Example: The mean and s.d. from several data points (3) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 17 Confidence region for multidimensional parameter From Numerical Recipes, Ch. 15 Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 18

10 Confidence region and confidence intervals (Figure from Numerical Recipes) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 19 Linear least-squares estimation (1) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 20

11 Linear least-squares estimation (2) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 21 Linear least-squares estimation (3) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 22

12 Linear least-squares estimation (4) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 23 Linear least-squares estimation (5) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 24

13 Uncertainties of the LSE (1) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 25 Uncertainties of the LSE (2) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 26

14 Uncertainties of the LSE (3) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 27 Uncertainties of the LSE (4) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 28

15 Confidence region and confidence intervals The interval AA' along the y axis (the projection of the ellipse for Δχ 2 = 1.00) is a 68% confidence interval, but the 68% confidence region is given by the ellipse for Δχ 2 = Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 29 Properties of MLE (summary) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 4, p. 30