Introduction to statistics. Laurent Eyer Maria Suveges, Marie Heim-Vögtlin SNSF grant

Transcription

1 Introduction to statistics Laurent Eyer Maria Suveges, Marie Heim-Vögtlin SNSF grant

2 Recent history at the Observatory Request of something on statistics from PhD students, because of an impression of lack of knowledge - Daniel Schaerer - Amaury Triaud, Richard Anderson - Maria Suveges, Damien Segransan, Stéphane Paltani, Laurent Eyer Cafés statistiques in :

3 Plan ~8 sessions Wednesday 15h - 17h 3 first lectures: 1) Statistical tests EYER, SUVEGES 2) Chi2 statistics, maximum likelyhood SEGRANSAN 3) Monte Carlo, Markov chain PALTANI 4) Robust, non-parametric statistics PALTANI, +SEGRANSAN, +EYER 5) Non-Gaussian statistics SUVEGES 6) Time series analysis EYER 7) Bayesian statistics EYER?, SEGRANSAN? 8) Biases?? - General Introduction, definitions, hypothesis testing, today - Chi2 statistics, maximum likelihood, wednesday 6 October Monte Carlo Markov chain, Wednesday 10 November 2010

4 Some statistical resources for this introduction Probabilités, analyse des données et statistique Gilbert Saporta Introduction à la statistique, Stephan Morgenthaler Advanced theory of statistics Maurice Kendall & Allan Stuart Statistics in theory and practice, Robert Lupton Introductory statistics with R, Peter Dalgaard wikipedia (it seems quite good, but always to take with care)...

5 Random Variables Discrete: - Spectral type (G2V, KIII) - Galaxy type, galaxy zoo Continuous: - magnitude, flux, colour, radial velocity, parallax/distance, temperature, elemental abundances, magnetic field, age, etc...

6 Distribution Definition: density is a function b Pr(a <X<b)= f(x) Distribution of one variable: univariate a such that: f(x)dx f(x) Pr(X = k) =p(k) Distribution of several variables: multivariate f(x, y,...) Marginalization: If and only if: u(x) = f(x, y)dy f(x, y) =u(x)v(y) the variables are independent Cumulative Distribution Function CDF: x F (x) = f(x )dx Digression on LateXiT

7 CDF Density Example: Gaussian / Normal distribution X N (µ, σ 2 ) f(x) = 1 exp (x µ)2 2πσ 2 2σ 2 Location: L2 from wikipedia

8 Poisson distribution Discrete probability distribution (no density) X Poisson(λ) Number of photons on a detector Number of people in a shop Pr(X = k) =exp( λ) λk k! For large λ N (λ, λ) from wikipedia

9 Moments of a distribution Information of location, the mean E(X) =µ = xf(x)dx Normal: μ Poisson: λ Information of dispersion, the variance Var(X) = Moment of order n about the mean: µ n = (x µ) 2 f(x)dx (x µ) n f(x)dx standard deviation σ = Var(X) Normal: σ2 Poisson: λ

10 3d and 4th moments of a distribution Skewness, asymmetry µ 3 /σ 3 = (x µ) 3 f(x)dx/σ 3 Normal: 0 Poisson: 1/ λ density Kurtosis from wikipedia µ 4 /σ 4 = µ 4 /σ 4 3 (x µ) 4 f(x)dx/σ 4 Normal: 0 Poisson: 1/λ

11 Example of different values of kurtosis: boxiness -- tail heaviness density x from wikipedia

12 Covariance and correlation Covariance Cov(X, Y )= (x µ x )(y µ y )f(x, y)dxdy Correlation Corr(X, Y )= Cov(X, Y ) σ X σ Y

13 Examples of correlation from wikipedia

14 Quantiles x p : p-quantiles of f(x) p = xp f(x)dx f(x) p Measure of location: Median 1/2 = x1/2 f(x)dx xp from Measure of dispersion: Inter-quantile range IQR = x 3/4 x 1/4

15 Data, samples Usually we have observations, e.g. additive process y i = f(t i )+ i i =1,...,n Deterministic random variable We want a characterisation of the deterministic and random parts Suppose something about the random variable, often normality: N (0, σ 2 )

16 Estimators Assumption of models Estimate the parameters of a distribution, moments - Exercise 1: Sample mean: - Exercise 2: Sample variance (bias): redefine ˆσ 2 = 1 n ˆµ = X = 1 n n (X i X) 2 i=1 ˆσ 2 = S 2 = 1 n 1 i=1 Sample quantiles are estimators of quantiles - Exercise 3: what is the sample median of {1, 2, 3, }? n i=1 X i n (X i X) 2 E( X) =µ E( ˆσ 2 )= n n 1 σ2

17 Central limit theorem The distribution of the mean of a sufficiently large number of random variables can be approximated by a Gaussian distribution! X i,i=1,...,n iid with E(X i )=µ Var(X i )=σ 2 iid= Independent identically distributed X µ σ/ n follows approximately N (0, 1) One reason why the Gaussian distribution is so important

18 Distribution derived from Normal distribution 1) Chi square distribution If X i iid N (0, 1) iid= Independent identically distributed mean: k variance: 2 k skewness: (8/k) kurtosis: 12/k f(x) = k Xi 2 χ 2 k i=1 1 2 k/2 Γ(k/2) xk/2 1 exp ( x/2) iid X i N (µ, σ) k (X i X) 2 /σ 2 χ 2 k 1 i=1 When k is large approximates a χ 2 k N (k, 2k) from wikipedia

19 Distribution derived from Normal distribution 2) Student distribution X µ σ/ n N (0, 1) X µ S/ n t n 1 f(x) = Note n+1 Γ( 2 ) 1+ x2 nπγ(n/2) n t = N (0, 1) (n+1)/2 mean: 0 n>1 NaN n=0,1 variance: n/(n-2) n>2 1<n 2 otherwise NaN skewness: 0 n>3 kurtosis: 6/(n-4) n>4

20 Estimators of Variance of different statistics from Kendall & Stuart

21 Graphical representation QQ Plots Comment on figures: label and numbers large enough, quantity and units X 1,...,X n X (1),...,X (n) X i iid F (x) F 1 ( 1 n +1 ) F 1 ( n n +1 ) Theoretical Normal Theoretical Quantiles Sample Quantiles t(n = 3) Theoretical Quantiles Sample Quantiles

22 End of the Introduction