Probability and Statistics Basic concepts II

Transcription

1 Probablty ad Statstcs Basc cocepts II (from a physcst pot of vew) Beot CLEMENT Uversté J. Fourer / LPSC bclemet@lpsc.p3.fr

2 Statstcs PHYSICS parameters θ SAMPLE Fte sze x Observable INFERENCE POPULATION f(x;θ) EXPERIMENT Parametrc estmato : to gve oe value to each parameter Iterval estmato : to derve a terval that probably cotas the true value No-parametrc estmato : to estmate the full pdf of the populato

3 Parametrc estmato From a fte sample {x } -> estmatg a parameter θ Statstc a fucto S f({x }) Ay statstc ca be cosdered as a estmator of θ To be a good estmator t eeds to satsfy : Cosstecy : lmt of the estmator for a fte sample. Bas : dfferece betwee the estmator ad the true value Effcecy : speed of covergece Robustess : sestvty to statstcal fluctuatos A good estmator should at least be cosstet ad asymptotcally ubased 3 Effcet / Ubased / Robust ofte cotradct each other dfferet choces for dfferet applcatos

4 Bas ad cosstecy As the sample s a set of realzato of radom varables (or oe vector varable), so s the estmator : θˆ s a realzato of Θˆ t has a mea, a varace, ad a probablty desty fucto Bas : Mea value of the estmator b( θ) ˆ E[Θ- ˆ θ0 ] μ - θ Θˆ 0 ubased estmator : b( θ) ˆ 0 asymptotcally ubased : b( θ) ˆ 0 + Cosstecy: formally P( θ-θ ˆ > ε) 0, ε + practce, f asymptotcally ubased σ 0 based asymptotcally ubased Θˆ + ubased 4

5 sˆ Emprcal estmator Sample mea s a good estmator of the populato mea -> weak law of large umbers : coverget, ubased Sample varace as a estmator of the populato varace : E[sˆ 1 μˆ x, μμ E[μ] ˆ μ, σμ E[(μ-μ) ˆ ˆ ˆ ] 1 ] (x 1 μ) ˆ σ σ 1 μˆ (x σ μ) σ ubased varace estmator : ( μ μˆ ) 1 σ 1 σˆ -1 σ based, asymptotcally ubased (x μ) ˆ 5 varace of the estmator (covergece) σ ˆ σ 4 σ 1 γ -1 + σ 4

6 Errors o these estmator Ucertaty Estmator stadard devato Use a estmator of stadard devato : (!!! Based ) Mea : σˆ 1 σ μˆ x, σμ Δμˆ ˆ σˆ σˆ Varace : 1 σ 4 σˆ (x μ) ˆ, σ Δσˆ σˆ σˆ -1 Cetral-Lmt theorem -> emprcal estmators of mea ad varace are ormally dstrbuted, for large eough samples 6 ˆ ± Δμˆ ; σˆ ± μ Δσˆ defe 68% cofdece tervals

7 Lkelhood fucto Geerc fucto k(x,θ) x : radom varable(s) θ : parameter(s) fx θ θ 0 (true value) fx x u (oe realzato of the radom varable) Probablty desty fucto f(x;θ) k(x,θ 0 ) f(x;θ) dx1 for Bayesa f(x θ) f(x;θ) Lkelhood fucto L (θ) k(u,θ) L (θ) dθ??? for Bayesa f(θ x) L (θ)/ L (θ)dθ 7 For a sample : depedet realzatos of the same varable X L(θ ) k(x,θ) f(x ; θ)

8 0 0 Estmator varace Start from the geerc k fucto, dfferetate twce, wth respect to θ, the pdf ormalzato codto: 1 k(x, θ)dx k dx θ k dx θ lk k θ lk k θ lk dx E θ dx + lk k θ lk (b + θ)e θ Now dfferetatg the estmator bas : lk dx E θ 0 lk E θ θ + b θ(x)k(x, ˆ θ)dx 1 b k lk + θ(x)k(x,θ)dx ˆ θ dx θk dx (θ-b-θ)k θ θ ˆ ˆ θ θ ˆ lk dx θ 8 Fally, usg Cauchy-Schwartz equalty ( 1+ b' ) b lk + (θ-b-θ) ˆ kdx k dx σ Θ θ θ lk E Cramer-Rao boud θ 1 ˆ

9 Effcecy For ay ubased estmator of θ, the varace caot exceed : 1 σ Θˆ ll E θ The effcecy of a coverget estmator, s gve by ts varace. A effcet estmator reaches the Cramer-Rao boud (at least asymptotcally) : Mmal varace estmator MVE wll ofte be based, asymptotcally ubased 9

10 Maxmum lkelhood 10 For a sample of measuremets, {x } The aalytcal form of the desty s kow It depeds o several ukow parameters θ eg. evet coutg : Follow a Posso dstrbuto, wth a parameter that depeds o the physcs : λ (θ) λ (θ ) x e λ (θ ) L(θ ) x! A estmator of the parameters of θ, are the oes that maxmze of observg the observed result. -> Maxmum of the lkelhood fucto L θ θ θ ˆ rem : system of equatos for several parameters rem : ofte mmze -ll : smplfy expressos 0

11 Propertes of MLE Mostly asymptotc propertes : vald for large sample, ofte assumed ay case for lack of better formato Asymptotcally ubased Asymptotcally effcet (reaches the CR boud) Asymptotcally ormally dstrbuted -> Multormal law, wth covarace gve by geeralzato of CR Boud : ˆ 1 f(θ;θ,σ) π Σ e ˆ (θ-θ) 1 Τ 1 Σ ˆ (θ-θ) ll E θ ll θj Goodess of ft The value of -ll( θˆ ) s Kh- dstrbuted, wth df sample sze umber of parameters Σ 1 j 11 p value + f (x;df)dx -ll ( θˆ ) χ Probablty of gettg a worse agreemet

12 Errors o MLE 1 ˆ Τ (θ-θ) Σ 1 ˆ ˆ 1 (θ-θ) f(θ;θ,σ) e 1 ll Σj E π Σ θ Errors o parameter -> from the covarace matrx ll θj 1 For oe parameter, 68% terval More geerally : ΔlL Cofdece cotour are defed by the equato : Values of β for dfferet umber of parameters θ ad cofdece levels α Δθ σˆ θˆ 1 ll θ 1 1 ll( ˆ θ ) ll( θ ) Σ (θ -θ ˆ )(θj -θˆ ΔlL,j β( θ α j j +,α) wth α oly oe realzato of the estmator -> emprcal mea of 1 value β ) f θ 0 χ O(θ (x; 1 3 (0. 5* θ ) 3 θ ) )dx

13 Least squares 13 Set of measuremets (x, y ) wth ucertates o y Theoretcal law : y f(x,θ) Naïve approach : use regresso w w( θ ) (y f(x,θ)), 0 θ Reweght each term by the error y f(x,θ) K K ( θ ), 0 y θ Maxmum lkelhood : assume each y s ormally dstrbuted wth a mea equal to f(x,θ) ad a varace equal to Δy 1 y f(x,θ) 1 Δ The the lkelhood s : y L(θ ) e πδy L l L K 0 0 Least squares or Kh- ft s θ θ θ the MLE, for Gaussa errors 1 Τ 1 Geerc case wth correlatos: K (θ) (y-f(x,θ) Σ (y-f(x,θ)

14 Example : fttg a le For f(x)ax+b 14 Δy 1 E, Δy y D, Δy x C, Δy x B, Δy x y A E 1.5 Δb, B 1.5 Δa C BE AC DB b, C BE DC AE a ˆ ˆ ˆ ˆ

15 Example : fttg a le dmesoal error cotours o a ad b 15

16 Cofdece terval For a radom varable, a cofdece terval wth cofdece level α, s ay terval [a,b] such as : b Probablty of fdg a P(X [a,b]) f (x)dx α a X realzato sde the terval Geeralzato of the cocept of ucertaty: terval that cotas the true value wth a gve probablty -> slghtly dfferet cocepts For Bayesas : the posteror desty s the probablty desty of the true value. It ca be used to derve terval : P(θ [a,b]) α 16 No such thg for a Frequetst : The terval tself becomes the radom varable [a,b] s a realzato of [A,B] P(A < θ ad B > θ) α Idepedetly of θ

17 Cofdece terval Mea cetered, symetrc terval μ + μ a [μ-a, μ+a] a f(x)dx α Mea cetered, probablty symetrc terval : [a, b], μ b α f(x)dx f(x)dx a μ 17 Hghest Probablty Desty (HDP) b [a, b] f(x)dx α a f(x) > f(y) for x [a,b] ad y [a,b]

18 Cofdece Belt To buld a frequetst terval for a estmator θˆ of θ 1. Make pseudo-expermets for several values of θ ad compute he estmator θˆ for each (MC samplg of the estmator pdf). For each θ, determe A(θ) ad B(θ) such as : θ ˆ < Ξ(θ) for a fracto (1-α)/ of the pseudo-expermets θ ˆ > Ω(θ) for a fracto (1-α)/ of the pseudo-expermets These curves are the cofdece belt, for a CL α Iverse these fuctos. The terval [Ω (θ), ˆ Ξ (θ)] ˆ satsfy: -1 P Ω (θ) ˆ < θ < Ξ (θ) ˆ 1 P Ξ (θ) ˆ < θ -P Ω (θ) ˆ > θ ( ) ( ) ( ) ( ) ( ) 1 P θˆ < Ξ(θ) -P θˆ > Ω(θ) α 18 Cofdece Belt for Posso parameter λ estmated wth the emprcal mea of 3 realzatos (68%CL)

19 Dealg wth systematcs The varace of the estmator oly measure the statstcal ucertaty. Ofte, we wll have to deal wth some parameters whose value s kow wth lmt precso. Systematc ucertates The lkelhood fucto becomes : L( θ, ν ) ν ν 0 ± Δν or ν + Δν 0-ΔΔ + The kow parameters ν are usace parameters 19

20 Bayesa ferece I Bayesa statstcs, usace parameters are dealt wth by assgg them a pror π(ν). Usually a multormal law s used wth mea ν 0 ad covarace matrx estmated from Δν 0 (+correlato, f eeded) f(θ,ν x) f(x θ,ν)π(θ)π(ν) f(x θ,ν)π(θ)π(ν)dθ The fal pror s obtaed by margalzato over the usace parameters dν 0 f(θ x) f(θ,ν x)dν f(x θ,ν)π(θ)π(ν)dν f(x θ,ν)π(θ)π(ν)dθ dν

21 Profle Lkelhood No true frequetst way to add systematc effects. Popular method of the day : proflg Deal wth usace parameters as realzato f radom varables : exted the lkelhood : L( θ, ν ) L' ( θ, ν ) G(ν ) G(v) s the lkelhood of the ew parameters (detcal to pror) For each value of θ, mmze the lkelhood wth respect to usace : profle lkelhood PL(θ). PL(θ) has the same statstcal asymptotcal propertes tha the regular lkelhood 1

22 No parametrc estmato Drectly estmatg the probablty desty fucto Lkelhood rato dscrmat Separatg power of varables Data/MC agreemet Frequecy Table : For a sample {x }, Defe successve tervals (bs) C k [a k,a k+1 [.Cout the umber of evets k C k Hstogram : Graphcal represetato of the frequecy table h(x) f x C k k

23 Hstogram 3 N/Z for stable heavy ucle 1.31, 1.357, 1.39, 1.410, 1.48, 1.446, 1.464, 1.41, 1.438, 1.344, 1.379, 1.413, 1.448, 1.389, 1.366, 1.383, 1.400, 1.416, 1.433, 1.466, 1.500, 1.3, 1.370, 1.387, 1.403, 1.419, 1.451, 1.483, 1.396, 1.48, 1.375, 1.406, 1.41, 1.437, 1.453, 1.468, 1.500, 1.446, 1.363, 1.393, 1.44, 1.439, 1.454, 1.469, 1.484, 1.46, 1.38, 1.411, 1.441, 1.455, 1.470, 1.500, 1.449, 1.400, 1.48, 1.44, 1.457, 1.471, 1.485, 1.514, 1.464, 1.478, 1.416, 1.444, 1.458, 1.47, 1.486, 1.500, 1.465, 1.479, 1.43, 1.459, 1.47, 1.486, 1.513, 1.466, 1.493, 1.41, 1.447, 1.460, 1.473, 1.486, 1.500, 1.56, 1.480, 1.506, 1.435, 1.461, 1.487, 1.500, 1.51, 1.538, 1.493, 1.450, 1.475, 1.500, 1.51, 1.55, 1.550, 1.506, 1.530, 1.487, 1.51, 1.54, 1.536, 1.518, 1.577, 1.554, 1.586, 1.586

24 Hstogram Statstcal descrpto : k are multomal radom varables. wth parameters : μ p k pk P(x Ck ) σ k p (1 p ) μ C k Cov( f X (x)dx, ) p k k k k k p << 1 k k r k r p << 1 For a large sample : lm k k + So fally : μ p k p k f(x) k C k f lm + δ 0 For small classes (wdth δ): X (x)dx h(x) The hstogram s a estmator of the probablty desty 1 δ δf(x c p pk ) lm δ 0 δ k 0 f(x) 4 Each b ca be descrbed by a Posso desty. The 1σ error o k s the : Δ k σˆ μˆ k k k

25 Kerel desty estmators Hstogram s a step fucto -> sometme eed smoother estmator O possble soluto : Kerel Desty Estmator Attrbute to each pot of the sample a kerel fucto k(u) x x u, k(u) k( u), k(u)du 1 w Tragle kerel : Parabolc kerel : Gaussa kerel : k(u) 1- u, for -1< u < 1 3 k(u) (1-u ), for -1< u < 1 k(u) w kerel wdth, smlar to b wdth of the hstogram The a pdf estmator s : K(x) 1 k(u ) 4 1 π 1 e u - k x x w 5 Rem : for multdmesoal pdf : u k x (k) x (k) w (k)

26 Kerel desty estmators If the estmated desty s ormal, the optmal wdth s : 3 w σ (d + ) 1 d+ 4 wth that sample sze ad d the dmeso As for the hstogram bg, o geerc result : try ad see 6

27 Statstcal tests am at: Statstcal Tests Checkg the compatblty of a dataset {x } wth a gve dstrbuto Checkg the compatblty of two datasets {x }, {y } : are they ssued from the same dstrbuto. Comparg dfferet hypothess : backgroud vs sgal+backgroud 7 I every case : buld a statstc that quatfy the agreemet wth the hypothess covert t to a probablty of compatblty/compatblty : p-value

28 Pearso test Test for bed data : use the Posso lmt of the hstogram Sort the sample to k bs C : Compute the probablty of ths class : p C f(x)dx The test statstcs compare, for each b the devato of the observato from the expected mea to the theoretcal stadard devato. χ Data bs ( p ) The χ follow (asymptotcally) a Kh- law wth k-1 degrees of freedom (1 costrat ) p Posso mea Posso varace 8 P-value : probablty of dog worse, p value + f (x;k-1)dx For a good agreemet χ /(k-1) ~ 1, More precsely χ (k 1) ± (k 1) (1σ terval ~ 68%CL) χ χ

29 Kolmogorov-Smrov test Test for ubed data : compare the sample comulatve desty fucto to the tested oe Sample Pdf (ordered sample) 0 x < x0 1 k f s(x) δ(x-) F s(x) xk x < xk+ 1 x > x The the Kolmogorov statstc s the largest devato : D sup F x (x) F(x) The test dstrbuto has bee computed by Kolmogorov: S 1 P(D > β ) ( 1) r 9 [0;β] defe a cofdece terval for D β0.9584/ for 68.3% CL β1.3754/ for 95.4% CL r 1 e r z

30 Example Test compatblty wth a expoetal law : f(x) , 0.036, 0.11, 0.115, 0.133, 0.178, 0.189, 0.38, 0.74, 0.33, 0.364, 0.386, 0.406, 0.409, 0.418, 0.41, 0.43, 0.455, 0.459, 0.496, 0.519, 0.5, 0.534, 0.58, 0.606, 0.64, 0.649, 0.687, 0.689, 0.764, 0.768, 0.774, 0.85, 0.843, 0.91, 0.987, 0.99, 1.003, 1.004, 1.015, 1.034, 1.064, 1.11, 1.159, 1.163, 1.08, 1.53, 1.87, 1.317, 1.30, 1.333, 1.41, 1.41, 1.438, 1.574, 1.719, 1.769, 1.830, 1.853, 1.930,.041,.053,.119,.146,.167,.37,.43,.49,.318,.35,.349,.37,.465,.497,.553,.56,.616,.739,.851, 3.09, 3.37, 3.335, 3.390, 3.447, 3.473, 3.568, 3.67, 3.718, 3.70, 3.814, 3.854, 3.99, 4.038, 4.065, 4.089, 4.177, 4.357, 4.403, 4.514, 4.771, 4.809, 4.87, 5.086, 5.191, 5.98, 5.95, 5.968, 6., 6.556, 6.670, 7.673, 8.071, 8.165, 8.181, 8.383, 8.557, 8.606, 9.03, 10.48, λe λx, λ D p-value σ : [0, ] 30

31 Hypothess testg Two exclusve hypotheses H 0 ad H 1 -whch oe s the most compatble wth data - how compatble s the other oe P(data H0) vs P(data H1) Buld a statstc, defe a terval w - f the observato falls to w : accept H else accept H 0 Sze of the test : how ofte dd you get t rght α L(x H 0 )dx w Power of the test : how ofte do you get t wrog! 1 β L(x H 1 )dx w Neyma-Pearso lemma : optmal statstc for testg hypothess s the Lkelhood rato (x H ) L λ < L (x H ) 0 kα 1

32 CL b ad CL s Two hypothess, for coutg expermet - backgroud oly : expect 10 evets - sgal+ backgroud : expect 15 evets You observe 16 evets CL b CL s+b CL b cofdece the backgroud hypothess (power of the test) Dscovery : 1- CL b < 5.7x10-7 CL s+b cofdece the sgal+backgroud hypothess (sze of the test) Rejecto : CL s+b < 5x10 - Test for sgal (o stadard) CL s CL s+b /CL b 3