Lecture 21: Order statistics

Transcription

1 Lecture : Order sttistics Suppose we hve N mesurements of sclr, x i =, N Tke ll mesurements nd sort them into scending order x x x 3 x N Define the mesured running integrl S N (x) = 0 for x < x = i/n for x i x < x i+ = for x x N 474

2 Order sttistics So S N (x) looks like this: /Ν /Ν 0 x x x N 475

3 Order sttistics We would like to compre this with the predicted x function F(x ) = P(x < x H 0 ) = f x H ) dx ( 0 /Ν /Ν 0 x x x N 476

4 Order sttistics The most fmous sttistic is D N mx S N (x) F(x) D N /Ν /Ν 0 x x x N 477

5 Kolmogorov-Smirnov test Turns out the probbility function for N / D N is distribution-free nd clculble for N lrge ( 0 or so) α = P(N / D N > z) = = ( ) exp( z ) z> few exp( z ) 478

6 Kolmogorov-Smirnov test D N nd P(N ½ D N > z) form the K-S test which is Independent of binning Vlid for smll/medium smple sizes (lso lrge, of course) Loses sensitivity ner endpoints (x or x N ), since both integrls tend to sme vlue (0 or ) (this shortcoming is ddressed in some of the vrints discussed in the book) Like χ, cn be generlized to the cse of compring two mesured distributions to see if they re drwn from different smples 479

7 Kolmogorov-Smirnov test Compring two different smples, our mesured running integrls re S M nd S N Defining D NM = mx S M (x) S N (x), the probbility of (MN/[M+N]) ½ D NM exceeds given vlue z is gin α = = P([NM/(N = ( ) (i.e. (MN/[M+N]) ½ D NM replces N ½ D N ) + M)] / exp( D z NM ) > z) 480

8 Crmer-Von Mises-Smirnov test Less well known is the sttistic W = ( S ( x) F( x)) f ( x) dx = ( S ( x) F( x)) df N 0 N The quntity NW is lso distribution free nd hs clculble probbility function α = P(NW > z) = = ( ) / (4 + ) / exp( [4 + ] /6z) K / 4 ([4 + ] /6z) Binomil coeff. Modified Bessel fn 48

9 Crmer-Von Mises-Smirnov test This test works for very smll smples (N 3 or so) is more sensitive thn K-S ner the endpoints is the most powerful -D test cnnot be esily generlized to compre -D mesured distributions cn be written in more esily evluted form: N (i ) W = + F( xi ) N N i= N 48

10 Modeling dt So fr, we hve discussed only hypothesis testing. Often, we ssume some composite hypothesis nd wish to constrin its dustble prmeters by experiment Suppose there re m unknown prmeters, (with =,m) nd N dt points 483

11 Modeling dt The dt re typiclly of two kinds: ) We vry some externl vrible x, nd mke N mesurements of some quntity y to obtin the results y i (x i ) ± σ i (with i =,N) where σ i is the Gussin uncertinty on y i Exmple: we dust the voltge x i = V i cross diode to N vlues nd mesure the current y i = I i ±σ i for ech vlue 484

12 Modeling dt The dt re typiclly of two kinds: ) We mesure N vlues of the rndom vrible x i in n experiment Exmple: mesurement of individul photon energies in X-ry stronomy 485

13 Modeling dt In both cses, we would like to determine the prmeter set tht best pproximtes the dt Need n estimtor which mesures how well the model mtches the dt For experiments of the first type, we commonly use the oint Gussin probbility function P( y ) exp( ½χ ( )) = N i= πσ i We wnt to mximize P(y ) by minimizing χ () (nd cn lso see if the model fits by clculting the significnce) 486

14 Confidence intervls Suppose we hve crried out the minimiztion of χ () The vlue of t the minimum yields our BEST ESTIMATE of the free prmeters, i cll this * Question: wht re our confidence limits on the i? 487

15 Confidence intervls Becuse our probbility distribution is symmetric under interchnge of nd y i ( x, ) i, we cn use y i P( y) exp( ½χ ( )) = P( y ) = N πσ i= i 488

16 489 Confidence intervls Let s use Tylor expnsion for χ (): where nd the probbility density is * ) ( k k C = χ Zero *) ( *) ( *) ( *) *)( ( *) ( *) ( ) ( *, * C T k k k + = + + = χ χ χ χ χ *)] ( *) ½( exp[ ) ½ exp( ) ( C P T = χ

17 Confidence intervls Define Δ = * Then Δχ = χ χ min with Δχ = Δ T C Δ C = Δ Δ T / Δχ which is clled the error mtrix 490

18 Confidence intervls The digonl elements of the error mtrix (C ) ii tell us (Δ i ) / Δχ for ech prmeter provided tht the nd order Tylor expnsion is dequte Thus, the Byesin probbility for given vlue of Δ i is P(Δ i y) exp( ½Δχ ) exp( ½(Δ i ) / (C ) ii ). Gussin with men 0 nd stndrd devition σ i = (C ) ii 49

19 The -D problem Let s consider the -D problem in which we hve ust two free prmeters nd The dt yield Δχ (, ) = χ (, ) χ 0 = [c (Δ ) + c (Δ ) +c (Δ Δ )] +. Note tht contours of fixed Δχ re ellipses in the (, ) plne 49

20 The -D problem Picture: Δχ = *+σ * * σ * σ * *+σ 493

21 The -D problem We find tht σ = (C ) = c /(c c c ) σ = (C ) = c /(c c c ) The 68% confidence limit on is * ± σ The 68% confidence limit on is * ± σ 494

22 The -D problem The oint probbility tht nd lie within the Δχ = ellipse is NOT 68%, however. The oint probbility content within the ellipse comes out to be exp( ½Δχ ) = 0.39 nd the ellipse with 68% probbility content hs Δχ =

23 The -D problem Esy to prove this for the cse where c = c = c, nd c = c = 0 Δχ = *+σ * * σ σ = σ = σ * σ * *+σ 496

24 The -D problem In this cse, the oint probbility is P exp( ½Δχ ) d d = exp( ½c( Δ = exp( ½c( r + Δ ))πrdr exp( ½Δχ ) d( Δχ ) )) d d = π exp( ½c( r )) dr nd thus the probbility tht Δχ exceeds some vlue, z, is simply e ½ z 497

25 Non Gussinity So fr we hve ssumed tht the nd order Tylor expnsion is dequte out to some vlue of Δχ This isn t necessrily the true for the desired vlue of Δχ In tht cse, we hve to plot the ctul contours nd get our confidence limits from them Leds to non-symmetric confidence intervls 498

26 Non Gussinity Recent exmple: limits on the pressure nd HD/H bundnce rtio in interstellr gs clouds observed with the Spitzer Spce Telescope Δχ = nd 499

27 Summry: sttisticl tsks nd tools Tsks Sttistic Test/method Hypothesis testing (reection criterion) ) Simple hypothesis H versus simple hypothesis H 0 b) Simple hypothesis H 0 lone Likelihood rtio, P(t H )/P(t H 0 ) Chi-squred Neymn-Person P(t H )/P(t H 0 ) > k P(χ > χ 0 H 0 ) < α c) Composite hypothesis H 0 lone d) Simple hypothesis H 0 lone, where the dt re the counts of smll number of events e) Hypothesis tht two dt sets re drw for the sme distribution Modeling of dt (prmeter fitting) Minimum χ Order sttistic, D N Order sttistic, W Chi-squred Order sttistic, D NM Chi-squred P(χ > χ min H 0 ) < α Kolmogorov-Smirnov Crmer-Von Mises- Smirnov P(χ > χ 0 H 0 ) < α Kolmogorov-Smirnov χ minimiztion with c.l. from Δχ contours or error mtrix 500