Likelihood Methods: A Companion to the NEPPSR analysis project. Colin Gay, Yale University

Transcription

1 Lkelhood Methods: A Companon to the EPPSR analyss project

2 Outlne We have arranged a hands-on mn-course on fttng technques usng the ROOT framework To ft data, we often use the MIUIT package from CER, whch s a fttng engne for numercally mnmzng It s bult-n to ROOT, or can be called stand-alone Instead of usng t as a black box, we thought we d show how conceptually smple ts operaton really s Quck revew of some bascs of probabltes Maxmum Lkelhood bascs Propertes of the ML method, usng specfc examples and YES, there wll be a test (Especally for repeat EPPSR offenders)

3 Probablty Bascs Suppose X s a random varable. The probablty of throwng X between x, x+dx s P(x)dx P(x) s called the probablty densty functon (pdf) for X P( x) dx = The Expectaton value of a functon f(x) over P(x) s: E( f) = f( x) P( x) dx The most common expectatons we use are the frst few moments of the pdf: E() = P( x) dx = E( x) = xp( x) dx = x E x x x x P x dx σ normalzaton mean (( ) ) = ( ) ( ) = varance See Crag Blocker s talk for a more detaled ntroducton

4 Probablty Bascs The Condtonal Probablty P(x a) s the pdf for X, gven that a s true For example, P(x d) = probablty that our detector measures a partcle passng a wre at dstance x, gven that the partcle s truly at dstance d Or: P(m m 0 ) = probablty of measurng mass m gven the true mass s m 0 We use pdfs all the tme n our Monte Carlos: we know true value of masses, trajectores, etc, and turn nto fnte samples of quanttes reconstructed by our detector va these pdfs Our job wth real data s the nverse: Gven a fnte sample of measurements of a quantty, to nfer our pdf and true value for the quantty

5 Samples Let X have pdf P(x). A Sample of sze s a set of {x } of throws of X. When we plot, e.g., the mass of all of our reconstructed top quarks, we are vsualzng our sample pulled from the pdf P(m m 0 ), where m 0 s the true top mass Our job s, based on our fnte sample, to estmate the true value m 0, and to quantfy how certan our estmaton s For ths, we need an estmator for m 0 Some estmators are better than others e.g. My estmator s 50 GeV, no matter what Ths s an estmator, but not a very good one!

6 Estmators What propertes would we lke n our estmators? Consstent: As our sample sze ncreases, we d lke our estmator to converge to the true value. Such an estmator s called consstent. Effcent: There s a theoretcal mnmum for the varance of an estmator about the true value, gven a sample of sze (called the Mnmum Varance Bound) An estmator wth varance equal to the MVB s effcent Unbased: An estmator whose expectaton value (mean) s equal to the true value s called unbased

7 Maxmum Lkelhood Estmators Suppose we have a sample of measurements of a varable m, and we know the pdf for m s P(m m 0 ). However, we don t know m 0 n fact, t s the physcs quantty we are nterested n measurng Form the lkelhood Lm ( ) = Pm ( m) 0 0 The Maxmum Lkelhood Estmator (MLE) m * for m 0 s the value of m 0 that maxmzes the jont lkelhood MLEs are not always unbased, but are consstent and effcent. They are also asymptotcally normal. Extractng the MLE for a quantty s called fttng the data

8 Least-squares ft (revew?) Frst ft most of us learn s a least-squares or χ ft Put data nto hstogram bns: centers x, value y wth uncertanty σ Choose functon f( x θ ) whch predcts the bn contents y as a functon of the ft parameters θ ( f( x θ ) y) Form χ = σ The Least-Squares Prncple states that the best estmate for the parameters are the ones whch mnmze χ θ

9 Bnned vs Unbnned fts The LSQ ft s an example of a bnned ft A LSQ ft has the nce property that n addton to supplyng the ft parameters, t also tells us how good the ft functon approxmates the data Bnned fts wth few (or zero) events per bn are problematc Gaussan approx of uncertanty σ on bn contents s 0 Contrbuton to χ undefned Root just gnores these bns => ft based hgh Lkelhood fts gve us the ablty to deal wth data n an unbnned way Ideal for small statstcs, or sparse data

10 MLE Propertes EPPSR project I ll develop the concepts of the ML ft usng an analytcally solvable case You wll wrte a fttng program, ft to data suppled by Stephane, Kevn and John, and can compare to analytc result I ll use a common real-world case fttng for the lfetme of a data sample of decay tmes

11 Lfetme Lkelhood We can wrte the probablty densty functon for an exponental decay n two ways: P( t ) t / = e or These are properly normalzed (more on ths later) P( t Γ ) =Γe Γt P( t ) dt = Construct the lkelhood L( ) = P( t ) from the tme measurements { t }, =,

12 Lfetme Lkelhood Rather than Maxmzng the lkelhood, we usually take the log umercally, the lkelhood can get extremely small, resultng n precson ssues. log s better. log s monotonc, so maxmzng logl s equvalent to L Our man fttng package, MIUIT, fnds Mnma, so we multply by - and Mnmze: log L( ) = log P( t ) = t / log( e ) t = (log + ) = log + t

13 Maxmzng the Lkelhood To fnd the mnmum, we set the frst dervatve to 0 ( log L) = 0 = = t * = = = = t t ( E( t) t mean) * s the ML estmator for the true lfetme

14 Lkelhood for a Lfetme

15 umercal method Whle we can solve ths system analytcally, n general we must buld up the lkelhood numercally by loopng over the events and addng up the log P: sum = 0; for (0; <; ++) { sum += log(tau) + t[]/tau; } Fnd mnmum sum by scannng n tau Ths allows for very complcated probablty functons to be handled n a straghtforward way

16 What about the wdth? What f we consdered the wdth to be our unknown, rather than the lfetme? Γt log L( Γ ) = log( Γ e ) = ( log Γ+Γt ) = log Γ+Γ t ( log L) = 0 = + t Γ Γ * Γ = * * (.e. Γ = / ) t

17 Unbasedness? Let s calculate the expectaton (mean) value of our * = t = estmator gven the true lfetme s E E t E t * ( ) = ( ) = ( ) = t / t e dt = = = t/ t/ t/ t e dt te e dt The mean value of our ft result for, f we repeat the experment many tmes, s the true value =>Unbased *

18 Bas (contnued) What about Γ? E Γ = E = P t Γ P t Γ dt dt Γ * ( ) ( )... ( )... ( )... t t * Hence Γ s a based estmator. Ths sounds bad However, fttng for lfetme or wdth gves the same answer: Remember * * Γ =/ You can get a fne ft wth ether. It should be no surprse that E( ) x E ( x ) so that f estmator x s unbased, /x must be based

19 MLE transformaton nvarance In fact, the result of the lkelhood ft s nvarant under parameter transformaton: If f(x) s any functon of the estmator x, then the MLE for f(x) satsfes * * ( f ( x)) = f( x ) That s, fttng for x, and then applyng the transformaton f gves the same result as fttng for f(x) Thus there s more than one way to skn any cat

20 Uncertanty on ML Estmator Taylor expand lkelhood about mnmum: log L log ( θ ) = log + ( θ θ ) L L θ * θ * θ= θ * θ = * θ = * logl θ θ L( ) L e θ * ( θ θ ) **If we consder ths a probablty densty for the true value of the parameter θ, we see t s a Gaussan, wth * mean and varance θ σ = V ( θ) = log L θ * θ = θ

21 Meanng of Lkelhood Depends on f you are Bayesan or Frequentst Whle most partcle physcsts are professed frequentsts, they also seem to me to be closet Bayesans at tmes Let s be a bt more careful n notaton. The lkelhood s L({ t} ) = P( t ) Bayes: We d lke to nvert the probablty, and consder ths as the probablty densty for the true value gven the observatons t In general P(A B) P(B A). In fact, Bayes thm s: P( B A) P( A) P( A B) = PB ( ) (follows from P( ABPB ) ( ) = PB ( APA ) ( ))

22 Meanng of Lkelhood Thus P( data) = P( data ) P( ) Pdata ( ) P( data) s some constant normalzaton If we take the so-called pror probablty functon to be flat, then the lkelhood s the probablty dstrbuton for the true value of P( ) P( data) = P( data ) = L( )

23 ML Uncertantes Wrtng n a more suggestve way: * ( ) log L ( ) log L θ θ = θ θ * σ Hence the values of the lkelhood for the true value of beng,, n σ from the central value are * log L( θ ± σ) = log L * + θ ± = + * log L( θ σ) log L * θ log L( θ ± nσ) = log L * + n θ * ( f x) For fts, χ = χ =, σ correspond to,, 3 σ excursons, 4, 9 Same for ML fts, except factor of / θ

24 Lkelhood Uncertanty

25 Estmated Varance of Lfetme ft In our case of the lfetme ft: log L= log + t = ( log L) = 0 = t ( log L) = + * * *3 = = + = t * * * σ = = ( log L) = * *

26 Lfetme ft Uncertantes ote that samples wth ftted lfetmes that fluctuate low ALSO have the smallest uncertanty! Ths can easly (and dd/does) cause world-averages of many measurements to be based low, as low values get the largest weght More correct to combne the lkelhood curves to average many experments Ths s now done behnd the scenes for many measurements Experts supply samplng of the lkelhood curve for ther ft L curves are ADDED Mnmum of summed L found, and L=/ gves combned uncertanty

27 Goodness of Ft Consder the two data sets below, made nto hstograms for vsualzaton. Both result n the same ML estmator for the lfetme. Surely, data set s more lkely than data set, rght? Surely, snce t s more exponental, the value of the lkelhood functon for the st should be larger than for the nd, rght? Each events probablty should be hgher, resultng n a net larger lkelhood.

28 Goodness of Ft Unfortunately, ths s wrong. Recall: log L= log + t and = * t = log L mn = + t = t + t * log log( ) * t = + + lo t = + ) * ( log g ) (log AY sample wth the same t produces the SAME * estmate wth the SAME value for the logl of = t / * (log + ) The two prevous fts are equally lkely! Ths s a weakness of the ML method t doesn t naturally supply a goodness of ft metrc

29 Moral The ML method wll not save you from yourself! Santy check of results essental! How lkely s lkely needs care: e.g. from a self-help mathematcs webste: Defnton of Unlkely Event The event that may not happen s an unlkely event. In other words, unlkely event s an event that s not lkely to happen.

30 Lkelhood ormalzaton You must be vglant that your lkelhood s properly normalzed (or at least ts normalzaton doesn t change) Easest way s to normalze each buldng-block probablty dstrbuton Ignorng ths can cause real problems: Consder a ML ft wth 0,000 data events Suppose the normalzaton of the underlyng pdfs changes from to ,000 log L= log P( t ) 0,000 0,000 0,000 0,000 log Pt ( ) = (log log Pt ( )) = (.000+ log Pt ( )) = log Pt ( ) More than sgma Change!

31 ML Ft wth Constrant It s easy to add external constrants on ft parameters Let s assume Gaussan uncertanty on constrant e.g. add n the world-average knowledge as gven by PDG L= P( PDG, σpdg ) P( t ) log L= log P(, σ ) + log P( t ) log PDG PDG PDG ( ) σ PDG PDG = log( e ) + π σ = Orgnal PDG PDG log Pt ( ) ( ) Pt ( ) + + log π σ σ χ penalty Constant PDG Wanderng by sgma from constrant costs ½ unt of lkelhood (= sgma)

32 Lfetme wth mperfect Detector All detectors have non-zero tme resoluton -> Let s add ths effect n to our ft Assume detector has a gaussan resoluton functon, wth mean = 0 (.e. unbased) and wdth Pt Expt G ' ( ) = (, ) (0, σ) = πσ 0 ' ( t t ) ' t / σ ' e e dt Consder the exponentals. The exponent s: ' ' t ( t t ) ' ' σ ' = ( t t t+ t + t ) σ σ ' ' σ ( ( ) t t t t ) = + σ 4 ' σ σ t σ = ( t ( t )) + + σ σ t σ = ( ( )) σ ' t t

33 Lfetme wth Imperfectons PDF becomes Pt ( ) = ' / ' ' σ / ' ' t σ e e e dt σ σ 0 ( t ( t )) σ σ σ t / ' σ ( t ) πσ t/ πσ t = e e e dt 0 πσ = = e e π σ ( t ( t )) σ t σ e e e dt t σ ( ) σ e x dx t / t σ = e e erfc( ( )) σ

34 Effect of Worsenng Resoluton σ / t t σ Pt ( ) = e e erfc( ( )) σ σ As grows, pdf looks less lke exponental, more lke the gaussan resoluton functon

35 Lfetme wth Cutoff Suppose we are measurng the ½ lfe of some radoactve materal, but we only have T days to run our experment Our probablty densty for observng a decay must cut off at T days, affectng the normalzaton Pt ( ) = e T / e t / log L( ) log( e ) T / e = t / T / = (log( e ) + log ) t ow solve numercally ote that the normalzaton changes wth

36 Lfetme wth Background Suppose we have a prompt (0 lfetme) background n our sample (measured wth resoluton σ ) We can easly handle ths by makng our per-event probablty: Pt f = f e G σ + fgt σ t / (, ) (0, ) ( ) ( 0, ) f = fracton of sgnal G = gaussan Form log L as usual, mnmze wrt both, f

37 Multple varables Maxmum Lkelhood ft easly handles multple ft varables smply multply n the probablty denstes for each new varable Least-squares ft has problems wth such fts As we add more and more dmensons to our ft, bnned methods encounter trouble Data gets spread thn over large number of hstogram bns, resultng n few entres (or zero) per bn => problematc for the ft e.g. Addng the reconstructed mass to the decay tme as varables to ft and dstngush sgnal from background Ptm f m f e G Gm m + ( f){ G( t 0, σ )( am+ b)} t / (,,, B) = { (0, σ)) ( B, σm)} Gaussan sgnal Lnear background

38 Summary Maxmum Lkelhood ft s a powerful, convenent technque for estmatng parameters from fnte samples Unbnned, so small statstcs, sparse data ok Best choce for many-parameter fts For large, gves unbased value, converges to true value, and has mnmum uncertanty possble Constant normalzaton crtcal Auxlary goodness of ft requred Vsualzaton can be dffcult Most serous fts you do n your career wll be Maxmum Lkelhood fts Stephane wll talk about the hands-on part of the project