Fall Analss of Epermental Measurements B. Esensten/rev. S. Errede More on : The dstrbuton s the.d.f. for a (normalzed sum of squares of ndependent random varables, each one of whch s dstrbuted as N (,. For such ndependent random varables (aka degrees of freedom: f ; = e Γ For = we should recover N (, : f ( ; = e = e = e Γ π π Defne: ( = and change varables: ( ; = should gve: g( f d g d = f ( ; However, solvng for as a functon of gves: = ± whch s a double-valued result! d d Ths stuaton s shown graphcall n the fgure below: 598AEM Lecture Notes 6
Fall Analss of Epermental Measurements B. Esensten/rev. S. Errede Snce s double-valued (.e. = ±, from the above graph t can be seen from smmetr that we have equal nfntesmal area element contrbutons da from both branches both of whch must be ncluded because both are assocated wth probablt denstes... Ths mples that: g Then for ( d ( f ; f ; f ; f ; = = = = + d + + d d = : f ( ; = e π Thus: g( = e e π = whch s ndeed (, N! Thus, we see that the Gaussan/normal dstrbuton s a specal case ( = of the dstrbuton! In fact, f we defne a varable Ch as: ( =, the.d.f. of s (, N. Also, recall that the sum of Gaussan/normall-dstrbuted random varables s tself a Gaussan/normall-dstrbuted random varable. One applcaton of the dstrbuton s n quanttatvel testng the compatblt of a set of epermentall measured values ( ( (, (,, ( mean/average values, ( ( (, (,, ( devaton uncertantes ( (, (,, ( wth a set of and the assocated -standard (. For a gven eperment wth the above epermental results, we can calculate the assocated wth comparng the epermental measurements to ther epected (or theoretcall-predcted values, whch s just a scalar quantt.e. just a number, rangng between and nfnt: = ( ( ( ( = We assume that the about ther the mean values ( s wth standard devatons ( s are random varables that are Gaussan/normall-dstrbuted. We then ask for the probablt that we would fnd a value of eceedng the epermentall observed, for degrees of freedom, f we were to repeat the epermental measurements that gave the orgnal set of ( (, (,, ( e.g. a gazllon tmes, ths s gven b the ntegral: rob( ; f ( ; d (% > = 598AEM Lecture Notes 6
Fall Analss of Epermental Measurements B. Esensten/rev. S. Errede Ths ntegral s shown graphcall n the fgure below: f ( ; ( ; ( ; (% rob > = f d The probablt rob( ; f ( ; d (% > = that we would fnd a value of eceedng the epermentall observed, for degrees of freedom also goes b other names the p-value, aka the Sngle-Sded Upper Confdence Level CL for degrees of freedom. Note that the cumulatve.d.f. s F( ; ( ; f d =, whch phscall s the probablt of fndng a value of less than or equal to the epermentall observed.e. rob( ; F ( ; f ( ; d (%, = s the unshaded area under the red curve n the above fgure. Thus, snce: rob( ; + rob( > ; = f ( ; d f ( ; d + = % we see that: rob > ; p-value CL f ; d ( f ( d F( rob( = ; = ; = % ; Tables of values of the ntegral rob( > ; and ( rob ; p-value CL are avalable see e.g. Crtcal Values of the Ch-Squared Dstrbuton on the 598AEM Lecture Notes web page. A plot of ( ; - ( ; rob > p value CL f d vs. the fgure below for varous choces of the N = = : 7 degrees of freedom: DoF s shown n 598AEM Lecture Notes 6 3
Fall Analss of Epermental Measurements B. Esensten/rev. S. Errede p-value = CL Cumulatve.D.F. The abscssa (-as s the value of our ( = α n above fgure and the ordnate (-as s: ( > ; - ( ;. rob p value CL f d So e.g. f our epermental measurement results n a = 5. for = 4 degrees of freedom, we see that rob( > = p value CL f ( = ; 4 - ; 4 9%, 5.e. 9% of the tme we would epect to fnd 5.. No roblem! Note that ths result has a normalzed per degree of freedom of N DoF = = 5. 4 =.5 ~.. Suppose =. We epect ths result <.% of the tme. Bad! Note that ths result has a normalzed per degree of freedom of N DoF = =. 4 = 5... Suppose we make Gaussan/normall-dstrbuted ndependent measurements of the,,,, and ther assocated -standard devaton uncertantes random varable : (,,, are apror known. Suppose that the apror known true mean of the random varable s ˆ, e.g. known from some knd of frst-prncples theor predcton. We can then defne a comparson between our epermental data vs. theor predcton as: ˆ = = 598AEM Lecture Notes 6 4
Fall Analss of Epermental Measurements B. Esensten/rev. S. Errede We can then calculate the correspondng numercal value of rob > ; p-value CL f ; d and see f t s acceptable or not. ( Now suppose that we don t apror know the true value of ˆ, but we have estmated ˆ e.g. b calculatng the weghted mean = ( =,,, = measurements. Then {here} the comparson between our epermental data vs. theor predcton becomes: of the = =. We now nvoke the LSQ rncple,.e. we must determne the best value for as the one whch mnmzes.e. we take the frst dervatve of w.r.t., set the result = * and then solve for, whch s our estmate of the true value ˆ : = = = = = = = = * = Ths relaton ndeed holds/s satsfed when: = = the weghted mean. * * Then, s the numercal value of when =,.e. =, = and rob( > ; p-value CL f ( ; d s the correspondng p-value/ CL. But wh are there degrees of freedom here n ths stuaton nstead of? Ths s not a trval queston the dervaton s a bt nvolved, but the answer(s are smple: If ˆ = = then s dstrbuted as the =, where ˆ s the apror known true mean of the random varable,.d.f. for degrees of freedom, f ( ; ; lower ; ; and hence rob CL F f d s dstrbuted as the Unform.D.F. U ( %,%,.e. ( ; lower ( ; ( ; rob CL F f d s flat/unforml-dstrbuted on the nterval ( %,%, 598AEM Lecture Notes 6 5
Fall Analss of Epermental Measurements B. Esensten/rev. S. Errede and hence: ( > ; - ( ; f ( d F( rob( rob p value CL f d = ; = ; = % = % ; s also flat/unforml-dstrbuted on the nterval ( %,%. If = = where the true mean ˆ s n fact unknown/unknowable, and s the aposteror-calculated weghted mean of the (,,, ndependent measurements of the random varable, then {here} s dstrbuted as the.d.f. for {not } f ;, because s a parameter that was post-facto derved/obtaned degrees of freedom from the measured data ( freedom, and hence,,,, and hence results n a reducton of one degree of ( ( ; ; ( ; lower (% rob CL F f d s dstrbuted as the Unform.D.F. U ( %,%,.e. ( ( ; ; ( ; lower (% rob CL F f d s flat/unforml-dstrbuted on the nterval ( %,% and hence ( > ; - ( ; f ( d F( rob( rob p value CL f d = ; = ; = % = % ; s also flat/unforml-dstrbuted on the nterval ( %,%. We wll prove ths latter case for the (algebracall smpler stuaton where all of the are equal,.e. = and apror known. Then the weghted mean s the same as the smple / arthmetc mean: = =. 598AEM Lecture Notes 6 6
Fall Analss of Epermental Measurements B. Esensten/rev. S. Errede Before we do ths, let us frst make a plausblt argument for the result: We have alread establshed for the case when the true mean ˆ (and standard devatons are apror known, that for repetton of ths eperment a gazllon tmes, the ˆ = = The correspondng s a random varable, dstrbuted as f ( ; for degrees of freedom. ( ( ; ; ( ; lower (% rob CL F f d U %,% s also a random varable and s dstrbuted as the Unform dstrbuton ( > ; - ( ; f ( d F( rob( rob p value CL f d = ; = ; = % = % ; s also a random varable and s dstrbuted as the Unform dstrbuton U ( %,%. and hence: Here, the true mean ˆ s an apror known/gven number, the standard devatons are apror known and ndependent measurements (,,, of the random varable have been made we sa that there are degrees of freedom assocated wth ths data set. However, n the stuaton where the true mean ˆ s not apror known, then: = = where we have aposteror used the ndependent measurements (,,, to calculate * an estmate = ˆ of the true mean, then we now n fact have a constrant among the. If s consdered to be an aposteror derved number, (.e. obtaned post-facto e.g. from a calculaton of the weghted mean usng the ndependent measurements (,,,, then one of the s can n fact be elmnated. Thus, n effect, we have used up one degree of freedom n the aposteror calculaton of the mean and hence onl degrees of freedom reman. And now for the proof... We begn wth a defned as: = =, where: = =, the smple/arthmetc mean. 598AEM Lecture Notes 6 7
Fall Analss of Epermental Measurements B. Esensten/rev. S. Errede Net, we defne a new set of varables b carrng out a so-called Helmert Transformaton : + 3 3 + + 3 34 3 43 + + 3 + + ( + + + + 3 It can be shown that ths s an orthogonal transformaton of varables, snce: (e.g. tr t for = 3. Net, we rewrte: = = = ( ( ( = = = + = + = + = = = ( = = = = = = = = = = + = + = = = = = = Thus, we see that: = = = = (whew! Now the s are lnear orthogonal combnatons of the s, whch (b hpothess are ndependent random varables, Gaussan/normall-dstrbuted as ( ˆ, are also random varables that are also Gaussan/normall-dstrbuted, but as (, N Therefore, the s N, but the are not all ndependent t wll turn out that of them are ndependent 598AEM Lecture Notes 6 8
Fall Analss of Epermental Measurements B. Esensten/rev. S. Errede Note that the epectaton value of s: E [ ] = E { E [ ] E [ ] } { ˆ ˆ} = = = In a smlar manner, we can show that the epectaton values E [ ] = for all. The epectaton value of s: E [ ] = E = E[ ] = E [ ] E [ ] + E [ ] However, the ( { },,, are a set of ndependent measurements of the random varable, and thus: E [ ˆ ˆ ˆ ] = E [ ] E [ ] = = and: E [ ] = E [ ] = E [ ]. Thus: E [ ] = E [ ] ˆ Smlarl, we can show that the epectaton values Thus, we see that the E [ ] = for all. s are ndeed Gaussan/normall-dstrbuted as (, are also Gaussan/normall-dstrbuted, but as (, N. N and thus the s dstrbuted as f ( = = ; = = for degrees of freedom, and hence the correspondng ( > ; - ( ; f ( d F( rob( rob p value CL f d = ; = ; = % = % ; s flat/unforml dstrbuted as U ( %,%. 598AEM Lecture Notes 6 9
Fall Analss of Epermental Measurements B. Esensten/rev. S. Errede The epectaton value E [ ] ˆ are apror known, then = = wll also be dfferent {here}. Recall that f the true mean ˆ (and and E[ ] =, the mean of the dstrbuton. Now we have where has been aposteror calculated from the sample mean = of the ndependent measurements of the random varable. However, we have also seen from the above that we can re-epress the as: = =, whch s the sum of squares of ndependent random varables, each of whch s dstrbuted as N (,. From ths latter form we can calculate e.g. for all E[ ] = = E[ ] = E[ ] =. = the same, that: We see that n ether case E [ ] = Number of Degrees of Freedom. Ths generalzes: If there are K lnear constrant equatons, then there reman onl K degrees of freedom, and the wll follow a f ( ; K.D.F. Thus, K <. The correspondng rob > ; K p-value CL f ; K d ( f ( K d F( K rob( K = ; = ; = % = % ; s flat/unforml dstrbuted as U ( %,%. If we were addtonall FITTING M λ -parameters, λ ( λ λ λ onl K M degrees of freedom, and the Thus, ( K + M <. The correspondng,,, M, then we would have wll follow a f ( ; K M.D.F. ( > ; - ( ; f ( K M d F( K M rob( K M rob K M p value CL f K M d = ; = ; = % = % ; s flat/unforml dstrbuted as U ( %,%. 598AEM Lecture Notes 6
Fall Analss of Epermental Measurements B. Esensten/rev. S. Errede A Smple Eample: Combnng and Comparng Results from Dfferent Eperments: Suppose we are e.g. wrtng a revew artcle on measurements of the lfetmes of elementar partcles. Suppose that there are 3 publshed measurements of the lfetme of the hperon whch have been reported b 3 dfferent eperments (n.b. the hperon has uds valence quark content, mass M 5.7 MeV c, and undergoes weak deca e.g. to pπ and/or nπ. The three eperment s publshed measurements of the lfetme are: τ =.66 ±. s τ =.6±. s τ 3 =.69 ±.3 s Here, we wll assume that the quoted uncertantes are the statstcal uncertantes and are Gaussan dstrbuted. (n.b. We should look at each publcaton to be certan!. We also assume (for smplct that an sstematc uncertantes are small compared to the quoted statstcal uncertantes. (n.b. Ths s frequentl not the case! Queston: What number should we quote as the World Average of the lfetme? roblem: The thrd eperment s lfetme measurement s sgnfcantl dfferent from the other two. Should t be ncluded n the World Average? In order to obtan an answer, we must frst nvestgate the detals of each eperment. If an one of them s suspcous, that s f an appear to have some dffcult whch suggests that ther resultng number ma be unrelable, then we shouldn t use t. We must also pa partcular attenton to the wa that each of the eperment s standard devatons were estmated! Suppose that all three of the eperments appear to be OK. Then we can quanttatvel test the hpothess that all three are, n fact, measurements of the same quantt. We form the for the three eperments as: 3 ( τ τwa ( τ τwa ( τ τwa ( τ3 τwa wa = = + + = τ τ τ τ3 3 τ τ τ τ + τ τ + τ 3 τ3 where τ wa s the weghted mean: τ wa = = =.63 + + Then: wa = τ τ τ τ (.66.63 (.6.63 (.69.63 = + + = 4.93...3 We then fnd, for = 3 = degrees of freedom, that: rob > 4.93; p-value CL f ; d 4.93 4.93 f ( d F rob( = ; = 4.93; = % = % 4.93; 8% 598AEM Lecture Notes 6
Fall Analss of Epermental Measurements B. Esensten/rev. S. Errede So 8% of the tme (on average, we would epect the to be at least ths large, e.g. f we conceptuall magned repeatng each of these three ndvdual eperments a gazllon tmes and calculated the world average lfetme and resultng for each repetton. We would (most lkel conclude that ths s an acceptable probablt, and that there s no reason to reject the hpothess that all three numbers are measurements of the same quantt. So we quote: Where τ wa =.63 ±.3 s τ s the -standard devaton settng-error uncertant on the weghted mean: τ = =.3 + + τ τ τ s Note however that the normalzed per degree of freedom s: wa = 4.93 =.465, whch s sgnfcantl greater than a normalzed N DoF =.. Thus, as dscussed n 598AEM Lect. Notes 4, p. 5-6, because wa = 4.93 =.465 >., we could alternatvel choose to keep the World Average weghted mean lfetme result, but nflate/ncrease the -standard devaton uncertant b a Scale Factor S, smpl defned as: τ Then the World Average τ S N = = 4.93 =.465 =.57, wa DoF wa = =. S.57.3 s. s.e. τ τ lfetme would thus be quoted as: =.63 ±. s, wth a scale factor S =.57 598AEM Lecture Notes 6
Fall Analss of Epermental Measurements B. Esensten/rev. S. Errede Another Eample: Comparson of a Data Hstogram to a Theor redcton Suppose we have n events dstrbuted n some manner. We classf the n events nto N classes or bns of a hstogram, and plot the number of events n each bn n versus the quantt that provdes the classfcaton, as shown n the fgure below. Call the numbers of events n the hstogram bns n, n,, nn where: n = n+ n + + nn = n. N Theor predcton n n 3 n n 4 n 5 "quantt" The theor predcton gves the apror probablt p that an event wll be classfed n the th bn. The theor predcts that n = npevents are epected (on average n the th bn. We then form the : N n n = n. How do we determne the n? If we choose the bn boundares such that each of the hstogram bns have n 3 entres, we epect the fluctuatons n n to be ~ Gaussan dstrbuted, and thus we can estmate n b So here we defne: n. hst N n np = n. In order to quanttatvel test whether or not the agreement between the theor predcton and the hstogram of the data s good, we calculate the probablt that wll eceed the above value of hst for N degrees of freedom, snce the n s are not all ndependent the are N related b the constrant n = n+ n + + nn = n. Thus, we would calculate: ( > hst ; - hst ( ; hst f ( N d F( hst N rob( hst N rob N p value CL f N d = ; = ; = % = % ; If ths result s reasonable,.e. sgnfcantl larger than e.g. ~ one few %, we would be nclned to accept the hpothess that the epermental data agrees wth the theor predcton. 598AEM Lecture Notes 6 3