PHYSICS 2150 EXPERIMENTAL MODERN PHYSICS. Lecture 3 Statistical vs. Systematic Errors,Rejection of Data; Weighted Averages

Transcription

1 PHYSICS 150 EXPERIMENTAL MODERN PHYSICS Lctur 3 Statistical vs. Systmatic Errors,Rjction of Data; Wightd Avrags

2 SO FAR WE DISCUSS STATISTICAL UNCERTAINTY. NOW WHAT ABOUT UNCERTAINTY INVOLVED WITH MEASUREMENT ITSELF? (SYSTEMATIC UNCERTAINTY)

3 ANALYZING ERROR ON A QUANTITY You ar in a car on a bumpy road on a rainy day, and ar trying to masur th lngth of th moving windshild wipr with a shaky rulr. You masur it 37 tims. 1σ rror on man Th histogram of your rsults is at right. It dosn t look vry gaussian. But, with only 37 masurmnts plottd in lots of bins, distributions oftn look ratty.

4 ANALYZING ERROR ON A QUANTITY Th 1σ uncrtainty on th man is 1.1 cm. If w assum th undrlying distribution is nvrthlss gaussian and cntrd on th tru valu, w can turn this into a confidnc lvl: th wipr lngth is btwn 55.1 and 57.3 cm with 68% confidnc. 1σ rror on man

5 ANALYZING ERROR ON A QUANTITY Stop th car, go outsid and masur th wipr proprly: it is 61.0 cm long! W said th wipr lngth was btwn 55.1 and 57.3 cm with 68% confidnc. W ar off by ovr 4 sigma. This is shockingly unlikly! Clarly thr is a systmatic shift. Th distribution dos not cntr on th tru valu. Mor data points won t gt us any closr to th tru valu. W nd to mak bttr masurmnts, or find th sourc of th rror and apply a corrction to th data.

6 STATISTICAL (RANDOM) VS. SYSTEMATIC UNCERTAINTIES Statistical Systmatic No prfrrd dirction Changs with ach data point: taking mor data rducs man. Gaussian modl is usually good (xcpt counting xprimnts) Bias on th masurmnt: Only on dirction Stays th sam for ach masurmnt: mor data won t hlp you Gaussian modl is usually trribl, but w us it anyway

7 STATISTICAL (RANDOM) VS. SYSTEMATIC UNCERTAINTIES Kp statistical, systmatic rrors sparat. Rport rsults as somthing lik: g = [965 ± 30(stat) ± 1(sys)] cm/s Add in quadratur (not that this assums Gaussian distribution) to compar with known valus: g = [965 ± 3(total)] cm/s

8 EXAMPLE: E/M EXPERIMENT Elctrons acclratd with 40, 60, or 80V Magntic fild prpndicular to vlocity Lorntz forc F = v B forcs lctrons into circular paths Cntriptal forc: Elctron nrgy: Magntic fild of Hlmholtz coils is This givs F = mv r 1 mv = V m =3.906 Va µ 0 N I r B = 8µ 0NI a 15

9 m =3.906 Va µ 0 N I r Variabl Dfinition How Dtrmind V Acclrating Potntial masurd a Hlmholtz Coil radius masurd μ0 Prmability of fr spac constant N Numbr of turns in ach coil givn r Elctron bam radius givn I=IT-I0 Nt currnt calculatd IT Total currnt masurd I0 Cancllation currnt masurd Want /m with systmatic and statistical uncrtaintis

10 DATA FOR m =3.906 Va µ 0 N I r Thr masurmnts of I0={0.17, 0.0, 0.3} I0=0.0±0.03A Comput /m for 14 diffrnt combinations of V and pin radii Entry I (A) r (m) V (V) /m (10 11 C/kg)

11 E/M: STATISTICAL UNCERTAINTY Man: x = 1 N Standard dviation: N i=1 x = 1 N 1 Uncrtainty on man: x i = C/kg N i=1 (x i x) 1 = C/kg x = x N = C/kg Rsult with statistical uncrtainty only: m =(1.946 ± 0.019) 1011 C/kg

12 E/M: SYSTEMATIC UNCERTAINTY m =3.906 Va µ 0 N I r Uncrtainty in a powr: q(x) =x n q q = n x x Error propagation givs: m m sys = V V + a a + I I + r r

13 E/M: SYSTEMATIC UNCERTAINTY m m sys = V V + a a + I I + r r Acclration Voltag: V V 0.1V V = 60V =0.003 Hlmholtz Coil radius: a a = 0.3 cm 33. cm =0.009 Elctron bam radius: r r = 0.00 cm 4.47 cm =0.004

14 E/M: SYSTEMATIC UNCERTAINTY Nt Currnt: I = I 0 + I T I = ( I T ) +( I 0 ) from statistical uncrtainty: I 0 =0.0 ± 0.03 A from mtr: ( A+0.01 A) = A I 0 = (0.03 A) + (0.01 A) =0.03 A from mtr: ( A A) = A I I = (0.03 A) + (0.019 A).65 A =0.013

15 E/M: SYSTEMATIC UNCERTAINTY m m sys = V V + a a + I I + r r m m sys = ( 0.009) + ( 0.013) + ( 0.004) m m sys =0.033 = 3.3% m sys = C/kg = C/kg

16 E/M: UNCERTAINTY Statistical: m stat = C/kg Systmatical: m sys = C/kg (Almost) Final Rsult: m =(1.946 ± 0.019(stat.) ± 0.064(sys.)) 1011 C/kg Final Rsult (significant digits!): m =(1.95 ± 0.0(stat.) ± 0.06(sys.)) 1011 C/kg

17 E/M: SUMMARY OF RESULTS Masurd Valu: m =(1.95 ± 0.0(stat.) ± 0.06(sys.)) 1011 C/kg Accptd Valu: m = ( ± ) 1011 C/kg Discrpancy from Accptd Valu: m =( ) 1011 C/kg = C/kg Significanc of Discrpancy: m total = =.8 th rsult is off by.8σ +0.06

18 HOW GOOD IS OUR RESULT? What is th probability for a valu to dviat mor than.8σ from th man for a Gaussian distribution? Probability that x insid.8! : µ+.8 µ.8 Probability that x outsid.8! : (x µ) dx = µ µ µ µ+ µ = = 0.51% Vry unlikly rsult This rsult rquirs furthr analysis of possibl rror sourcs

19 PREVIOUS LECTURE: GAUSS DISTRIBUTION 1.5 p(x µ, )= 1 1 ( x µ ) µ=, σ= µ=3, σ= µ=4, σ=

20 WE CAN NOW ANSWER WHY ERRORS ADD IN QUADRATURE Masur indpndnt quantitis A and B and calculat sum 1.0 p(a µa, σa) p(b µb, σb)

21 WHAT IS p(a + B µ A+B, A+B)? Probability to masur A AND B simultanously: p(a µ A, A) p(b µ B, B) 1 ( A µ A A ) 1 ( B µ B B ) 1 h ( A µ A A ) +( B µ B B )i W now hav in fact probability dnsity for A+B and Z: p(a + B,Z µ A + µ B, ( A + B ) 1 ) 1 x o + y p 1 = (x + y) o + p + (px oy) op(o + p) (A+B µ A µ B ) A + B 1 Z (A+B µ A µ B ) A + B 1 Z

22 HOW ABOUT Z? W only car about A+B, so w intgrat ovr all valus of Z: p(a + B) = p(a + B,z)dz 1 (A+B µ A µ B ) A + B 1 Z dz Probability dnsity for A+B is also a Gaussian p(a + B) = 1 A + B with th standard dviation A+B = A + B 1 (A+B µ A µ B ) A + B

23 WE CAN NOW ANSWER WHY ERRORS ADD IN QUADRATURE 1.0 p(a µa, σa) A+B = A + B p(b µb, σb) p(a+b µa+µb, (σb +σb ) 1/ )

24 WE CAN ALSO JUSTIFY THE MEAN BEING THE BEST ESTIMATE Obtain data finit data st x1, x,...,xn and want to find th tru valu X 60 0 p(x)? V -5V -4V -3V -V -1V Elctrostatic Grain Potntial If w would know th limiting distribution p(x), w would also know X, but w don t!

25 DO WE REALLY NEED THE LIMITING DISTRIBUTION? Lt s assum that th dviation of an individual masurmnt xi from X follows a Gaussian distribution p(x i )= 1 x 1 ( xi X ) Th probability to obtain th data st x1,...,xn is thn p(x 1,x,..., x N )=p(x 1 ) p(x )... p(x N ) 1 N 1 ( x 1 X )... 1 xn X 1 N P 1 N i (x 1 X)

26 MAXIMUM LIKELIHOOD PRINCIPLE p(x 1,x,..., x N ) 1 N P 1 N i (x 1 X) Which is th most liklist valus for X for our data st x1, x,..., xn? th X for which p(x1, x,..., xn) is maximum p(x1, x,..., xn) is maximum if th xponnt is minimum Nd to find minimum of chi squar : d or dx = N i (x i X)=0 X = 1 N Th man is th bst stimat for X N i=1 N = i=1 x (x i X)

27 REJECTING DATA DON T!!!!!!! Bst way is to tak mor data!

28 REJECTING DATA W oftn find suspicious data points Diffrnt way th data was collctd? 4 Error during data rcording? - It is vr lgitimat to discard thm?

29 REJECTING DATA B 10 vry carful - you ar trading in th footstps of a long lin of practitionrs of pathological scinc! 8 Thr should b an xtrnal rason for rjcting data! 6 But 4 vn this may not bn nough: Th data may just b in conflict with our xpctation By rjcting data w may bias th data st and produc bogus rsults

30 REJECTING DATA 10 8 Thr ar no gnral rcips for rjcting data! All procdurs for rmoving suspicious data ar controvrsial! Will dscrib on which is popular in txtbooks (but not in ral lif): Chauvnt s critrion

31 A CAUTIONARY TALE: HOW TO LOOK FOR A PARTICLE 1.Look in high-nrgy collisions for vnts with multipl output particls that could b dcay products (displacd from primary intraction, if particl is longlivd as with th K 0 ). Thos of you doing th K mson xprimnt hav alrady sn this.rconstruct a rlativistic invariant mass from th momnta of th dcay products.

32 A CAUTIONARY TALE: HOW TO LOOK FOR A PARTICLE 3.Mak a histogram of th masss from candidat vnts 4.Look for a pak, indicating a stat of wll-dfind mass

33 A CAUTIONARY TALE: ONE PEAK OR TWO? MV using thir background and rsonanc assumptions, on obtains an accptabl confidnc lvl for th dipol. On also obtains an accptabl dipol fit ovr th whol mass spctrum if on assums a scond-ordr background. Furthrmor, on has to not that th xtrmly crucial background bhavior at both nds of th spctrum is basd on -6 vnts pr 10-MV bin. Th sam procdurs incras th confidnc lvl for a dipol in th p ir+ vnts by a considrabl amount. Asid from statistics and background considrations, on must bar in mind th vry gnral fact that it is much asir not to s a splitting than to s it, bcaus of a varity of rsolution-killing ffcts that ar normally hard to track down, both in countr and bubbl-chambr xprimnts. Exciting nw rsults on th nutral A wr rportd, at th Kiv Intrnational High Enrgy Confrnc in Sptmbr, by T. Massam of th group at CERN hadd by A. Zichichi. In th first rportd obsrvation of th splitting in An, th CERN countr group masurd th rcoil nutron in th chargxchang raction CERN xprimnt in lat 1960s obsrvd A msons Particl appard to b a a. a. doublt o UJ CO Statistical significanc of split is vry high 7I-- + p - * n + A at a bam momntum of 3. GV/c. Thy saw a markd dip at th cntr of th Afl. Confidnc lvls for a singl pak, incohrnt doubl pak and dipol wr 1%, 3% and 67% rspctivly. Thr is rally only on particl!! Dpndnc of splitting MISSING MASS (GEV) Fits to th two-pak structur of data from th CERN missing-mass and boson spctromtr group for th A, Th black curv is th fit for two cohrnt To arriv at som conclusions concrning th A splitting w will look for variabls th ffct may dpnd on. Th dpndnc or indpndnc might giv a clu to th natur of th A. W will discuss th possibl dpndnc of th A splitting on four quantitis: bombarding nrgy, final stat, production raction and momntum transfr. Th ffct of symmtric splitting has

34 A CAUTIONARY TALE: HOW DID THIS HAPPEN? MV using thir background and rsonanc assumptions, on obtains an accptabl confidnc lvl for th dipol. On also obtains an accptabl dipol fit ovr th whol mass spctrum if on assums a scond-ordr background. Furthrmor, on has to not that th xtrmly crucial background bhavior at both nds of th spctrum is basd on -6 vnts pr 10-MV bin. Th sam procdurs incras th confidnc lvl for a dipol in th p ir+ vnts by a considrabl amount. Asid from statistics and background considrations, on must bar in mind th vry gnral fact that it is much asir not to s a splitting than to s it, bcaus of a varity of rsolution-killing ffcts that ar normally hard to track down, both in countr and bubbl-chambr xprimnts. Exciting nw rsults on th nutral A wr rportd, at th Kiv Intrnational High Enrgy Confrnc in Sptmbr, by T. Massam of th group at CERN hadd by A. Zichichi. In th first rportd obsrvation of th splitting in An, th CERN countr group masurd th rcoil nutron in th chargxchang raction In an arly run, a dip showd up. It was a statistical fluctuation, but popl noticd it and suspctd it might b ral a. a. Subsqunt runs wr lookd at as o UJ CO thy cam in. If no dip showd up, th run was invstigatd for problms. Thr s usually a minor problm somwhr in a complicatd xprimnt, so most of ths runs wr cut from th sampl. 7I-- + p - * n + A at a bam momntum of 3. GV/c. Thy saw a markd dip at th cntr of th Afl. Confidnc lvls for a singl pak, incohrnt doubl pak and dipol wr 1%, 3% and 67% rspctivly. Dpndnc of splitting MISSING MASS (GEV) Fits to th two-pak structur of data from th CERN missing-mass and boson spctromtr group for th A, Th black curv is th fit for two cohrnt To arriv at som conclusions concrning th A splitting w will look for variabls th ffct may dpnd on. Th dpndnc or indpndnc might giv a clu to th natur of th A. W will discuss th possibl dpndnc of th A splitting on four quantitis: bombarding nrgy, final stat, production raction and momntum transfr. Th ffct of symmtric splitting has

35 A CAUTIONARY TALE: HOW DID THIS HAPPEN? MV using thir background and rsonanc assumptions, on obtains an accptabl confidnc lvl for th dipol. On also obtains an accptabl dipol fit ovr th whol mass spctrum if on assums a scond-ordr background. Furthrmor, on has to not that th xtrmly crucial background bhavior at both nds of th spctrum is basd on -6 vnts pr 10-MV bin. Th sam procdurs incras th confidnc lvl for a dipol in th p ir+ vnts by a considrabl amount. Asid from statistics and background considrations, on must bar in mind th vry gnral fact that it is much asir not to s a splitting than to s it, bcaus of a varity of rsolution-killing ffcts that ar normally hard to track down, both in countr and bubbl-chambr xprimnts. Exciting nw rsults on th nutral A wr rportd, at th Kiv Intrnational High Enrgy Confrnc in Sptmbr, by T. Massam of th group at CERN hadd by A. Zichichi. In th first rportd obsrvation of th splitting in An, th CERN countr group masurd th rcoil nutron in th chargxchang raction Whn a dip appard, thy didn t look as carfully for a problm. a. So an insignificant fluctuation was a. o boostd into a compltly wrong discovry. UJ CO 400-7I-- + p - * n + A at a bam momntum of 3. GV/c. Thy saw a markd dip at th cntr of th Afl. Confidnc lvls for a singl pak, incohrnt doubl pak and dipol wr 1%, 3% and 67% rspctivly. Lsson: Don t lt rsult influnc which data sts you us/want. Dpndnc of splitting MISSING MASS (GEV) Fits to th two-pak structur of data from th CERN missing-mass and boson spctromtr group for th A, Th black curv is th fit for two cohrnt To arriv at som conclusions concrning th A splitting w will look for variabls th ffct may dpnd on. Th dpndnc or indpndnc might giv a clu to th natur of th A. W will discuss th possibl dpndnc of th A splitting on four quantitis: bombarding nrgy, final stat, production raction and momntum transfr. Th ffct of symmtric splitting has