PHYSICS 2150 EXPERIMENTAL MODERN PHYSICS. Lecture 3 Rejection of Data; Weighted Averages

Transcription

1 PHYSICS 15 EXPERIMENTAL MODERN PHYSICS Lecture 3 Rejection of Data; Weighted Averages

2 PREVIOUS LECTURE: GAUSS DISTRIBUTION 1.5 p(x µ, )= 1 e 1 ( x µ ) µ=, σ=.5 1. µ=3, σ=.5.5 µ=4, σ=

3 WE CAN NOW ANSWER WHY ERRORS ADD IN QUADRATURE Measure independent quantities A and B and calculate sum 1. p(a µa, σa).8.6 p(b µb, σb)

4 WHAT IS p(a + B µ A+B, A+B)? Probability to measure A AND B simultaneously: p(a µ A, A) p(b µ B, B) e 1 ( A µ A A ) e 1 ( B µ B B ) e 1 h ( A µ A A ) +( B µ B B )i We now have in fact probability density for A+B and Z: p(a + B,Z µ A + µ B, ( A + B ) 1 ) e 1 x e o + y p 1 = (x + y) o + p + (px oy) op(o + p) (A+B µ A µ B ) A + B e 1 Z (A+B µ A µ B ) A + B e 1 Z

5 HOW ABOUT Z? We only care about A+B, so we integrate over all values of Z: p(a + B) = p(a + B,z)dz e 1 (A+B µ A µ B ) A + B e 1 Z dz Probability density for A+B is also a Gaussian p(a + B) = 1 A + B e with the standard deviation A+B = A + B 1 (A+B µ A µ B ) A + B

6 WE CAN NOW ANSWER WHY ERRORS ADD IN QUADRATURE 1. p(a µa, σa) A+B = A + B p(b µb, σb) p(a+b µa+µb, (σb +σb ) 1/ )

7 WE CAN ALSO JUSTIFY THE MEAN BEING THE BEST ESTIMATE Obtain data finite data set x1, x,...,xn and want to find the true value X 6 p(x)? V -5V -4V -3V -V -1V Electrostatic Grain Potential If we would know the limiting distribution p(x), we would also know X, but we don t!

8 DO WE REALLY NEED THE LIMITING DISTRIBUTION? Let s assume that the deviation of an individual measurement xi from X follows a Gaussian distribution p(x i )= 1 x e 1 ( xi X ) The probability to obtain the data set x1,...,xn is then p(x 1,x,..., x N )=p(x 1 ) p(x )... p(x N ) 1 N e 1 ( x 1 X )... e 1 xn X 1 N e P 1 N i (x 1 X)

9 MAXIMUM LIKELIHOOD PRINCIPLE p(x 1,x,..., x N ) 1 N e P 1 N i (x 1 X) Which is the most likeliest values for X for our data set x1, x,..., xn? the X for which p(x1, x,..., xn) is maximum p(x1, x,..., xn) is maximum if the exponent is minimum Need to find minimum of chi square : d or dx = N i (x i X)= X = 1 N The mean is the best estimate for X N i=1 N = i=1 x (x i X)

10 REJECTING DATA DON T!!!!!!! Best way is to take more data!

11 REJECTING DATA We often find suspicious data points Different way the data was collected? 4 Error during data recording? - It is ever legitimate to discard them?

12 REJECTING DATA Be 1 very careful - you are treading in the footsteps of a long line of practitioners of pathological science! 8 There should be an external reason for rejecting data! 6 But 4 even this may not been enough: The data may just be in conflict with our expectation By rejecting data we may bias the data set and produce bogus results

13 REJECTING DATA 1 8 There are no general recipes for rejecting data! All procedures for removing suspicious data are controversial! Will describe one which is popular in textbooks (but not in real life): Chauvenet s criterion

14 A CAUTIONARY TALE: HOW TO LOOK FOR A PARTICLE 1.Look in high-energy collisions for events with multiple output particles that could be decay products (displaced from primary interaction, if particle is longlived as with the K ). Those of you doing the K meson experiment have already seen this.reconstruct a relativistic invariant mass from the momenta of the decay products.

15 A CAUTIONARY TALE: HOW TO LOOK FOR A PARTICLE 3.Make a histogram of the masses from candidate events 4.Look for a peak, indicating a state of well-defined mass

16 A CAUTIONARY TALE: ONE PEAK OR TWO? MeV using their background and resonance assumptions, one obtains an acceptable confidence level for the dipole. One also obtains an acceptable dipole fit over the whole mass spectrum if one assumes a second-order background. Furthermore, one has to note that the extremely crucial background behavior at both ends of the spectrum is based on -6 events per 1-MeV bin. The same procedures increase the confidence level for a dipole in the p ir+ events by a considerable amount. Aside from statistics and background considerations, one must bear in mind the very general fact that it is much easier not to see a splitting than to see it, because of a variety of resolution-killing effects that are normally hard to track down, both in counter and bubble-chamber experiments. Exciting new results on the neutral A were reported, at the Kiev International High Energy Conference in September, by T. Massam of the group at CERN headed by A. Zichichi. In the first reported observation of the splitting in An, the CERN counter group measured the recoil neutron in the chargeexchange reaction CERN experiment in late 196s observed A mesons 5 - Particle appeared to be a a. a. doublet o UJ CO Statistical significance of split is 4 - very high 7I-- + p - * n + A at a beam momentum of 3. GeV/c. They saw a marked dip at the center of the Afl. Confidence levels for a single peak, incoherent double peak and dipole were 1%, 3% and 67% respectively. There is really only one particle!! Dependence of splitting MISSING MASS (GEV) Fits to the two-peak structure of data from the CERN missing-mass and boson spectrometer group for the A, The black curve is the fit for two coherent To arrive at some conclusions concerning the A splitting we will look for variables the effect may depend on. The dependence or independence might give a clue to the nature of the A. We will discuss the possible dependence of the A splitting on four quantities: bombarding energy, final state, production reaction and momentum transfer. The effect of symmetric splitting has

17 A CAUTIONARY TALE: HOW DID THIS HAPPEN? MeV using their background and resonance assumptions, one obtains an acceptable confidence level for the dipole. One also obtains an acceptable dipole fit over the whole mass spectrum if one assumes a second-order background. Furthermore, one has to note that the extremely crucial background behavior at both ends of the spectrum is based on -6 events per 1-MeV bin. The same procedures increase the confidence level for a dipole in the p ir+ events by a considerable amount. Aside from statistics and background considerations, one must bear in mind the very general fact that it is much easier not to see a splitting than to see it, because of a variety of resolution-killing effects that are normally hard to track down, both in counter and bubble-chamber experiments. Exciting new results on the neutral A were reported, at the Kiev International High Energy Conference in September, by T. Massam of the group at CERN headed by A. Zichichi. In the first reported observation of the splitting in An, the CERN counter group measured the recoil neutron in the chargeexchange reaction In an early run, a dip showed up. It was a statistical fluctuation, but people noticed it and suspected it might be real. 5 - a. a. Subsequent runs were looked at as o UJ CO 4 - they came in. If no dip showed up, the run was investigated for problems. There s usually a minor problem somewhere in a complicated experiment, so most of these runs were cut from the sample. 7I-- + p - * n + A at a beam momentum of 3. GeV/c. They saw a marked dip at the center of the Afl. Confidence levels for a single peak, incoherent double peak and dipole were 1%, 3% and 67% respectively. Dependence of splitting MISSING MASS (GEV) Fits to the two-peak structure of data from the CERN missing-mass and boson spectrometer group for the A, The black curve is the fit for two coherent To arrive at some conclusions concerning the A splitting we will look for variables the effect may depend on. The dependence or independence might give a clue to the nature of the A. We will discuss the possible dependence of the A splitting on four quantities: bombarding energy, final state, production reaction and momentum transfer. The effect of symmetric splitting has

18 A CAUTIONARY TALE: HOW DID THIS HAPPEN? MeV using their background and resonance assumptions, one obtains an acceptable confidence level for the dipole. One also obtains an acceptable dipole fit over the whole mass spectrum if one assumes a second-order background. Furthermore, one has to note that the extremely crucial background behavior at both ends of the spectrum is based on -6 events per 1-MeV bin. The same procedures increase the confidence level for a dipole in the p ir+ events by a considerable amount. Aside from statistics and background considerations, one must bear in mind the very general fact that it is much easier not to see a splitting than to see it, because of a variety of resolution-killing effects that are normally hard to track down, both in counter and bubble-chamber experiments. Exciting new results on the neutral A were reported, at the Kiev International High Energy Conference in September, by T. Massam of the group at CERN headed by A. Zichichi. In the first reported observation of the splitting in An, the CERN counter group measured the recoil neutron in the chargeexchange reaction When a dip appeared, they didn t 5 - look as carefully for a problem. a. So an insignificant fluctuation was a. o boosted into a completely wrong discovery. UJ CO 4-7I-- + p - * n + A at a beam momentum of 3. GeV/c. They saw a marked dip at the center of the Afl. Confidence levels for a single peak, incoherent double peak and dipole were 1%, 3% and 67% respectively. Lesson: Don t let result influence which data sets you use/want. Dependence of splitting MISSING MASS (GEV) Fits to the two-peak structure of data from the CERN missing-mass and boson spectrometer group for the A, The black curve is the fit for two coherent To arrive at some conclusions concerning the A splitting we will look for variables the effect may depend on. The dependence or independence might give a clue to the nature of the A. We will discuss the possible dependence of the A splitting on four quantities: bombarding energy, final state, production reaction and momentum transfer. The effect of symmetric splitting has

19 CHAUVENET S CRITERION Assume that your data points (xi; i=1,...,n) are normally distributed, i.e. p(x µ x, x) = 1 x e 1 ( x µx x ) Find the number of standard deviations by which a suspicious data point xsus differs from t sus = x sus µ x x Calculate probability for a legitimate measurement to deviate by that much from = µ x x = µ x P rob(outside t sus x )=1 erf t sus Calculate the expected number of data points as deviant as xsus N P rob(outside t sus x ) <.5 xsus may be rejected

20 CHAUVENET S CRITERION: EXAMPLE Student makes 1 measurements of length (in mm): 46, 48, 44, 38, 45, 47, 58, 44, 45, mm value looks suspicious, so we compute x = 45.8 x =5.1 Number of standard deviations and probability of xsus: t sus = =.4 P rob(outside.4 x )=.16 Thus, in 1 measurements he expects only 1*.16=.16 measurements as deviant as xsus, which is <.5. If he rejects xsus=58, the mean and standard deviation need to be recomputed: x = 44.4 x =.9

21 CHAUVENET S CRITERION: PROBLEMS Can not be used iteratively, i.e. recalculate mean and sigma with remaining data and throw out some more points Most experimental data have non-gaussian tails V -5V -4V -3V -V -1V Electrostatic Grain Potential

22 FINAL COMMENT ON REJECTING DATA Rejecting data is a last resort measure! You should try first: Take more data Ask what could have gone wrong Was there a problem with data recording or calibration? Document everything (including the rejected data) in your report! Never reject data points because you don t like the answer!

23 WEIGHTED AVERAGE From previous lecture: Data for the e/m example Entry V (V) e/m (1 11 C/kg) sys stat ( sys + stat) ±.1 ± ±.1 ± ±.9 ± ±.8 ± ±.9 ± ±.5 ± ±.7 ±.7 ± ±.6 ± ±.8 ± ±.7 ± ±.1 ± ±.9 ± ±.8 ± ±.4 ±.4

24 e/m (1 11 C/kg) E/M EXAMPLE Entry How to deal with different errors and what do they mean?

25 WEIGHTED AVERAGES If you measure the same thing twice and the errors are different, how do you combine the results? A proper averaging gives more weight to measurements with smaller uncertainties. The reported error on the average must reflect this.

26 WHICH IS THE BEST ESTIMATE FOR X IN THIS CASE? we again employ the maximum likelihood principle : p(x 1,x,..., x N ) e 1 x1 X i... e 1 xn X N now the chi square is = N i=1 x 1 which is minimum if X = N i=1 1 i x i N i=1 1 i i X = 1 w i N i=1 Weighted Average w i x i with weights w i = 1 i

27 e/m (1 11 C/kg) E/M EXAMPLE weighted average for e/m Entry