Sensitivity of oscillation experiments to the neutrino mass ordering

Transcription

1 Sensitivity of oscillation experiments to the neutrino mass ordering September 10, 2014, NOW2014

2 1 Our usual approach

3 1 Our usual approach 2 How much of an approximation?

4 1 Our usual approach 2 How much of an approximation? 3 Summary and conclusions

5 1 Our usual approach 2 How much of an approximation? 3 Summary and conclusions

6 Parameter estimation sensitivity Define the test statistic χ 2 [ ] χ 2 L(θ d) (θ) = 2 log sup θ L(θ d) Assume it is χ 2 distributed with n degrees of freedom Use the data set without statistical fluctuations (Asimov data) Quote result

7 The interpretation Probability to observe a non-zero θ 13 at 99.73% CL in T2K 2π χ 2 is asymptotically χ 2 (Wilks theorem) The Asimov data is representative Several requirements, not always fulfilled 3π/2 π π/ true sin 2 2θ 13 Schwetz, Phys.Lett. B648 (2007) 54 true δ CP < 20% 20%-50% 50%-90% 90%-99% > 99%

8 Not a nested hypothesis Mass ordering is not nested Wilks theorem not applicable Some different choices of test statistic ) 2 PDF( χ NH MC IH MC NH Norm. Approx. IH Norm. Approx. χ 2 = χ 2 NO min χ 2 T = χ 2 IO χ 2 NO T = χ 2 (θ) min χ 2 All have different distributions We concentrate on T T 0 = value for Asimov data 0.02 T N (T 0, 2 T 0 ) χ Qian, et al., Phys.Rev. D86 (2012)

9 1 Our usual approach 2 How much of an approximation? 3 Summary and conclusions

10 Back to basics Test hypothesis NO, alternative hypothesis IO Can we reject NO if IO is true? Critical region defined by T c Reject NO if T < T c Test significance (CL): 1 α = 1 P(T < T c NO) α: Probability to reject NO if NO is true Test power: 1 β = P(T < T c IO) β: Probability of failing to reject false NO if IO is true Both α and β are relevant

11 What is sensitivity? Other Sensitivity (median) What is the expected rejection of a false ordering? (Given a parameter set) What is the probability of ruling out a false hypothesis at xσ? At what CL is exactly one ordering ruled out?

12 Simple hypotheses 6 Distribution of test statistic independent of parameter values Straight forward Applicable to reactor experiments sensitivity (σ) median (2 sided) median (1 sided) crossing (2 sided) crossing (1 sided) T 0 MB, Coloma, Huber, Schwetz, JHEP 03(2014)028

13 Composite hypotheses 150 NOvA 100 NO rejected 50 at 95 CL 0 50 IO rejected at 95 CL T 150 LBNE 34kt Distributions have parameter dependence To reject = Rejecting all parameter sets (θ 13, m31 2, δ, etc) Distribution of T is parameter dependent IO rejected at 95 CL NO rejected at 95 CL T MB, Coloma, Huber, Schwetz, JHEP 03(2014)028 T c = min T c (θ) θ Extensive Monte Carlo simulations Approximation (must check validity) T (θ) = N (T 0 (θ), 2 T 0 (θ))

14 Accelerator experiments - results MC Β 0.5 NOvA MC Β 0.5 LBNE 10kt Sensitivity Σ Gaussian 1 sided Gaussian 2 sided Α Β Sensitivity Σ Gaussian 1 sided Gaussian 2 sided Α Β MB, Coloma, Huber, Schwetz, JHEP 03(2014)028 Green: 68 % of outcomes Yellow: 95 % of outcomes

15 Comparison of experiments - specific β 7 6 True NO LBNE 34kt 7 6 True IO LBNE 34kt Sensitivity for Β 0.5 Σ NOvA PINGU INO JUNO LBNE 10kt Sensitivity for Β 0.5 Σ NOvA PINGU INO JUNO LBNE 10kt Date MB, Coloma, Huber, Schwetz, JHEP 03(2014)028 Note: Bands have different meanings! Date

16 Comparison of experiments - specific α True NO LBNE 34kt True IO LBNE 34kt p 1 Β for 3Σ PINGU JUNO LBNE 10kt p 1 Β for 3Σ PINGU JUNO LBNE 10kt 0.2 INO 0.2 INO NOvA NOvA Date MB, Coloma, Huber, Schwetz, JHEP 03(2014)028 Note: Bands have different meanings! Date

17 How to interpret the median sensitivity It is representative for how well the experiment will do 50 % probability of not reaching it 50 % probability of doing better Not 50 % probability of being wrong Not the only relevant quantity, distribution matters (do Brazilian bands!) Personal preference: Quote the power 1 β for a target sensitivity

18 Testing both orderings Depending on the significance it may be possible to: Reject exactly one hypothesis Reject both hypotheses Not reject any hypothesis It is natural, the true hypotheses should be rejected with probability α rejection probability α 10 0 both rejected NO rejected IO rejected both accepted α critical value T c MB, Coloma, Huber, Schwetz, JHEP 03(2014)028

19 What about δ CP? λ(α) 10 χ 2 for 2 dof 9 8 δ CP = 0 7 δ CP = π χ 2 for 1 dof sin 2 2θ 13 Schwetz, Phys.Lett. B648 (2007) 54 The test statistic may be very different from a χ 2 distribution Checked with full Feldman Cousins approach Several devious details, caveat venditor! See talk by Enrique Fernandez Martinez

20 What about δ CP? Minakata, Nunokawa, Parke, PLB537 (2002) 249 The test statistic may be very different from a χ 2 distribution Checked with full Feldman Cousins approach Several devious details, caveat venditor! See talk by Enrique Fernandez Martinez

21 1 Our usual approach 2 How much of an approximation? 3 Summary and conclusions

22 Summary and conclusions Wilks theorem is not applicable to the neutrino mass ordering, the test statistic is not χ 2 distributed Regardless, T 0 is still a good approximation of the (median) sensitivity Frequentist (as well as Bayesian not discussed) methods are perfectly applicable Critical values will depend on the experiment and underlying assumptions May be relevant for δ, please stay for Enrique s talk

23 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods 4 Backup frequentist 5 Backup distributions and results 6 Backup Bayesian basics 7 Bayesian methods

24 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods At the end of the day For the simple hypotheses: Two Gaussians, H ± : N (±T 0, 2 T 0 ) For H +, typical (median) result is +T 0 +T 0 is T 0 ( T 0 ) = 2T 0 away from the expected H result 2T 0 2T 0 /(2 T 0 ) = T 0 See also: Vitelis, Read,

25 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods Interpolating parameters Introduce an interpolating continuous parameter α such that P(α = ±1, θ) = P(θ, NO/IO) see, e.g., Capozzi, Lisi, Marrone, arxiv: Wilks theorem now applies Put bounds on α If α = 1 ( 1) is allowed, NO (IO) is allowed α h Entries 1764 Mean x Mean y RMS x NH TRUE RMS y mee /10 [ev ] Capozzi, Lisi, Marrone, arxiv: Personal comment: α = 0 is not special, it being allowed a priori does not affect the rejection of NO/IO

26 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods CP violation sensitivity Distribution under CP conserving hypothesis: High precision: Slightly shifted to higher χ 2 Low precision: Shifted to lower χ 2 Median values: High precision: Do not change much Low precision: Shifted to lower χ 2 1 CDF Σ 2 Σ 3 Σ NOΝA LBNE Χ 2 T2HK MB,Coloma,Fernandez-Martinez, ESS

27 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods CP violation sensitivity Distribution under CP conserving hypothesis: High precision: Slightly shifted to higher χ 2 Low precision: Shifted to lower χ 2 Median values: High precision: Do not change much Low precision: Shifted to lower χ 2 Σ T2HK Sc Median ESS LBNE NOΝA S Asimov MB,Coloma,Fernandez-Martinez,

28 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods Octant sensitivity NuFIT 1.3 (2014) Degeneracies closer Wilk s theorem still violated A priori, a dedicated study is needed χ 2 χ Global w/o ATM Global with SK1 4 (old) Global with SK1 4 (new) Normal Ordering Inverted Ordering sin 2 θ 23 δ CP

29 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods LBNO predictions p = 1-β σ σ Test power for NH Exposure (/1e20 POT) LBNO collaboration, arxiv:

30 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods 4 Backup frequentist 5 Backup distributions and results 6 Backup Bayesian basics 7 Bayesian methods

31 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods Reactor experiments Distribution of T essentially parameter independent Simple hypotheses Gaussian limit well satisfied Median sensitivity nσ: n [ ( )] 2 erfc erfc T0 2 JUNO, 4320 kt GW yr, 3% E-resol. true IO 0.01 T c,no true NO 0.01 T c,io T MB, Coloma, Huber, Schwetz, JHEP 03(2014)028

32 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods Atmospheric experiments PINGU true IO true NO T MB, Coloma, Huber, Schwetz, JHEP 03(2014)028 Distribution of T mainly depends on θ 23 Gaussianity is still well satisfied for each θ 23 Analytic as function of T 0 (θ 23 ) Boils down to evaluating the Asimov data: T 0

33 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods Accelerator experiments - distributions NOνA Not always Gaussian! Typical for low-sensitivity experiments Need to perform Monte Carlo studies for accuracy Rejection power depends on the true parameters LBNE true IO true IO true NO true NO T MB, Coloma, Huber, Schwetz, JHEP 03(2014) T

34 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods 4 Backup frequentist 5 Backup distributions and results 6 Backup Bayesian basics 7 Bayesian methods

35 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods Bayesian basics Assign a degree of belief in each hypothesis P(H) Update the degree of belief depending on observations Bayes theorem P(A, B) = P(A; B)P(B) = P(B; A)P(A) P(A; B) = Take A = hypothesis H, B = data d P(B; A)P(A) P(B) P(H; d) = L H(d)P(H) P(d)

36 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods 4 Backup frequentist 5 Backup distributions and results 6 Backup Bayesian basics 7 Bayesian methods

37 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods Bayesian hypothesis testing Study the relative degrees of belief in two hypotheses P(H 1 ; d) P(H 2 ; d) = L H 1 (d) P(H 1 ) L H2 (d) P(H 2 ) Strength of the evidence for H 1 : [ ] P(H1 ; d) κ = 2 log P(H 2 ; d) Kass-Raftery scale: Strength of evidence for H κ Posterior odds Degree of belief Barely worth mentioning 0 to 2 ca 1 to 3 < 73.11% Positive 2 to 6 ca 3 to 20 > 73.11% Strong 6 to 10 ca 20 to 150 > 95.26% Very strong > > 99.33%

38 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods What can be said about the future? Can compute probability of obtaining evidence at least strength κ 0 for the true ordering P(κ 0 ) = P(κ > κ 0 ; H 1 )P(H 1 ) + P(κ < κ 0 ; H 2 )P(H 2 ) Typical choice P(H 1 ) = P(H 2 ) = 0.5 Takes into account information on oscillation parameters L H (d) = L H(θ) (d)π(θ)dθ Compactifies all of the available information to one number Easy to simulate through Monte Carlo methods Prior dependent

39 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods Example: NOνA CDF(T) T PDF(T) NOνA experiment GLoBES implementation Only δ and m 2 31 varying Flat and 10 % Gaussian priors, respectively T MB, JHEP 01(2014)139

40 Backup frequentist Backup distributions and results Backup Bayesian basics Bayesian methods Example: NOνA, results 1 CDF(K) False Positive p [%] Positive Strong Very strong K MB, JHEP 01(2014)139 Probability of obtaining evidence with at least strength K for the correct ordering