Likelihood Statistics

Transcription

1 Likelihood Statistics Anja Vest IEKP, Uni Karlsruhe October 25 Event weights Likelihood plots Confidence levels TLimit root class

2 Likelihood Ratio Experimental result = configuration of events that agrees with either a pure background (b) hypothesis signal + background (s+b) hypothesis Discriminator: e.g. M rec (peak in M rec distribution signal observation ) (also a 2D discriminator possible, e.g. M rec and b tag content of an event) Divide M rec into bins: i = 1, 2,..., N bins each containing N i observed candidates histogram Likelihood ratio: ranks an experimental result between being b like oder s+b like Anja Vest, IEKP, Uni Karlsruhe 1

3 Likelihood Ratio Likelihood ratio: Q = L(s + b) L(b) = P P oisson(data s + b) P P oisson (Data b) = exp( (s tot + b tot )) exp( b tot ) N bins i=1 ( s i + b i b i ) N i more simple: weighted sum of all observed events: 2lnQ(M H ) = 2s tot 2 N bins i=1 N i ln(1 + s i(m H ) b i ) Anja Vest, IEKP, Uni Karlsruhe 2

4 The likelihood probability density function Likelihood p.d.f.: histogram generated when performing a large number of experiments Generate for example 5 experiments without signal hypothesis (b-only) For each experiment: event configuration (N i ) and M H 2lnQ(M H ) p.d.f. Repeat the same for the signal + background hypothesis (s + b) Anja Vest, IEKP, Uni Karlsruhe 3

5 p.d.f. for a Higgs test mass of 115 GeV events/3 GeV Signal+Background Experiment 1 events/3 GeV Signal+Background Experiment 2 events/3 GeV Signal+Background Experiment events/3 GeV Reconstructed mass Signal+Background Experiment 4 events/3 GeV Reconstructed mass Signal+Background Experiment 5 p.d.f Reconstructed mass Test mass = 115 GeV Reconstructed mass Reconstructed mass s+b like lnQ b like Anja Vest, IEKP, Uni Karlsruhe 4

6 Likelihood p.d.f. s for diffeternt Higgs masses p.d.f. p.d.f. p.d.f Test mass = 112 GeV Test mass = 115 GeV -2lnQ Test mass = 118 GeV -2lnQ lnQ -2lnQ median likelihood of bg experiments likelihood of mh=115 median likelihood of s+b experiments Test mass Anja Vest, IEKP, Uni Karlsruhe 5

7 The likelihood probability density function -2 ln(q) LEP Observed Expected background Expected signal + background m H (GeV/c 2 ) Anja Vest, IEKP, Uni Karlsruhe 6

8 The likelihood probability density function Example: observed likelihood: 2lnQ = 3 Anja Vest, IEKP, Uni Karlsruhe 7

9 Confidence levels CL b : background confidence level, measures the compatibility of the experiment with the background 1 CL b : probability for a b only experiment to give a more s+b like likelihood than the observed one < 1 CL b >=.5 irrespective of the Higgs mass Correspondance between 1 CL b and the resulting significance (Gaussian approximation): 1 CL b Significance 1σ 2σ 3σ 4σ 5σ Anja Vest, IEKP, Uni Karlsruhe 8

10 The likelihood probability density function 1-CL b LEP Observed Expected for signal+background Expected for background m H (GeV/c 2 ) 2σ 3σ 4σ Anja Vest, IEKP, Uni Karlsruhe 9

11 Confidence levels CL s+b : measures the compatibility of the experiment with the s+b hypothesis A larger CL s+b means that the experimental result is more s+b like, but not neccessarily more s like CL s : If CL s+b is small (< 5%), the s+b hypothesis can be excluded at more than 95% confidence level, but it does not mean, that the signal hypothesis is excluded at that level. There is no way to directly measure the signal confidence level, because of the presence of significant background The signal confidence level is apriori defined to be: CL s = CL s+b CL b Anja Vest, IEKP, Uni Karlsruhe 1

12 The likelihood probability density function CL s LEP Observed Expected for background m H (GeV/c 2 ) Anja Vest, IEKP, Uni Karlsruhe 11

13 Confidence levels CL = 1 CL s (obviously conservative since the coverage probability is in general larger than the CL) Anja Vest, IEKP, Uni Karlsruhe 12

14 Using ROOT to calculate Significances and Limits TLimit is a ROOT add on. It computes limits using the Likelihood ratio method (originally implemented by Tom Junk in fortran 77) Classes: TLimitDataSource Takes the signal, background and data histograms to form a channel. More channels can be added using AddChannel(), as well as different systematic sources. TLimit Actual algorithm to compute 95% C.L. limits using the Likelihood ratio semibayesian method. It takes TLimitDataSource as input and runs a set of MC experiments in order to compute the limits. If needed, the inputs (s i and b i ) are fluctuated according to their systematics. The output is a TConfidenceLevel. TConfidenceLevel Final result of the TLimit algorithm. It is created just after the time consuming part and can be stored in a TFile for further processing. It contains light methods to return CLs, CLb and other interesting quantities, e.g. Get5sProbability(). Anja Vest, IEKP, Uni Karlsruhe 13

15 Using ROOT to calculate Significances and Limits TConfidenceLevel* ComputeLimit (TLimitDataSource* data, Int t nmc, TRandom* generator, Double t(*statistic) (Double t, Double t, Double t) stat) data: the input TLimitDataSource nmc: number of MC experiments to produce generator: MC generator used. Default: TRandom3 stat: function used as statistic. Default: TLimit::LogLikelihood Anja Vest, IEKP, Uni Karlsruhe 14

16 Using the root class TConfidenceLevel // Get the histograms TFile* infile=new TFile("Results.root","READ"); infile->cd(); TH1D* sh=(th1d*)infile->get("signal_histo"); TH1D* bh=(th1d*)infile->get("background_histo"); TH1D* dh=(th1d*)infile->get("data_histo"); // Compute the limits cout << "Computing limits... " << endl; TLimitDataSource* mydatasource = new TLimitDataSource(signal,background,data); TConfidenceLevel *myconfidence = TLimit::ComputeLimit(mydatasource,5); cout << "CLs : " << myconfidence->cls() << endl; cout << "CLsb : " << myconfidence->clsb() << endl; cout << "CLb : " << myconfidence->clb() << endl; cout << "< CLs > : " << myconfidence->getexpectedcls_b() << endl; cout << "< CLsb > : " << myconfidence->getexpectedclsb_b() << endl; cout << "< CLb > : " << myconfidence->getexpectedclb_b() << endl; cout << "3 sigma probability : " << myconfidence->get3sprobability() << endl; cout << "5 sigma probability : " << myconfidence->get5sprobability() << endl; myconfidence->draw(); Anja Vest, IEKP, Uni Karlsruhe 15

17 Example: mytest.c Signal and background compared to data... Using a set of randomly created histograms: Output: root [].L mytest.c root [1] limit() Computing limits... CLs :.263 CLsb : CLb : < CLs > : < CLsb > :.7688 < CLb > :.52 3 sigma probability : sigma probability : lnQ b_hist Entries 5 Mean RMS Anja Vest, IEKP, Uni Karlsruhe 16

18 Example: mytest.c Signal and background compared to data... Using a set of randomly created histograms: Output: root [].L mytest.c root [1] limit() Computing limits... CLs : CLsb : CLb : < CLs > : < CLsb > :.7688 < CLb > :.52 3 sigma probability : sigma probability : lnQ b_hist Entries 5 Mean RMS Anja Vest, IEKP, Uni Karlsruhe 17

19 Example: mytest.c Signal and background compared to data... Using a set of randomly created histograms: Output: root [].L mytest.c root [1] limit() Computing limits... CLs : e-1 CLsb : e-1 CLb : < CLs > : e-1 < CLsb > : e-1 < CLb > :.52 3 sigma probability : sigma probability : lnQ b_hist Entries 5 Mean RMS Anja Vest, IEKP, Uni Karlsruhe 18