Statistical Combination of ATLAS and CMS Higgs Searches

Transcription

1 Statistical Combination of ATLAS and CMS Higgs Searches Kyle Cranmer, ew York University for the ATLAS and CMS experiments 1

2 The Standard Model of 2

3 The Standard Model of L SM = 1 4 W µν W µν 1 4 B µνb µν 1 4 Ga µνg µν a }{{} kinetic energies and self-interactions of the gauge bosons + Lγ µ (i µ 1 2 gτ W µ 1 2 g YB µ )L + Rγ µ (i µ 1 2 g YB µ )R }{{} kinetic energies and electroweak interactions of fermions + 1 (i µ gτ W µ 1 2 g YB µ ) φ 2 V (φ) }{{} W ±,Z,γ,and Higgs masses and couplings + g ( qγ µ T a q) G a µ }{{} + (G 1 LφR + G2 Rφ Lφc R + h.c.) }{{ c L } interactions between quarks and gluons fermion masses and couplings to Higgs 2

4 The Success & Challenges of the Standard Model The standard model makes many predictions that are testable in very different experimental environments. on-trivial aspects of the theory have been tested to < 1 ppm 3

5

6 The ATLAS Experiment 5

7

8

9

10 Standard Model Higgs Properties in t, b H out "(pp! H+X) [pb] 1 pp! H (LO+LL QCD + LO EW) pp! qqh (LO QCD + LO EW) pp! WH (LO QCD + LO EW) pp! ZH (LO QCD +LO EW) s= 7 TeV LHC HIGGS XS WG 20 W, Z H -1 pp! tth (LO QCD) W, Z W, Z H t, b H t, b M H [GeV] The Higgs boson can be produced via different interactions. Production cross section σ depends on the unknown Higgs mass 8

11 Standard Model Higgs Properties in t, b H out Branching ratios "(pp! H+X) [pb] 1-1 1!! pp! H (LO+LL QCD + LO EW) pp! qqh (LO QCD + LO EW) pp! WH (LO QCD + LO EW) pp! ZH (LO QCD +LO EW) cc bb gg WW s= 7 TeV ZZ LHC HIGGS XS WG 20 LHC HIGGS XS WG 20 W, Z H W, Z W, Z H t, b H t, b pp! tth (LO QCD) " " Z" M H [GeV] M H [GeV] The Higgs boson then decays in one of several possible final states 3 The fraction of each decay mode also depends on the unknown Higgs mass 2 HC HIGGS XS WG 20 8

12 Triggering Higgs cross-section is ~ pb Total cross-section for protonproton collisions is ~0 mb (most interactions are not interesting) s/b ~ -! For each combination of production and decay, a search is performed. 9

13

14 Cross-sections and event rates From the many, many collision events, we impose some criteria to select n candidate signal events. We hypothesize that it is composed of some number of signal and background events. Pois(n s + b) The number of events that we expect from a given interaction process is given as a product of L : a time-integrated beam intensity (units 1/cm 2 ) that serves as a measure of the amount of data that we have collected or the number of trials we have had to produce signal events σ : cross-section (units cm 2 ) a quantity that can be calculated from theory ε : fraction of signal events selected by selection criteria The selection efficiency and the theoretical cross-section have experimental and theoretical systematic uncertainties and we parametrize them with nuisance parameters α s = L(α)σ(α) 11

15 Theoretical Predictions In addition to the rate of interactions, our theories predict the distributions of angles, energies, masses, etc. of particles produced we form functions of these called discriminating variables m, and use Monte Carlo techniques to estimate f(m) In addition to the hypothesized Higgs signal process, there are known background processes. thus, the distribution of f(m) is a mixture model the full model is a marked Poisson process signal process P (m s) = Pois(n s + b) n j background process sf s (m j )+bf b (m j ) s + b 12

16 Example model Here is an example prediction from search for H ZZ and H WW 6 3 Control of background rates from data sometimes multivariate techniques are used ] -1 Events [fb events / bin ATLAS Preliminary CMS (simulation) Preliminary 3 H " ll!! (m =300 GeV, s = 7 TeV) H 2 Signal, Signal m =130 GeV H W+Jets, Total tw BG di-boson tt tt ZZ Drell-Yan WZ WW Z W events / bin 4 CMS Preliminary =170 GeV 3 2 Signal, m H W+Jets, tw di-boson tt Drell-Yan m = eural Transverse etwork Mass Output [GeV] m = eural etwork Output Figure 5: outputs for signal (blue squares) and background (red circles) events for m H = 130 GeV (left) and m H = 170 GeV (right). Both distributions are normalized to 1 fb 1 of integrated luminosity. P (m s) = Pois(n s + b) is performed. The background measurement in the normalization region is used as a reference j to estimate the magnitude of the background in the signal region by multiplying the measured background events in the normalization PhyStat 2011, January, region ( 18, 2011 ) by the ratio of the efficiencies: 13 n sf s (m j )+bf b (m j ) s + b

17 Data-driven background determination Regions in the data with negligible signal expected are used as control samples simulated events are used to estimate extrapolation coefficients extrapolation coefficients have large theoretical and experimental uncertainties events / bin 4 CMS Preliminary e + e Signal, m =160 GeV H W+Jets, tw di-boson tt Drell-Yan Channel 1 H S.R. WW WW Top = α WW α Top S.R. WW C.R. WW = S.R. Top C.R. Top C.R.(WW ) WW Top W+jets C.R.(Top) Top β Top = C.R.(WW ) Top C.R.(Top) Top m ll [GeV/c ] W+jets α W +jets = S.R. W +jets C.R. W +jets C.R.(W +jets) W+jets β W +jets = C.R.(WW) W +jets C.R.(W +jets) W +jets Figure : Flow chart describing the four data samples used in the H WW ( ) lνlν analysis. S.R and C.R. stand for signal and control regions, respectively. 14

18 Data-driven background determination Regions in the data with negligible signal expected are used as control samples simulated events are used to estimate extrapolation coefficients extrapolation coefficients have large theoretical and experimental uncertainties events / bin 4 CMS Preliminary e + e αww Signal, m =160 GeV H W+Jets, tw di-boson tt Drell-Yan Channel H S.R. WW WW Top = α WW α Top S.R. WW C.R. WW = S.R. Top C.R. Top C.R.(WW ) WW Top W+jets C.R.(Top) Top β Top = C.R.(WW ) Top C.R.(Top) Top 1 S.R. C.R m ll [GeV/c ] W+jets α W +jets = S.R. W +jets C.R. W +jets C.R.(W +jets) W+jets β W +jets = C.R.(WW) W +jets C.R.(W +jets) W +jets Figure : Flow chart describing the four data samples used in the H WW ( ) lνlν analysis. S.R and C.R. stand for signal and control regions, respectively. 14

19 Constraints on uisance Parameters Many uncertainties have no clear statistical description or it is impractical to provide Traditionally, we use Gaussians, but for large uncertainties it is clearly a bad choice quickly falling tail, bad behavior near physical boundary, optimistic p-values,... For systematics constrained from control samples and dominated by statistical uncertainty, a Gamma distribution is a more natural choice [PDF is Poisson for the control sample] longer tail, good behavior near boundary, natural choice if auxiliary is based on counting For factor of 2 notions of uncertainty log-normal is a good choice can have a very long tail for large uncertainties one of them are as good as an actual model for the auxiliary measurement, if available To consistently switch between frequentist, Bayesian, and hybrid procedures, need to be clear about prior vs. likelihood function PDF Prior Posterior Gaussian uniform Gaussian Poisson uniform Gamma Log-normal reference Log-ormal Projection of gprior Truncated Gaussian Gamma Log-normal beta 15

20 Combinations within an experiment Each experiment combines multiple searches for the Higgs to improve power The Standard Model Higgs imposes relations between the different searches There is only one Higgs boson with unknown mass mh giving rise to a mild form of the look-elsewhere effect (LEE) There are well defined branching ratios for a given value of mh There is a common production cross-section σsm for a given value of mh though we often choose to consider µ = σ/σ SM as a parameter assuming the branching ratios are given by Standard Model In other theories, these relations are violated, exacerbating the LEE The different searches also suffer from common systematic uncertainties from detector performance, luminosity uncertainty, etc. the ability to incorporating these correlations imposes some constraints in the strategy employed by the individual searches 16

21 Combinations across experiments The theory imposes the same relationships among searches performed by different experiments (eg. ATLAS and CMS) uncertainties associated with detector performance are uncorrelated However, we use the same theoretical tools for predicting the rates and distributions associated with the signal and several backgrounds Even for data-driven approaches, we often rely on simulation for extrapolation requires coordination between experiments events / bin 4 CMS Preliminary S.R. Signal, m =160 GeV H W+Jets, tw di-boson tt Drell-Yan - e + e Channel α WW C.R m ll [GeV/c ] There is a long history of combining Higgs searches across experiments At LEP collider, combining four experiments (around 1999) At Tevatron, combining two experiments Combinations at the LHC pose new challenges -- toy exercise in 20 RooStats: a new tools to address these challenges [see talk by G. Schott, Wed.] 17

22 The 20 ATLAS+CMS Higgs Combination Exercise The exercise was based on toy data and models, though realistic in complexity An intense effort between in June 20, toy results shown July 6 Initial meetings were mainly focused on aligning language, philosophy, strategy, and priorities. discussion practical and technical issues Early on we decided the initial combination would be based on H WW+0j and that the analyses would be number counting in a few channels attempt to provide inputs in a technology neutral way as well as a RooStats workspace format early discussions on form of constraint terms (Gaussian, gamma, lognormal) later discussions on methods, test statistics, etc. Took ~1 month to prepare and validate inputs Four days from the time the inputs were shared to final results! Very impressive and encouraging exercise... but still an exercise. 18

23 Tables, Formulae, and Workspaces The ATLAS input: Poisson terms 3 signal regions and 6 control regions Uncertainties in extrapolation coefficients treated with truncated Gaussians and individual systematics on extrapolation coefficients were summed in quadrature thus, unable to identify any correlated systematic (eg. theory uncertainty) after discussions, decided to use this approach for initial exercise, but the need to evolve parametrization for real combination was recognized. L j,eµ Pois = P( j SR n s(sr) j + α j WW ν α j n j WW (CR) + α j ν WW tt α j n j (TB) + α j tt tt Wjets ν α j n j Wjets (LL) + Lσ j DY (SR)) Wjets lumiuncert lumiconstraint nominallumi gaus_ttc_0j P( j CR n s(cr) j + n j WW (CR) + β j ν tt β j n j (TB) + β j tt tt Wjets ν β j n j Wjets (LL) + Lσ j DY (CR)) Wjets gaus_wjc_0j gaus_wj_0j ctr_ww_em_0j RooProdPdf atlas_ww_0j gaus_tt_0j RooProdPdf nlc P( j TB n j tt (TB) + Lσ j Wjets (TB)) P( j LL n j Wjets (LL)) ctr_ww_ee_0j ctr_wj_ee_0j ctr_wj_em_0j gaus_ww_0j ctr_tt_em_0j ctr_ww_mm_0j sig_ee_0j_nlc sig_em_0j_nlc sig_mm_0j_nlc gaus_j_0j gaus_e_0j gaus_m_0j epsmean_m_0j RooConstVar 0.01 epsmean_e_0j RooConstVar epsmean_j_0j RooAddition sig_mm_0j_2_nlc obs_s_mm_0j obs_s_em_0j RooAddition sig_em_0j_2_nlc obs_s_ee_0j RooAddition sig_ee_0j_2_nlc obs_ww_mm_0j RooAddition ctr_ww_mm_0j_2 obs_tt_em_0j ctr_tt_em_0j_2 epsmean_ww_0j RooConstVar obs_wj_em_0j ctr_wj_em_0j_2 obs_wj_ee_0j ctr_wj_ee_0j_2 obs_ww_ee_0j RooAddition ctr_ww_ee_0j_2 RooConstVar 1.08 epsmean_tt_0j obs_ww_em_0j RooAddition ctr_ww_em_0j_2 epsmean_wj_0j RooConstVar 1 epsmean_wjc_0j RooConstVar 0.74 epsmean_ttc_0j sig_mm_0j_2_1_nlc sig_ee_0j_2_1_nlc sig_mm_0j_2_2 sig_em_0j_2_2 sig_ee_0j_2_2 ctr_ww_mm_0j_2_1 sig_mm_0j_2_3 sig_em_0j_2_3 sig_ee_0j_2_3 mc_tt_ctrl_tt mc_wj_ctrl_em_wj ctr_ww_ee_0j_2_1 ctr_ww_em_0j_2_1 ctr_ww_mm_0j_2_2 sig_em_0j_2_4 mc_wj_ctrl_ee_wj sig_ee_0j_2_4 ctr_ww_em_0j_2_2 ctr_ww_ee_0j_2_2 ctr_ww_ee_0j_2_3 ctr_ww_em_0j_2_3 n = µlσ SM mc_ww_ctrl_em_wj eps_wjc_0j RooConstVar 0 mc_ww_ctrl_ee_tt eps_ttc_0j mc_ww_ctrl_em_tt nu_wj_0j eps_wj_0j mc_sig_em_wj mc_ww_ctrl_mm_tt mc_ww_ctrl_em_ww mc_ww_ctrl_ee_ww nu_tt_0j mc_sig_ee_tt eps_tt_0j mc_sig_em_tt mc_sig_mm_tt mc_ww_ctrl_mm_ww nu_ww_0j mc_sig_ee_ww eps_ww_0j mc_sig_em_ww mc_sig_mm_ww eps_e_0j mc_sig_ee_sig sig_ee_0j_2_1_1_nlc mutimeslratio sig_em_0j_2_1_nlc mc_sig_em_sig eps_j_0j sig_mm_0j_2_1_1_nlc mc_sig_mm_sig eps_m_0j mu lumiratio 19

24 Tables, Formulae, and Workspaces The CMS input: cleanly tabulated effect on each background due to each source of systematic systematics broken down into uncorrelated subsets used lognormal distributions for all systematics, Poissons for observations Started with a txt input, defined a mathematical representation, and then prepared the RooStats workspace n i k ij ijk j 0,1,.. k b rs ij ijk k i i! j 0,1,.. k L exp n f( ) RooProdPdf model model_bin1 model_bin2 model_bin3 3 observables and 37 nuisance parameters yield_bkg_bin3 yield_tot_bin3 yield_bkg_bin2 nobs_bin3 nobs_bin2 yield_tot_bin2 nobs_bin1 yield_tot_bin1 yield_bkg_bin1 n = µlσ SM nexp_bin3_cat1 err1_bin3_cat1 err4_bin3_cat1 err26_bin3_cat1 yield_bin3_cat1 nexp_bin3_cat6 err26_bin3_cat6 err29_bin3_cat6 err31_bin3_cat6 err32_bin3_cat6 err33_bin3_cat6 err34_bin3_cat6 err35_bin3_cat6 err36_bin3_cat6 err37_bin3_cat6 err34_bin3_cat5 err35_bin3_cat5 x4 yield_bin3_cat6 nexp_bin3_cat5 err26_bin3_cat5 err29_bin3_cat5 err31_bin3_cat5 err32_bin3_cat5 err33_bin3_cat5 err36_bin3_cat5 err37_bin3_cat5 yield_bin3_cat5 yield_bin3_cat4 nexp_bin3_cat4 err12_bin3_cat4 x12 err15_bin3_cat4 x15 err26_bin3_cat4 err35_bin3_cat4 err36_bin3_cat4 nexp_bin3_cat3 err7_bin3_cat3 err8_bin3_cat3 err9_bin3_cat3 err24_bin3_cat3 yield_bin3_cat3 x24 err25_bin3_cat3 err35_bin3_cat3 nexp_bin3_cat2 err5_bin3_cat2 err23_bin3_cat2 x23 yield_bin3_cat2 err26_bin3_cat2 err28_bin3_cat2 err35_bin3_cat2 x1 err26_bin2_cat1 yield_bin2_cat1 nexp_bin2_cat1 err2_bin2_cat1 x2 err4_bin2_cat1 nexp_bin3_cat0 err22_bin3_cat0 x22 err25_bin3_cat0 err27_bin3_cat0 yield_bin3_cat0 err30_bin3_cat0 err31_bin3_cat0 err32_bin3_cat0 err33_bin3_cat0 err34_bin3_cat0 err35_bin3_cat0 err36_bin3_cat0 err37_bin3_cat0 x34 x32 err34_bin2_cat6 err35_bin2_cat6 nexp_bin2_cat6 err26_bin2_cat6 err37_bin2_cat6 yield_bin2_cat6 err32_bin2_cat6 err36_bin2_cat6 err29_bin2_cat6 x29 nexp_bin2_cat5 err26_bin2_cat5 err37_bin2_cat5 x26 x31 x37 yield_bin2_cat5 x33 err29_bin2_cat5 err32_bin2_cat5 err34_bin2_cat5 err35_bin2_cat5 err36_bin2_cat5 x36 nexp_bin2_cat4 err11_bin2_cat4 yield_bin2_cat4 x11 err14_bin2_cat4 x14 err26_bin2_cat4 err35_bin2_cat4 err36_bin2_cat4 x35 err7_bin2_cat3 err8_bin2_cat3 x9 nexp_bin2_cat3 yield_bin2_cat3 err35_bin2_cat3 x8 x7 err9_bin2_cat3 err21_bin2_cat3 x21 err25_bin2_cat3 yield_bin2_cat2 x6 err20_bin2_cat2 x20 err26_bin2_cat2 x25 nexp_bin2_cat2 err6_bin2_cat2 err35_bin2_cat2 x28 err28_bin2_cat2 x5 x27 yield_bin2_cat0 nexp_bin2_cat0 err19_bin2_cat0 err35_bin2_cat0 err36_bin2_cat0 err37_bin2_cat0 x30 err34_bin2_cat0 x19 err25_bin2_cat0 err27_bin2_cat0 err30_bin2_cat0 err32_bin2_cat0 yield_bin1_cat1 err3_bin1_cat1 x3 err26_bin1_cat1 nexp_bin1_cat1 err1_bin1_cat1 yield_bin1_cat6 nexp_bin1_cat6 err26_bin1_cat6 err29_bin1_cat6 err31_bin1_cat6 err33_bin1_cat6 err35_bin1_cat6 err36_bin1_cat6 err37_bin1_cat6 yield_bin1_cat5 err31_bin1_cat5 err33_bin1_cat5 err35_bin1_cat5 nexp_bin1_cat5 err26_bin1_cat5 err29_bin1_cat5 err36_bin1_cat5 err37_bin1_cat5 yield_bin1_cat4 nexp_bin1_cat4 err_bin1_cat4 x err13_bin1_cat4 x13 err26_bin1_cat4 err35_bin1_cat4 err36_bin1_cat4 yield_bin1_cat3 err18_bin1_cat3 x18 err25_bin1_cat3 err9_bin1_cat3 err8_bin1_cat3 nexp_bin1_cat3 err7_bin1_cat3 err35_bin1_cat3 yield_bin1_cat2 err17_bin1_cat2 x17 err26_bin1_cat2 nexp_bin1_cat2 err5_bin1_cat2 err35_bin1_cat2 err28_bin1_cat2 yield_bin1_cat0 err27_bin1_cat0 nexp_bin1_cat0 err16_bin1_cat0 x16 err25_bin1_cat0 err37_bin1_cat0 err36_bin1_cat0 err35_bin1_cat0 err33_bin1_cat0 err31_bin1_cat0 err30_bin1_cat0 rxsec 20

25 Visualization of the ATLAS+CMS Workspace 21 top level model parameter of interest µ = σbr σ SM BR SM RooProdPdf combmodel RooProdPdf atlas_ww_0j_atlas_combmodel RooProdPdf nlc_atlas_combmodel sig_mm_0j_nlc_atlas_combmodel RooAddition sig_mm_0j_2_nlc_atlas_combmodel sig_mm_0j_2_1_nlc_atlas_combmodel sig_mm_0j_2_1_1_nlc_atlas_combmodel mutimeslratio_atlas_combmodel lumiratio_atlas mu eps_m_0j_atlas eps_j_0j_atlas mc_sig_mm_sig_atlas sig_mm_0j_2_2_atlas nu_ww_0j_atlas eps_ww_0j_atlas mc_sig_mm_ww_atlas sig_mm_0j_2_3_atlas nu_tt_0j_atlas eps_tt_0j_atlas mc_sig_mm_tt_atlas obs_s_mm_0j_atlas sig_em_0j_nlc_atlas_combmodel RooAddition sig_em_0j_2_nlc_atlas_combmodel sig_em_0j_2_1_nlc_atlas_combmodel eps_e_0j_atlas mc_sig_em_sig_atlas sig_em_0j_2_2_atlas mc_sig_em_ww_atlas sig_em_0j_2_3_atlas mc_sig_em_tt_atlas sig_em_0j_2_4_atlas nu_wj_0j_atlas eps_wj_0j_atlas mc_sig_em_wj_atlas obs_s_em_0j_atlas sig_ee_0j_nlc_atlas_combmodel RooAddition sig_ee_0j_2_nlc_atlas_combmodel sig_ee_0j_2_1_nlc_atlas_combmodel sig_ee_0j_2_1_1_nlc_atlas_combmodel mc_sig_ee_sig_atlas sig_ee_0j_2_2_atlas mc_sig_ee_ww_atlas sig_ee_0j_2_3_atlas mc_sig_ee_tt_atlas sig_ee_0j_2_4_atlas RooConstVar 0 obs_s_ee_0j_atlas ctr_ww_mm_0j_atlas obs_ww_mm_0j_atlas RooAddition ctr_ww_mm_0j_2_atlas ctr_ww_mm_0j_2_1_atlas mc_ww_ctrl_mm_ww_atlas ctr_ww_mm_0j_2_2_atlas eps_ttc_0j_atlas mc_ww_ctrl_mm_tt_atlas ctr_wj_em_0j_atlas obs_wj_em_0j_atlas ctr_wj_em_0j_2_atlas mc_wj_ctrl_em_wj_atlas ctr_tt_em_0j_atlas obs_tt_em_0j_atlas ctr_tt_em_0j_2_atlas mc_tt_ctrl_tt_atlas ctr_ww_em_0j_atlas obs_ww_em_0j_atlas RooAddition ctr_ww_em_0j_2_atlas ctr_ww_em_0j_2_1_atlas mc_ww_ctrl_em_ww_atlas ctr_ww_em_0j_2_2_atlas mc_ww_ctrl_em_tt_atlas ctr_ww_em_0j_2_3_atlas eps_wjc_0j_atlas mc_ww_ctrl_em_wj_atlas ctr_wj_ee_0j_atlas obs_wj_ee_0j_atlas ctr_wj_ee_0j_2_atlas mc_wj_ctrl_ee_wj_atlas ctr_ww_ee_0j_atlas obs_ww_ee_0j_atlas RooAddition ctr_ww_ee_0j_2_atlas ctr_ww_ee_0j_2_1_atlas mc_ww_ctrl_ee_ww_atlas ctr_ww_ee_0j_2_2_atlas mc_ww_ctrl_ee_tt_atlas ctr_ww_ee_0j_2_3_atlas gaus_ww_0j_atlas RooConstVar epsmean_ww_0j_atlas gaus_wjc_0j_atlas epsmean_wjc_0j_atlas RooConstVar 1 gaus_tt_0j_atlas RooConstVar 1.08 epsmean_tt_0j_atlas gaus_ttc_0j_atlas RooConstVar 0.74 epsmean_ttc_0j_atlas gaus_wj_0j_atlas epsmean_wj_0j_atlas gaus_e_0j_atlas RooConstVar 0.01 epsmean_e_0j_atlas gaus_m_0j_atlas epsmean_m_0j_atlas gaus_j_0j_atlas RooConstVar epsmean_j_0j_atlas lumiconstraint_atlas nominallumi_atlas lumiuncert_atlas RooProdPdf modeluis_cms_combmodel RooProdPdf model_cms_combmodel model_bin1_cms_combmodel yield_tot_bin1_cms_combmodel yield_bin1_cat0_cms nexp_bin1_cat0_cms err16_bin1_cat0_cms x16_cms err25_bin1_cat0_cms x25_cms err27_bin1_cat0_cms x27_cms err30_bin1_cat0_cms x30_cms err31_bin1_cat0_cms x31_cms err33_bin1_cat0_cms x33_cms err35_bin1_cat0_cms x35_cms err36_bin1_cat0_cms x36_cms err37_bin1_cat0_cms x37_cms yield_bkg_bin1_cms yield_bin1_cat1_cms nexp_bin1_cat1_cms err1_bin1_cat1_cms x1_cms err3_bin1_cat1_cms x3_cms err26_bin1_cat1_cms x26_cms yield_bin1_cat2_cms nexp_bin1_cat2_cms err5_bin1_cat2_cms x5_cms err17_bin1_cat2_cms x17_cms err26_bin1_cat2_cms err28_bin1_cat2_cms x28_cms err35_bin1_cat2_cms yield_bin1_cat3_cms nexp_bin1_cat3_cms err7_bin1_cat3_cms x7_cms err8_bin1_cat3_cms x8_cms err9_bin1_cat3_cms x9_cms err18_bin1_cat3_cms x18_cms err25_bin1_cat3_cms err35_bin1_cat3_cms yield_bin1_cat4_cms nexp_bin1_cat4_cms err_bin1_cat4_cms x_cms err13_bin1_cat4_cms x13_cms err26_bin1_cat4_cms err35_bin1_cat4_cms err36_bin1_cat4_cms yield_bin1_cat5_cms nexp_bin1_cat5_cms err26_bin1_cat5_cms err29_bin1_cat5_cms x29_cms err31_bin1_cat5_cms err33_bin1_cat5_cms err35_bin1_cat5_cms err36_bin1_cat5_cms err37_bin1_cat5_cms yield_bin1_cat6_cms nexp_bin1_cat6_cms err26_bin1_cat6_cms err29_bin1_cat6_cms err31_bin1_cat6_cms err33_bin1_cat6_cms err35_bin1_cat6_cms err36_bin1_cat6_cms err37_bin1_cat6_cms nobs_bin1_cms model_bin2_cms_combmodel yield_tot_bin2_cms_combmodel yield_bin2_cat0_cms nexp_bin2_cat0_cms err19_bin2_cat0_cms x19_cms err25_bin2_cat0_cms err27_bin2_cat0_cms err30_bin2_cat0_cms err32_bin2_cat0_cms x32_cms err34_bin2_cat0_cms x34_cms err35_bin2_cat0_cms err36_bin2_cat0_cms err37_bin2_cat0_cms yield_bkg_bin2_cms yield_bin2_cat1_cms nexp_bin2_cat1_cms err2_bin2_cat1_cms x2_cms err4_bin2_cat1_cms x4_cms err26_bin2_cat1_cms yield_bin2_cat2_cms nexp_bin2_cat2_cms err6_bin2_cat2_cms x6_cms err20_bin2_cat2_cms x20_cms err26_bin2_cat2_cms err28_bin2_cat2_cms err35_bin2_cat2_cms yield_bin2_cat3_cms nexp_bin2_cat3_cms err7_bin2_cat3_cms err8_bin2_cat3_cms err9_bin2_cat3_cms err21_bin2_cat3_cms x21_cms err25_bin2_cat3_cms err35_bin2_cat3_cms yield_bin2_cat4_cms nexp_bin2_cat4_cms err11_bin2_cat4_cms x11_cms err14_bin2_cat4_cms x14_cms err26_bin2_cat4_cms err35_bin2_cat4_cms err36_bin2_cat4_cms yield_bin2_cat5_cms nexp_bin2_cat5_cms err26_bin2_cat5_cms err29_bin2_cat5_cms err32_bin2_cat5_cms err34_bin2_cat5_cms err35_bin2_cat5_cms err36_bin2_cat5_cms err37_bin2_cat5_cms yield_bin2_cat6_cms nexp_bin2_cat6_cms err26_bin2_cat6_cms err29_bin2_cat6_cms err32_bin2_cat6_cms err34_bin2_cat6_cms err35_bin2_cat6_cms err36_bin2_cat6_cms err37_bin2_cat6_cms nobs_bin2_cms model_bin3_cms_combmodel yield_tot_bin3_cms_combmodel yield_bin3_cat0_cms nexp_bin3_cat0_cms err22_bin3_cat0_cms x22_cms err25_bin3_cat0_cms err27_bin3_cat0_cms err30_bin3_cat0_cms err31_bin3_cat0_cms err32_bin3_cat0_cms err33_bin3_cat0_cms err34_bin3_cat0_cms err35_bin3_cat0_cms err36_bin3_cat0_cms err37_bin3_cat0_cms yield_bkg_bin3_cms yield_bin3_cat1_cms nexp_bin3_cat1_cms err1_bin3_cat1_cms err4_bin3_cat1_cms err26_bin3_cat1_cms yield_bin3_cat2_cms nexp_bin3_cat2_cms err5_bin3_cat2_cms err23_bin3_cat2_cms x23_cms err26_bin3_cat2_cms err28_bin3_cat2_cms err35_bin3_cat2_cms yield_bin3_cat3_cms nexp_bin3_cat3_cms err7_bin3_cat3_cms err8_bin3_cat3_cms err9_bin3_cat3_cms err24_bin3_cat3_cms x24_cms err25_bin3_cat3_cms err35_bin3_cat3_cms yield_bin3_cat4_cms nexp_bin3_cat4_cms err12_bin3_cat4_cms x12_cms err15_bin3_cat4_cms x15_cms err26_bin3_cat4_cms err35_bin3_cat4_cms err36_bin3_cat4_cms yield_bin3_cat5_cms nexp_bin3_cat5_cms err26_bin3_cat5_cms err29_bin3_cat5_cms err31_bin3_cat5_cms err32_bin3_cat5_cms err33_bin3_cat5_cms err34_bin3_cat5_cms err35_bin3_cat5_cms err36_bin3_cat5_cms err37_bin3_cat5_cms yield_bin3_cat6_cms nexp_bin3_cat6_cms err26_bin3_cat6_cms err29_bin3_cat6_cms err31_bin3_cat6_cms err32_bin3_cat6_cms err33_bin3_cat6_cms err34_bin3_cat6_cms err35_bin3_cat6_cms err36_bin3_cat6_cms err37_bin3_cat6_cms nobs_bin3_cms RooProdPdf nuisprior_cms pdf_x1_cms pdf_x2_cms pdf_x3_cms pdf_x4_cms pdf_x5_cms pdf_x6_cms pdf_x7_cms pdf_x8_cms pdf_x9_cms pdf_x_cms pdf_x11_cms pdf_x12_cms pdf_x13_cms pdf_x14_cms pdf_x15_cms pdf_x16_cms pdf_x17_cms pdf_x18_cms pdf_x19_cms pdf_x20_cms pdf_x21_cms pdf_x22_cms pdf_x23_cms pdf_x24_cms pdf_x25_cms pdf_x26_cms pdf_x27_cms pdf_x28_cms pdf_x29_cms pdf_x30_cms pdf_x31_cms pdf_x32_cms pdf_x33_cms pdf_x34_cms pdf_x35_cms pdf_x36_cms pdf_x37_cms ATLAS part CMS part The full model has 12 observables and 50 parameters At this point, no correlated systematics across experiments

26 Comparison of statistical methods RooStats supports several statistical methods used in high energy physics Common test statistics simple likelihood ratio (LEP) Q LEP = L s+b (µ = 1)/L b (µ = 0) ratio of profiled likelihoods (Tevatron) Q TEV = L s+b (µ =1, ˆν)/L b (µ =0, ˆν ) profile likelihood ratio (LHC) λ(µ) =L s+b (µ, ˆν)/L s+b (ˆµ, ˆν) Sampling strategies toy MC randomizing nuisance parameters according to π(ν) a Bayes-frequentist hybrid (prior-predictive) toy MC with nuisance parameters fixed (eyman Construction) assuming asymptotic distribution (Wilks and Wald) Bayesian (different priors for the parameter of interest) During the next four days, we tried to obtain results with as many of these methods as possible 22

27 Results of exercise Despite the complexity, we were able to go from inputs to results in 4 days! not only did we get results for the combination, we did it with six techniques a testament to the power and flexibility of the workspace technology and the RooFit/RooStats tools The results were based upon loosely representative toy models. The CMS results were more powerful, as they were using multivariate analyses and systematic uncertainties are not so extreme.!"#$%&'()*('*(+(%*(%,*'&%*($%#-(%./* A 0BC A 0BC! computing the p-value for significance in this approach is challenging: " speed improvements would be useful " or use importance sampling techniques! CMS distribution (and results previous slide) made with a RooFit-independent tool "#$%&#'()*+&,,(-(./0.)-12)(3,4,53,5*3(677,58%(-(9:;9!;<9:9(!./0.) 12),73,(3,4,53,5*3 35%85>5*48*7(?8&(3'3,;@ B 0CD B /CE λ?µ@ B 0CD B /CE λ?µ@ F;<G -!;==(H(9;9= %85>5*48*7(?A5,+(3'3,;@ F;9<(H(9;9: =;G(H(9;: - I;<<(H(9;9= J(I;! I;F(H(9;: B 0CD - J(I;! 1K2LM B - J(F; /CE - - λ?µ@ #!"#$%&$'(()*$+,-,./0$*)/'+./$1,.2$/3/.)-4.,5/$6)75)(.$,8$,9:,54.):$;.2)*1,/)< 12)(6&=7>(?5,+(@048=)@,7*+85>?7,73,(3,4, #?@7 346A@58% B0(./0.) B0(12) B0(1C2DE F7@G648-1&?3583(H8&( 3'3,;I link to agenda "#$%&#'()*+&,,(-(./0.)-12)(3,4,53,5*3(677,58%(-(9:;9<;=9:9( λhµi )JD,&'3 9;KL(M(9;9 - - O#&P5@7(0Q(HR5@S3I λhµi )JD 43'6A,&,5* 9;<L 9;=! 9;= F7@G648-1&?3583JJ λhµi )JD,&'3 9;<!(M(9;9 9;=K(M(9;9= 9;=T(M(9;9= U'V#5G W 0XO ),&'3 Y(9;K! 9;=L(M(9;9T( H048G)I U'V#5G W 0XO )JD,&'3 Y(9;K: - - D4' Z4[(P@4,(A#5&#(&8(# 2121\ 9;<= 9;T: 9;=! -! 23

28 Some lessons learned In general, this combination has been a great success in our first meeting we were already discussing correlated systematics between ATLAS and CMS We need to identify each of the backgrounds estimated from theory, because they are affected by luminosity uncertainty their theoretical uncertainties are correlated between experiments separate production modes: the qg, qq, and gg parts uncertainties in the parton density functions affect different processes in a different way, lumping them all together may be missing some essential physics. We need to separate and individually parametrize the effect of individual systematics the ability to correlate across experiments (and for different channels within the same experiment) requires the ability to relate parameters in the model in a consistent way consistent procedures are needed for assessing effect of common systematics Attempt to directly incorporate model for control samples when feasible superior to approximating by Gaussian, Gamma, etc. (though often not feasible) 24

29 ext Steps Since our toy exercise in July, ATLAS and CMS have formed an official LHC Higgs Combination Group kick-off meeting was in December first working meeting was last week focusing on validation of RooStats [link] The goal for the group is to show a combined ATLAS+CMS Higgs combination this summer -- with real data! Good luck to the LHC-HCG in 2011! 25

30 Backup 26

31 Essential Higgs Physics We know that the W, Z bosons are massive, but explicit mass terms for the W,Z break the electroweak gauge symmetry. massless W,Z only have transverse polarizations Higgs mechanism: φ Add, a new [complex doublet of] scalar field [s] with specific potential and V (φ) interactions with W,Z generates masses for W,Z Goldstone modes (become longitudinal polarizations of massive W,Z) V (φ) =µ 2 φ 2 + λ φ 4 vacuum expectation v fluctuations φ = 1 2 (v + h) Interactions with fermions: f φ f yfφf coupling arbitrary, but proportional to mass yv ff + 2 m f y 2 fhf interaction 27

32 28

33 Power-Constrained limits The ATLAS+CMS statistics committees are looking into a different way to avoid setting limits where we have no sensitivity (instead of CLs) idea: don t quote limit below some threshold defined by an -σ downward fluctuation of b-only pseudo-experiments 95% Expected cross-section Limit limit b-only expectation -2σ background fluctuation Observed limit is too lucky for comfort m H (GeV) 29

34 Incorporating systematics Let s consider a simplified problem that has been studied quite a bit to gain some insight into our more realistic and difficult problems number counting with background uncertainty in our main measurement we observe n on with s+b expected Pois(n on s + b) and the background has some uncertainty but what is background uncertainty? Where did it come from? maybe we would say background is known to % or that it has some pdf then we often do a smearing of the background: P (n on s) = db Pois(n on s + b) π(b), π(b) π(b) Where does come from? did you realize that this is a Bayesian procedure that depends on some prior assumption about what b is? 30

35 The on/off problem P (n on s) = db Pois(n on s + b) π(b), ow let s say that the background was estimated from some control region or sideband measurement. We can treat these two measurements simultaneously: main measurement: observe n on with s+b expected sideband measurement: observe n off with expected P (n on,n off s, b) joint model In this approach background uncertainty is a statistical error justification and accounting of background uncertainty is much more clear How does this relate to the smearing approach? τb = Pois(n on s + b) main measurement Pois(n off τb). sideband while π(b) is based on data, it still depends on a prior π(b) =P (b n off )= P (n off b)η(b) dbp (noff b)η(b). η(b) 31

36 Separating the prior from the objective model Recommendation: where possible, one should express uncertainty on a parameter as a statistical (random) process explicitly include terms that represent auxiliary measurements in the likelihood Recommendation: when using a Bayesian technique, one should explicitly express and separate the prior from the objective part of the probability density function Example: By writing P (n on,n off s, b) = Pois(n on s + b) Pois(n off τb). the objective statistical model is for the background uncertainty is clear One can then explicitly express a prior π(b) =P (b n off )= η(b) and obtain: P (n off b)η(b) dbp (noff b)η(b). 32

37 Hybrid Solutions Goal of Bayesian-frequentist hybrid solutions is to provide a frequentist treatment of the main measurement, while eliminating nuisance parameters (deal with systematics) with an intuitive Bayesian technique. P (n on s) = db Pois(n on s + b) π(b), p = P (n s) n=n obs Principled version (eg. ZΓ): clearly state prior η(b) ; identify control samples (sidebands) and use: π(b) =P (b n off )= P (n off b)η(b) dbp (noff b)η(b). Ad-hoc version (eg. Z): unable or unwilling to justify π(b), so go straight to some distribution eg. a Gaussian, truncated Gaussian, log normal, Gamma, etc... often the case for real systematic uncertainty (eg. MC generators, different background estimation techniques, etc.) Recommendation: Avoid ad hoc priors if possible. 33

38 Hypothesis Testing ow on a real PROOF cluster with 30 machines real world example throws millions of toys experiments, does full fit on 50 parameters for each toy. also supports producing simple shells scripts for use with GRID or batch queues ow importance sampling is also implemented, following presentation at Banff with particle physics & statistics experts allows for 00x speed increase! Still being tested in detail signalplusbackground background test statistic data 2-channel 3.35σ channel 4.4σ signalplusbackground background test statistic data Profile Likelihood Ratio Profile Likelihood Ratio 34