Template morphing in many dimensions, with several fit parameters

Transcription

1 Template morphing in many dimensions, with several fit parameters Stefan Gadatsch (U. Siegen) Supervisors: Max Baak (CERN), Andreas Hoecker (CERN), Wouter Verkerke (NIKHEF) December, 9 Statistics Forum Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 1

2 Outline 1 Introduction Applications of template morphing A basic example of our technique Building the morph p.d.f. Calculation of weights for one morph parameter Construction of morph p.d.f. Extending the morph to a second parameter Changing the basis 3 Proof-of-principle: Determination of msugra parameters Extraction of SUSY parameters Prediction of SUSY shapes Fitting one SUSY parameter Fitting two SUSY parameters 4 Comparison of different morphing techniques 5 Conclusion and Prospects Stefan Gadatsch (U. Siegen) Statistics Forum December, 9

3 Outline 1 Introduction Applications of template morphing A basic example of our technique Building the morph p.d.f. 3 Proof-of-principle: Determination of msugra parameters 4 Comparison of different morphing techniques 5 Conclusion and Prospects Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 3

4 Applications of template morphing W-boson mass measurements (LEP, Tevatron) Fitting for systematical uncertainties from shape distortions MCLimit calculator at Tevatron Eg. matching parameters of QCD background modelling Toy MC generation For fully generated Monte Carlo samples, only a limited set of model points generated Eg. Higgs masses, SUSY points, QCD parameters This proof-of-principle talk: implementation, validation and study of a new, simple morphing technique. Projection of g1 A RooPlot of "x" Projection of p.d.f.s x Input templates Prediction from morphing the surrounding shapes using RooMomentMorph.5 Events / ( 1 x.5 ) Histogram of hh x_alpha alpha hh x_alpha Entries 16 Mean x Mean y.4999 RMS x RMS y.887 x Observable Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 4

5 A basic example of our technique Main question Assuming that we only know data samples at different points, how does a distribution in between look like? Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 5

6 A basic example of our technique Main question Assuming that we only know data samples at different points, how does a distribution in between look like? Answer Simple morphing of the known shapes to describe new points. Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 5

7 A basic example of our technique Main question Assuming that we only know data samples at different points, how does a distribution in between look like? Answer Simple morphing of the known shapes to describe new points. The basic concept 1 Input templates at m = and m = 1. How does the shape look like for m = 1? Projection of p.d.f.s Input templates Observable Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 5

8 A basic example of our technique Main question Assuming that we only know data samples at different points, how does a distribution in between look like? Answer Simple morphing of the known shapes to describe new points. Projection of p.d.f.s Input templates Observable The basic concept 1 Input templates at m = and m = 1. How does the shape look like for m = 1? Projection of p.d.f.s Input templates Divided Take the half of each template Observable Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 5

9 A basic example of our technique Main question Assuming that we only know data samples at different points, how does a distribution in between look like? Answer Simple morphing of the known shapes to describe new points. Projection of p.d.f.s Input templates Projection of p.d.f.s Input templates Divided Observable Observable The basic concept 1 Input templates at m = and m = 1. How does the shape look like for m = 1? Take the half of each template. 3 Shift their means to the point m = 1. Projection of p.d.f.s Input templates Divided + Shifted Observable Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 5

10 A basic example of our technique Main question Assuming that we only know data samples at different points, how does a distribution in between look like? Answer Simple morphing of the known shapes to describe new points. Projection of p.d.f.s Input templates Projection of p.d.f.s Input templates Divided Observable Observable The basic concept 1 Input templates at m = and m = 1. How does the shape look like for m = 1? Take the half of each template. 3 Shift their means to the point m = 1. Projection of p.d.f.s Input templates Divided + Shifted + Added Observable Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 5

11 A basic example of our technique Main question Assuming that we only know data samples at different points, how does a distribution in between look like? Answer Simple morphing of the known shapes to describe new points. Projection of p.d.f.s Input templates Projection of p.d.f.s Input templates Divided Projection of p.d.f.s Input templates Divided + Shifted Observable Observable Observable The basic concept 1 Input templates at m = and m = 1. How does the shape look like for m = 1? Take the half of each template. 3 Shift their means to the point m = 1. 4 Stretch the shapes and add them. Projection of p.d.f.s Input templates Divided + Shifted + Stretched Observable Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 5

12 A basic example of our technique Main question Assuming that we only know data samples at different points, how does a distribution in between look like? Answer Simple morphing of the known shapes to describe new points. Projection of p.d.f.s Input templates Projection of p.d.f.s Input templates Divided Projection of p.d.f.s Input templates Divided + Shifted Projection of p.d.f.s Input templates Divided + Shifted + Stretched Observable Observable Observable Observable The basic concept 1 Input templates at m = and m = 1. How does the shape look like for m = 1? Take the half of each template. 3 Shift their means to the point m = 1. 4 Stretch the shapes and add them. Projection of p.d.f.s Input templates Divided + Shifted + Stretched + Added Observable Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 5

13 A basic example Mean and width shift linearly with m in this simple example. mean 3 5 width m m Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 6

14 A basic example Mean and width shift linearly with m in this simple example. mean 3 5 width m m What if we have several shapes to morph between? Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 6

15 Outline 1 Introduction Building the morph p.d.f. Calculation of weights for one morph parameter Construction of morph p.d.f. Extending the morph to a second parameter Changing the basis 3 Proof-of-principle: Determination of msugra parameters 4 Comparison of different morphing techniques 5 Conclusion and Prospects Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 7

16 Calculation of weights for one morph parameter Consider an arbitrary function f(x; m), depending on a parameter m and a set of observables x. Taylor expansion at the point m i f(x; m i ) = f(x; m ) + ( m i ) f (x; m ) 1! + ( m i ) f (x; m )! +... (1) with m i = m i m.... and as vectorial equation for several reference points f(x; m ) f(x; m ) 1 f(x; m 1 ) f = f(x; m ) = 1 ( m 1 ) ( m 1 ) f (x; m ) 1 ( m ) ( m ) 1! f (x; m ) = M f !. () Inverting this equation yields to f = M 1 f (3) Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 8

17 Calculation of weights for one morph parameter Calculate the value of f at a new point m n+1 : 1 ( m n+1 ) f(x; m n+1 ) = ( m n+1 ). T f(x; m ) M 1 f(x; m 1 ). f(x; m n ) (4) Fractions for shifting functions 1 ( m) c i = ( m). T n 1 M 1 ( = M 1 ) ji ( m)j where j= c i = 1 i (5) Note that c i (m j ) = δ ij for a point included in the input templates. Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 9

18 Construction and normalization of morph p.d.f. For each p.d.f. p i (x; m) also shift mean and rms for every observable in the set of p.d.f.s. Apply a linear transformation to the observables to get their values at the new point. Add the shifted p.d.f.s weighted with the fractions c i. Definition of morph p.d.f. p(x; m) = i c i (m) p i [a i (m) x + b i (m)] pi [a i (m) x + b i (m)]dx (6) Usually normalization time-consuming for arbitrary p.d.f.s which do not have an analytical integral, but owing to the construction of RooMomentMorph very fast. Calculation of p goes linearly with the number of templates. Easily extendable to two and more morph parameters. Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 1

19 Extending the morph to a second parameter Analogical to the 1D case. Here two dimensional Taylor expansion for an arbitrary function f(x; m 1,m ) at the point (m 1i,m i ): f(x; m 1i,m i ) = f(x; m 1,m ) + 1 [ ( m 1i ) f(x; m 1,m ) 1! m 1 ] + ( m i ) f(x; m 1,m ) + 1 m! [...] +... (7) Writing the expansions for n m = k reference points as vectorial equation yields the searched fractions. Searched fractions for the D fully non-linear setting c i = n m 1 j= ( M 1 ) ji ( m) j (8) Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 11

20 Changing the basis Write f(m) = i c i Ψ i in an arbitrary basis Ψ i. Fractions can be written in terms of this new basis f(m)ψ i (m)dm = c i (9) assuming Ψ i to be an orthonormal set. Substituting f(m) = i d i(m)f(m i ) in (9) yields f(m) [ ] f(m j ) Ψ i (m )dm Ψ j (m) j i m j (1) Fractions in a new basis Ψ i d i (m) = Ψ i (m )dm Ψ j (m) (11) i m j Not yet implemented, but think of using eg. Bernstein polynomials. Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 1

21 Outline 1 Introduction Building the morph p.d.f. 3 Proof-of-principle: Determination of msugra parameters Extraction of SUSY parameters Prediction of SUSY shapes Fitting one SUSY parameter Fitting two SUSY parameters 4 Comparison of different morphing techniques 5 Conclusion and Prospects Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 13

22 Extraction of SUSY parameters SUSY parameters have so far been extracted by studying (set of) kinematic edge measurements. Interpretation of measurements assumes specific SUSY particle decay chains. Inclusive measurements: shapes not studied for SUSY parameter constraints only cross-sections are used. Difficult to invert shape information into SUSY parameters Ideal problem for template morphing! Complementary to existing model parameter determination techniques. -1 Entries/4 GeV/ 1 fb [GeV] m 1/ ! / ndf 4.11 / 45 Prob.679 Endpoint ± Norm ±.563 Smearing.73 ± ATLAS m(ll) [GeV] $ # 1 LSP ~ g (. TeV) ~ g (1.5 TeV) ~ g (1. TeV) ~ g (.5 TeV) ATLAS 5! discovery MSUGRA tan " ~ q (.5 TeV) ~ q (1. TeV) $ + % 1 = 1 ~ q (1.5 TeV) (13 GeV) 4 jets lepton 4 jets 1 lepton 4 jets leptons OS 1 jet 3 leptons ~ q (. TeV) ~ q (.5 TeV) NO EWSB m [GeV] Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 14

23 Prediction of SUSY shapes Observable M eff, morph parameter m 1 Events / ( 4 ) Model for m 1/ = 57 m =114 Events / ( 4 ) Model for m 1/ = 69 m =114 Events / ( 4 ) Model for m 1/ = 75 m = M effective M effective M effective M eff = ET miss + 4 i=1 pjet,i T + p lep T Input templates: kernel p.d.f.s for m = 114 fixed, A = fixed, tan β = 1 fixed, sgn(µ) = +1 fixed, m 1 = 57,69,75 Events / ( 4 ) 6 m 1/ = 63 m = Predict shape for m 1 = 63 M effective Prediction from morphing the surrounding shapes using RooPolyMorph for m 1/ = 63 Input m = 57 1/ Input m = 69 1/ Input m = 75 1/ 3 Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 15

24 Prediction of SUSY shapes Observable M trans, morph parameter m 1 Events / ( 1 ) Model for m 1/ = 57 m =114 Events / ( 1 ) Model for m 1/ = 69 m =114 Events / ( 1 ) 6 Model for m 1/ = 75 5 m = TransMass TransMass TransMass M trans = p lep T Emiss T (1 cos( φ) Input templates: kernel p.d.f.s for m = 114 fixed, A = fixed, tan β = 1 fixed, sgn(µ) = +1 fixed, m 1 = 57,69,75 Events / ( 1 ) 1 m 1/ = 63 m = Predict shape for m 1 = 63 TransMass Prediction from morphing the surrounding shapes using RooPolyMorph for m 1/ = 63 Input m = 57 1/ Input m = 69 1/ Input m = 75 1/ 3 Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 16

25 Fitting one SUSY parameter Observable: M eff, fit parameter m 1 Input templates: kernel p.d.f.s for m = 114 fixed, A = fixed, tan β = 1 fixed, sgn(µ) = +1 fixed, m 1 = 57,69,75 Events / ( 4 ) Model for m 1/ = 63 m =114 Prediction from morphing the surrounding shapes using RooPolyMorph: = 67.1 ± 6. Fit data at m 1 = 63. M effective Result: m 1 = 67.1 ± 6.. m 1/ Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 17

26 Fitting one SUSY parameter Observable: M trans, fit parameter m 1 Input templates: kernel p.d.f.s for m = 114 fixed, A = fixed, tan β = 1 fixed, sgn(µ) = +1 fixed, m 1 = 57,69,75 Events / ( 1 ) Model for m 1/ = 63 m = Prediction from morphing the surrounding shapes using RooPolyMorph: = 648 ± 16 Fit data at m 1 = 63. TransMass Result: m 1 = 648 ± 16. M trans less sensitive to m 1. m 1/ 3 Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 18

27 Toy Monte Carlo studies Test of morph pdf fit functionality: Generate O(1) random signal Toy MC samples for specific m 1 values. 5 events per sample. No backgrounds considered. Fit each sample with the morph p.d.f. to extract m 1. Only shape information is considered, e.g. x-section information not yet used! Plot the pull distribution. Observable: M eff Events / (.4 ) pullmean = ±.3 pullsigma =.96 ± m1 Pull Observable: M trans Events / (.4 ) pullmean = -.1 ±.31 pullsigma =.986 ± m1 Pull Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 19

28 Fitting two SUSY parameters Toy MC study of fit stability from morph p.d.f. Observables: M eff and M trans. Fit parameters: m 1 and m Input templates: smoothed hist p.d.f.s for a 4 4 grid in (m,m 1 ) space. Generate toys at a point in this grid. Fit the toy data with morphed shape to extract both parameters simultaneously. True/generated values: m,true = 114 m 1,true = 63 Stefan Gadatsch (U. Siegen) Statistics Forum December, 9

29 Fitting two SUSY parameters Toy MC study of fit stability from morph p.d.f. m,true = 114, m 1,true = 63 Events Events m m1 Events Events m Error m1 Error Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 1

30 Fitting two SUSY parameters Toy MC study of fit stability from morph p.d.f. Errors of m and m 1 are often asymmetrical. Projection of nll m m1 error high Need to use Minos (asmymetrical left/right) errors! m1 error low Stefan Gadatsch (U. Siegen) Statistics Forum December, 9

31 Fitting two SUSY parameters Toy MC study of fit stability from morph p.d.f. m,true = 114, m 1,true = 63 m m Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 3

32 Fitting two SUSY parameters Toy MC study of fit stability from morph p.d.f. m,true = 114, m 1,true = 63 Events χ / ndf 17.4 / 91 Constant 1.1 ± Mean.1467 ±.139 Sigma 1.11 ± m Pull Events / ndf χ / 69 Constant 16.4 ± ± Mean.136 Sigma.996 ± m1 Pull Width are consistent with 1. Pull at level of less than %. Stable fit results! Reminder: no x-section information used! Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 4

33 A typical example Test of morph prediction from surrounding points. Morph p.d.f. uses 5 5 grid in (m,m 1 ) space. Example: remove m = 16 line in the grid and from morph p.d.f. Fit dataset of (m = 16,m 1 = 69) with morph p.d.f. Events / ( 5 ) FourMeff Events / ( ) m = 134 ± 6 m1 = ± TransMass Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 5

34 Fit results for two parameters For each line in table, corresponding line with same m -values removed from grid. Truth m m 1 Fit (linear) m m ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± 5.6 m 1 follows pretty well, m sometimes has problems. Note also less sensitivity to m. Note: also test of spacing-size of available grid-points. If grid not fine enough, morphing prediction is useless. Ambiguities in m (see backup slides for details). Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 6

35 Outline 1 Introduction Building the morph p.d.f. 3 Proof-of-principle: Determination of msugra parameters 4 Comparison of different morphing techniques 5 Conclusion and Prospects Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 7

36 Comparison of different morphing techniques Morphing is not an unique technique. Several ways to do. Each method has its advantages and disadvantages, so you cannot say which one is right or wrong! What else is on the market? Linear interpolation of histograms by A.L. Read. Implemented in RooFit as RooIntegralMorph. Projection of p.d.f.s Input templates Prediction from morphing the surrounding shapes using RooMomentMorph or A.L. Read's morph Observable Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 8

37 Comparison of different morphing techniques Baak/Gadatsch Read linear morphing non-linear morphing " " " % number of input templates unlimited number of observables unlimited 1 number of fit parameters extendable 1 computationally expense low high self-normalized " % not cont. diff. p.d.f.s problematic " numerical stability " % cut-offs "/ % %/ " Our technique is more easily extendable to higher dimensional problems. Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 9

38 Outline 1 Introduction Building the morph p.d.f. 3 Proof-of-principle: Determination of msugra parameters 4 Comparison of different morphing techniques 5 Conclusion and Prospects Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 3

39 Conclusion and Prospects Implemented and validated a new technique based on weighted template morphing. Very promising first results. Designed to make heavy use of latest RooFit caching technology (RooWorkspaces). Should be easy to extend morph up to > 1 templates, w/ several observables and fit parameters. Plan to integrate with other techniques developed by CERN-SUSY group, eg. TILES method. Plan to include x-section information in SUSY parameter fit. One fit parameter case available in ROOT 5.5/4, called RooMomentMorph. Two parameter fit available, working on n-dimensional fit parameter case. Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 31

40 Thank you for your attention! Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 3

41 Bibliography RADEMAKERS, FONS and BRUN, RENE: ROOT A Data Analysis Framework, VERKERKE, WOUTER and KIRKBY, DAVID: The RooFit toolkit for data modeling, READ, ALEXANDER L.: Linear interpolation of histograms, Nucl. Instrum. Meth. A45, (1999). THE ATLAS COLLABORATION: Expected Performance of the ATLAS Experiment - Detector, Trigger and Physics (9), [arxiv:hep-ex/91.51]. MARTIN, STEPHEN P.: A Supersymmetry Primer (1997), [arxiv:hep-ph/979356]. CRANMER, KYLE S.: Kernel estimation in high-energy physics, Comput. Phys. Commun. 136, (1), [arxiv:hep-ex/1157]. THE ATLAS COLLABORATION: Background Estimation for Inclusive SUSY Searches The Tiles Method (9) [ATL-PHYS-PUB ATL-COM-PHYS-9-5]. Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 33

42 Outline 6 Backup slides Calculation different settings Two morph parameters linear setting Two morph parameters fully non-linear setting Available Monte Carlo samples Ambiguity in fit solution Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 34

43 Calculation different settings Negative fractions for fully non-linear calculation possible Morphed p.d.f. can become negative Different settings for avoiding this: Linear fractions Non-linear fractions for shifting mean and rms, but linear fractions for shifting p.d.f.s Calculate the fractions non-linear, but take only the positive dn N 1 dm ATLAS RooPolyMorph, Linear RooPolyMorph, NonLinear RooPolyMorph, NonLinearPosFractions dn N 1 dm ATLAS RooPolyMorph, Linear RooPolyMorph, NonLinear RooPolyMorph, NonLinearPosFractions.1 RooPolyMorph, NonLinearLinFractions RooLinearMorph.1 RooPolyMorph, NonLinearLinFractions RooLinearMorph given p.d.f.s given p.d.f.s m ( GeV ) c^ m ( GeV ) c^ Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 35

44 Two morph parameters linear setting Take the smallest square, composed of four reference points, that includes the interesting point. Perform a linear extrapolation between these points similar to the basic example, but with respect to the mixing terms: with and c i = 3 (M 1 ) ji ( m) j (1) j= 1 M = 1 m 11 m 1 m 11 m 1 1 m 1 m m 1 m (13) 1 m 31 m 3 m 31 m 3 1 m = m 1 m (14) m 1 m where m ki = m ki m i and m i = m i m i. Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 36

45 Two morph parameters fully non-linear setting Searched fractions for the D fully non-linear setting c i = n m 1 j= ( M 1 ) ji ( m) j (15) with 1 1 ( m 1 ) ( m 1 ) m ( m 11 ) n ( m 1 ) m M = 1 ( m ) ( m ) m ( m 1 ) n ( m ) m ( m k ) ( m k ) m ( m 1k ) n ( m k ) m (16) 1 ( m l ) and m =. ( m l ) m, where m ik = m ik m i. (17). ( m l ) n ( m l ) m Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 37

46 Available Monte Carlo samples Available templates: [4 jets + 1 lepton] samples, msugra, AtlFast1 tan β = 1, sgn(µ) = +1, A = 5 5 raster, with m = 6 and m 1 = 1 Standard CSC selections applied for [4 jets + 1 lepton] Templates used: Construct the morph p.d.f. with line or grid of available templates, eg.: Line in m : (114,57), (114,63), (114,69), (114,75) (Corners of) Grid in (m,m 1 ): (9,57), (138,57), (9,81), (138,81) Tested upto grid size of 5 5 templates sofar. Template can have several observables, eg. used: (M eff ) or (M eff, M trans ) Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 38

47 Ambiguity in fit solution 3 3 grid in (m,m 1 ) space. Fit dataset at (m = 16,m 1 = 63), not included in the input templates. Fit can end up at a wrong point! Too little information to distinguish between these points. Can be solved by adding more information to the morph: Increase grid-size. Reduce distance between grid-points. Add another, more sensitive observable, eg. x-section. Events / ( 7. x 3.6 ) m m m m Stefan Gadatsch (U. Siegen) Statistics Forum December, 9 39